(Advanced Data structures and Algorithms (ADSA)

DYNAMIC PROGRAMMING

 Algorithms designed using dynamic programming are similar to those

developed using divide-and-conquer.
 Both solve a problem by breaking it down into several sub-problems that
can be solved recursively.
 The difference between the two is that in the dynamic programming
approach, the results obtained from solving smaller sub-problems are
reused in the calculation of large sub-problems.
 Thus dynamic programming is a bottom-up technique that usually begins
by solving the smaller sub-problems, saving these results, and then
reusing them to solve larger sub-problems until the solution to the original
problem is obtained.
 This is in contrast to the divide-and-Conquer approach, which solves
problems in a top-down method.

The 0/1 Knapsack problem :

We have a Knapsack (bag) with capacity, W = 5. the weights and values

of five items are given below.
item( 1 2 3 4
i)
wi 2 3 4 5
vi 4 8 9 11

The objective of knapsack problem is to fill the knapsack with items to

maximize the total value subject to its capacity. It is known as 0/1 knapsack
problem. Because we put one item into the knapsack or not (may not include a
fraction of an item).

To formulate the problem in terms of dynamic programming, we define a

sub-problem S[k,u], which will be the optimal solution for the first k terms,

SRET
-1-
(Advanced Data structures and Algorithms (ADSA)

involving up to u weight. S[k,u] is a recursive formula, which will specify how

the sub-problems can be solved. The real insight of dynamic programming is
that we will take this recursive solution and convert it into an iterative solution.

We will formulate this in terms of the solution that includes any of the
items 1…k-1. if we have the optimal solution for a given weight u for these k-1
items, then the optimal solution for the first k items will be described as follows:
there are two main cases:

(1) case: wk>u. as the weight of item k is greater than the weight
constraint that we are working with, item k cannot be
included in the knapsack.

(2) Case: wk<u. if item k is included, the value will be equal to the
value of item k plus the maximum value of k-1 previous items,
subject to a total weight of u-w k , since we have to include w k
without exceeding the weight limit u. this is equivalent to S[k-
1, u-wk] + vk. this leave us two sub-cases.

(a) if this total is less than the maximum for the first k-1 values,
we will not included item k. the new value will be S[k-
1,u].

(b) if it is larger, then we will include item k, and S[k-1,u-wk] + vk

will be the new value.

Thus, the recursive definition for the optimum is given as follows:

{
S[k-1,u] if wk > u
S[k,u] =
Max{S[k-1,u],S[k-1,u-wk]+vk} otherwise

SRET
-2-
(Advanced Data structures and Algorithms (ADSA)

We can store the sub-problems in a two-dimensional array, with first

subscript being the item that we choose from 1 to n, and the second being the
weight that we fill from 0 to W. for any given k, we can find all of the values of
S[k,u] for u=0 to W.

SRET
-3-
(Advanced Data structures and Algorithms (ADSA)

Algorithm :

Algorithm Knapsack(w[1…n],v[1…n],W)
{
For u=0 to W
S[0,u] = 0
End for
For i=0 to n
S[i,0] = 0
End for
For i=0 to n
For u=0 to W
If (w[i] < u) then
If(v[i]+S[i-1,u-w[i]]>s[i-1,u]) then
S[i,u] = v[i]+S[i-1,u-w[i]]
Else
S[i,u] = s[i-1,u]
Else
S[i,u] = s[i-1,u]
End for
End for
Print S[n][w]
i=n,k=W
while (i>0 and k>0)
if(S[i][k] != S[i-1][k])
print i
k = k – w[i]
end if
i=i-1
end while

SRET
-4-
(Advanced Data structures and Algorithms (ADSA)

}
Example : Let us trace this algorithm through the following example. Capacity of knapsack is 5.
item( 1 2 3 4
i)
wi 2 3 4 5
vi 4 8 9 11

After the initialization step of the first row and first column, the matrix S will be completed as
follows:
Weight(u)
Item(i 0 1 2 3 4 5
)
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0

For i=1, the computations for inner loop (u = 0,1,2,3,4,5) are as follows:
w[1] = 2, v[1] = 4.
u=0, w[1]>u, S[1,0] = S[0,0] = 0.
u=1, w[1]>u, S[1,1] = S[0,1] = 0.
u=2, w[1]=u and (v[1]+s[0,0])>S[0,2], S[1,2] = v[1] + S[0,0] =
4+0 = 4
u=3, w[1]=u and (v[1]+s[0,1])>S[0,3], S[1,3] = v[1] + S[0,1] =
4+0 = 4
u=4, w[1]<u and (v[1]+s[0,2])>S[0,4], S[1,4] = v[1] + S[0,2] =
4+0 = 4
u=5, w[1]<u and (v[1]+s[0,3])>S[0,5], S[1,5] = v[1] + S[0,3] =
4+0 = 4
Weight(u)
Item(i 0 1 2 3 4 5

SRET
-5-
(Advanced Data structures and Algorithms (ADSA)

)
0 0 0 0 0 0 0
1 0 0 4 4 4 4
2 0
3 0
4 0

For i=2, the computations for inner loop (u = 0,1,2,3,4,5) are as follows:
w[2] = 3, v[2] = 8.
u=0, w[2]>u, S[2,0] = S[1,0] = 0.
u=1, w[2]>u, S[2,1] = S[1,1] = 0.
u=2, w[2]>u, S[2,2] = S[1,2] = 4.
u=3, w[2]=u and (v[2]+s[1,0])>S[1,3], S[2,3] = v[2] + S[1,0] =
8+0 = 8
u=4, w[2]<u and (v[2]+s[1,1])>S[1,4], S[2,4] = v[2] + S[1,1] =
8+0 = 8
u=5, w[2]<u and (v[2]+s[1,2])>S[1,5], S[2,5] = v[2] + S[1,2] = 8+4 = 12
Weight(u)
Item(i 0 1 2 3 4 5
)
0 0 0 0 0 0 0
1 0 0 4 4 4 4
2 0 0 4 8 8 1
2
3 0
4 0

For i=3, the computations for inner loop (u = 0,1,2,3,4,5) are as follows:
w[3] = 4, v[3] = 9.
u=0, w[3]>u, S[3,0] = S[2,0] = 0.

SRET
-6-
(Advanced Data structures and Algorithms (ADSA)

u=1, w[3]>u, S[3,1] = S[2,1] = 0.

u=2, w[3]>u, S[3,2] = S[2,2] = 4.
u=3, w[3]>u, S[3,3] = S[2,3] = 8.
u=4, w[3]=u and (v[3]+S[2,0])>S[2,4], S[3,4] = v[3] + S[2,0] =
9+0 = 9
u=5, w[3]<u and (v[3]+S[2,1])<S[2,5], S[3,5] = S[2,5] = 12
Weight(u)
Item(i 0 1 2 3 4 5
)
0 0 0 0 0 0 0
1 0 0 4 4 4 4
2 0 0 4 8 8 1
2
3 0 0 4 8 9 1
2
4 0

For i=4, the computations for inner loop (u = 0,1,2,3,4,5) are as follows:
w[4] = 5, v[4] = 11.
u=0, w[4]>u, S[4,0] = S[3,0] = 0.
u=1, w[4]>u, S[4,1] = S[3,1] = 0.
u=2, w[4]>u, S[4,2] = S[3,2] = 4.
u=3, w[4]>u, S[4,3] = S[3,3] = 8.
u=4, w[4]>u, S[4,4] = S[3,4] = 9.
u=5, w[4]=u and (v[4]+S[3,0])<S[3,5], S[4,5] = S[3,5] = 12
Weight(u)
Item(i 0 1 2 3 4 5
)
0 0 0 0 0 0 0
1 0 0 4 4 4 4
2 0 0 4 8 8 1

SRET
-7-
(Advanced Data structures and Algorithms (ADSA)

2
3 0 0 4 8 9 1
2
4 0 0 4 8 9 1
2

We have essentially found the maximum value that can be places in the
knapsack for any weight u, using all of the items up to i. to find the solution for
the whole problem, we look at S[n,W] . in this case, the maximum value, S[4,5]
from the above table is 12.
To reconstruct the solution, we note that if S[i,k] ≠ S[i-1,k], this
happened because we added item I to the knapsack to increase the value.
Thus, we can backtrack for i=n, k=W, adding an item to the list every time we
see that S[i,k] ≠ S[i-1,k]

For our example, we start with i=4 and k = 5. since S[4,5] = S[3,5] =
S[2,5], we know that items 4 and 3 are not in the knapsack. How ever, when
i=2, S[2,5] ≠ S[1,5], so item 2 is in the knapsack, and we update k to be 3.
when i=1,S[1,3] ≠ S[0,3], so we add item 1, and we are done. The final
knapsack contains items 1 and 2, for total weight of 5 and a value of 12.

SRET
-8-
(Advanced Data structures and Algorithms (ADSA)

ALL - PAIRS SHORTEST PATHS :

Let G = (V, E) be a direction graph with n vertices. Let cost be a cost

adjacency matrix for G such that cost ( i , i ) = 0 , 1 < i < n . Then cost ( i, j ) is
the length ( or cost) of edge ( i , j ) if ( i j ) € E(G) and cost( i , j ) = ∞ if i ≠ j
and ( i , j ) ¢ E(G). The all-pairs shortest-path problem is to determine matrix A
such that A( i , j ) is the length of a shortest path from i to j . The matrix A can be
obtained by solving n single-source problem using the algorithm Shortest Path.

A( i ,j ) = min{ { Ak-1 ( i , k ) + Ak-1 ( k, j )}, cost( i, j )}

Cleary, A0 ( i , j )
= cost( i , j ) , 1 ≤ i ≤ n. We can obtain a recurrence for A k (i, j) using an
argument similar to that used before. A shortest path from i to j going through
no vertex higher than k either goes through vertex k or it does not. If it does, A k
k-1
( i, j ) = A ( i , k ) + Ak-1 ( k, j ). If it does not then no intermediate vertex has
k-1
index greater than k-1. Hence Ak (i, j) = A (i, j). Combining, we get

Ak( i ,j ) = min{ A ( i , k ) + Ak-1 ( k, j )} , k≥1

k-1 k-1
(i,j),A

Algorithm: function to compute lengths of shortest paths

Algorithm AllPaths (cost, A, n)

// cost [1: n, 1: n] is the cost adjacency matrix of a graph with n vertices; A [i, j]
// is the cost of a shortest path from vertex i to vertex j, cost [i, j] = 0.0, for 1≤i≤n.
{
for i: = 1 to n do
for j: = 1 to n do
A[i, j] := cost[i,j] ; // copy cost into A.
for k := 1 to n do
for i := 1 to n do
for j := 1 to n do

A [i, j]:= min(A[i, j],A[i, k] +A[k, j]);

}

Example : 6
The given graph 1
4
2 and has the A0 1 2 3 cost matrix
11 2 1 0 4 11
3
2 6 0 2
3 3 3 ∞ 0

SRET
-9-
(Advanced Data structures and Algorithms (ADSA)

The initial A matrix, A(0) , plus its values after 3 iterations A (1), A(2), A(3) are
calculated as,

For A(1) matrix, where k=1,

Ak( i ,j ) = min{ A ( i , k ) + Ak-1 ( k, j )}

k-1 k-1
(i,j),A // From the above matrix A0
min{ A ( 1 , 2 ) , A ( 1 , 1 ) + A0 ( 1, 2 )}
0 0
 A1( 1 ,2 ) =
= min{ 4, 0+4 } = 4
min{ A ( 1 , 3 ) , A ( 1 , 1 ) + A0 ( 1, 3 )}
0 0
 A1( 1 ,3 ) = A(1) 1 2 3
= min{ 11, 0+11 } = 11 1 0 4 11
min{ A ( 2 , 1 ) , A ( 2 , 1 ) + A ( 1, 1 )}
0 0
 1
A ( 2 ,1 ) = 0
2 6 0 2
= min{ 6, 6+0 } = 6
3 3 7 0
min{ A ( 2 , 3 ) , A ( 2 , 1 ) + A0 ( 1, 3 )}
0 0
 A1( 2 ,3 ) =
= min{ 2, 6+11 } = 2
min{ A ( 3 , 1 ) , A ( 3 , 1 ) + A0 ( 1, 1 )}
0 0
 A1( 3 ,1 ) =
= min{ 3, 3+0 } = 3
min{ A ( 3 , 2 ) , A ( 3 , 1 ) + A0 ( 1, 2 )}
0 0
 A1( 3 ,2 ) =
= min{ ∞, 3+4 } = 7

For A(2) matrix, where k=2,

Ak( i ,j ) = min{ A ( i , k ) + Ak-1 ( k, j )}

k-1 k-1
(i,j),A // From the above matrix A1
min{ A ( 1 , 2 ) , A ( 1 , 2 ) + A1 ( 2, 2 )}
1 1
 A2( 1 ,2 ) =
= min{ 4, 4+0 } = 4
min{ A ( 1 , 3 ) , A ( 1 , 2 ) + A1 ( 2, 3 )}
1 1
 A2( 1 ,3 ) = A(2) 1 2 3
= min{ 11, 4+2 } = 6 1 0 4 6
min{ A ( 2 , 1 ) , A ( 2 , 2 ) + A1 ( 2, 1 )}
1 1
 A2( 2 ,1 ) = 2 6 0 2
= min{ 6, 6+0 } = 6
3 3 7 0
min{ A ( 2 , 3 ) , A ( 2 , 2 ) + A ( 2, 3 )}
1 1
 2
A ( 2 ,3 ) = 1

= min{ 2, 0+2 } = 2
min{ A ( 3 , 1 ) , A ( 3 , 2 ) + A1 ( 2, 1 )}
1 1
 A2( 3 ,1 ) =
= min{ 3, 7+6 } = 3
min{ A ( 3 , 2 ) , A ( 3 , 2 ) + A1 ( 2, 2 )}
1 1
 A2( 3 ,2 ) =
= min{ 7, 7+0 } = 7
For A(3) matrix, where k=3,

SRET
- 10 -
(Advanced Data structures and Algorithms (ADSA)

Ak( i ,j ) = min{ A ( i , k ) + Ak-1 ( k, j )}

k-1 k-1
(i,j),A // From the above matrix A2
min{ A ( 1 , 2 ) , A ( 1 , 3 ) + A2 ( 3, 2 )}
2 2
 A3( 1 ,2 ) =
= min{ 4, 6+7 } = 4
min{ A ( 1 , 3 ) , A ( 1 , 3 ) + A2 ( 3, 3 )}
2 2
 A3( 1 ,3 ) =
= min{ 6, 6+0 } = 6
min{ A ( 2 , 1 ) , A ( 2 , 3 ) + A2 ( 3, 1 )}
2 2
 A3( 2 ,1 ) =
= min{ 6, 2+3 } = 5
A(3) 1 2 3
min{ A ( 2 , 3 ) , A ( 2 , 3 ) + A ( 3, 3 )}
2 2
 3
A ( 2 ,3 ) = 2
1 0 4 6
= min{ 2, 2+0 } = 2
2 5 0 2
min{ A ( 3 , 1 ) , A ( 3 , 3 ) + A2 ( 3, 1 )}
2 2
 A3( 3 ,1 ) =
= min{ 3, 0+3 } = 3 3 3 7 0

min{ A ( 3 , 2 ) , A ( 3 , 3 ) + A2 ( 3, 2 )}
2 2
 A3( 3 ,2 ) =
= min{ 7, 0+7 } = 7

Finally the values of A(3) table are the shortest paths between any pair of
vertices in the given graph.

OPTIMAL BINARY SEARCH TREES:

 Get a fixed set of identifiers.

 we wish to create a binary search tree organization.
 We may expect different binary search trees for the same identifier set to
have different performance characteristics.
 The following tree(a), in the worst case, requires four comparisons to find
an identifier,

(a) fo (b)
r fo
r
do whil
e do whil
e
int
if int

if

SRET
- 11 -
(Advanced Data structures and Algorithms (ADSA)

 whereas the above tree(b), which requires only three comparisons to find
an identifier.
 On the average the two trees need 12/5 and 11/5 comparisons,
respectively.
For example, in the case of tree(a), it takes 1,2,3, and 4 comparisons,
respectively, to find the identifiers for ,do, while , int and if. Thus the average
number of comparisons is (1+2+3+4)/5 = 12/5. This calculation assumes that
each identifier is searched for with equal probability and that no unsuccessful
searches ( i.e., searches for identifiers not in the tree) are made.
In general situations, we can expect different identifiers to be searched
for with different frequencies (or probabilities). In addition, we can expect
unsuccessful searches also to be made.
 Let us assume that the given set of identifiers is {a1, a2… an} with a1 < a2
< ….<an .
 Let p( i ) be the probability with which we search for a i.
 Let q(i) is probability of an unsuccessful search.
 Given this data, we wish to construct an optimal binary search tree for {a1,

a2,…, an}.
Algorithm :

Algorithm OBST(p,q,n)
{
For i:=0 to n-1 do
{
w[i,i] := q[i]; r[i,i] = 0; c[i,i] =0;
// Optimal tree with one node
w[i,i+1] := q[i] + q[i+1] + p[i+1];
r[i,i+1] := i+1
c[i,i+1] := q[i] + q[i+1] + p[i+1];
}
For m:=2 to n do // find optimal trees with m nodes

SRET
- 12 -
(Advanced Data structures and Algorithms (ADSA)

For i:=0 to n-m do

{
j := i+m;
w[i,j] := w[i,j-1] + p[j] +q[j];
k := find(c,r,i,j); // it returns the value to the k that
minimizes
//the c
c[i,j] := w[i,j] + c[i,k-1]+c[k,j];
r[i,j] := k;
}
}
Example:
Given problem,
Let n = 4 and (a1, a2, a3, a4 ) = ( do, if, int, while ).
Let p(1: 4) = (3, 3, 1, 1) and
q(0: 4) = (2, 3, 1, 1, 1).
Initially, we have the given constraints,
(1) w(i, i ) = q(i),
(2) c(i, i) = 0,
(3) r(i, i) = 0, 0≤ i ≤ 4.
(4) w(i, j ) = p(j) + q(j) + w(i,j-1),
(5) c(i, j) = w(i,j) + min {c[i,k-1]+c[k,j]},
i<k<j
(6) r(i, j) = is the value of k that minimizes
c(i,j).

from the above observations, we get

Solution :

Initially,
w(0,0) = 2 w(1,1) = 3 w(2,2) = 1 w(3,3) = 1 w(4,4) = 1

SRET
- 13 -
(Advanced Data structures and Algorithms (ADSA)

c(0,0) = 0 c(1,1) =0 c(2,2) =0 c(3,3) = 0

c(4,4) = 0
r(0,0) = 0 r(1,1) =0 r(2,2) =0 r(3,3) = 0
r(4,4) = 0

by using the above equation c(i,j) and the observation w(i,j), we get

=> w (0, 1) = p(1) + q (1) + w (0, 0) = 8

c (0, 1) = w (0, 1) + min{c(0, 0) + c(1, 1)} = 8
0<k<1 (for k=1)
r (0, 1) =1
=> w (1, 2) = p (2) + q(2) + w(1, 1) = 7
c (1, 2) = w(1, 2) + min{c(1, 1) + c(2, 2)} = 7
1<k<2 (for k=2)
r (1, 2) =2

=> w (2, 3) = p(3) + q(3) + w(2, 2) = 3

c (2, 3) = w(2, 3) + min{c(2, 2) + c(3, 3)} = 3
2<k<3 (for k=3)
r (2, 3) =3
=> w (3, 4) = p(4) + q(4) + w(3, 3) = 3
c (3, 4) = w(3, 4) + min{c(3, 3) + c(4, 4)} = 3
3<k<4 (for k=4)
r (3, 4) =4
knowing w(i,i+1) and c(i,i+1), 0< i < 4, we can again use the same equation
c(i,j) and the observation w(i,j) to compute w(i,i+2), c(i, i+2) and r(i, i+2) , 0≤ i
≤ 3.

=> w (0, 2) = p(2) + q (2) + w (0, 1) = 12

c (0, 2) = w (0, 2) + min{c(0, 0) + c(1, 2), c(0, 1) + c(2,
2)} = 8

SRET
- 14 -
(Advanced Data structures and Algorithms (ADSA)

0<k<2 ( for k=1) (for

k=2)
= 12 + min{ 0+7 , 8
+0 }
= 12 + 7 (7 is minimum for k=1)
= 19
r (0, 2) =1
=> w (1, 3) = p (3) + q(3) + w(1, 2) = 1+1+7 = 9
c (1, 3) = w(1, 3) + min{c(1, 1) + c(2, 3) , c(1, 2) + c(3,
3) } = 12
1<k<3 (for k=2) (for k=3)

r (1, 3) =2
=> w (2, 4) = p(4) + q(4) + w(2, 3) = 5
c (2, 4) = w(2, 4) + min{c(2, 2) + c(3, 4) , c(2, 3) + c(4,
4)} = 8
2<k<4 (for k=3) (for k=4)

r (2, 4) =3

This process can be repeat until w(0, 4), c(0,4), and r(0, 4) are obtained.

The table of Figure shows in below that shows the results of this
computation.
The box in row i and column j shows the values of w(j, j + i), c(j, j + i) and
r(j, j + i) respectively.
The computation is carried out by row from row 0 to row 4.
j
0 1 2 3 4
i w22 = w33 = w44 =
1 1 1
0 w00 = 2 w11 = 3
c22 = c33 = c44 =
c00 = 0 c11 = 0
0 0 0
r00 = 0 r11 = 0 r22 = r33 = r44 =
0 0 0
1 w01 = 8 w12 = 7 w23 = w34 =

SRET
- 15 -
(Advanced Data structures and Algorithms (ADSA)

3 3
c23 = c34 =
c01 = 8 c12 = 7
3 3
r01 = 1 r12 = 2 r23 = r34 =
3 4
w24 =
w02 = w13 = 9 5
12
c13 = c24 =
2 c02 =
12 8
19
r13 = 2 r24 =
r02 = 1
3
w03 = w14 =
14 11
3 c03 = c14 =
25 19
r03 = 2 r14 = 2
w04 =
19
4 c04 =
32
r04 = 2

From the table we see that c(0, 4) = 32 is the minimum cost of s binary
search tree for (a1, a2, a3, a4 ). The root of tree t04 is a2. Hence, the left sub tree
is t01 and the right sub tree t24. Tree t01 has root a1 and sub tree t00 and t11. Tree
t24 has root a3; its left sub tree is t 22 and its right sub tree t 34. Thus, with the data
in the table it is possible to reconstruct t 04. the following figure shows t04 .

if

do int

TRAVELING SALESPERSON PROBLEM whil

e
Let,
 G = (V,E) be a directed graph with edge costs cij.
 The variable cij is defined such that Cij>0 for all i and j and cij = ∞ if <i,j> ¢ E.

SRET
- 16 -
(Advanced Data structures and Algorithms (ADSA)

 let |V| = n and assume n>1.

 A tour of G is a directed simple cycle that includes every vertex in V.
 The cost of a tour is the sum of the cost of the edges on the tour.
The traveling sales person problem is to find a tour of minimum cost.

The traveling sales person problem finds application in a variety of situations.

Suppose,
 we have to route a postal van to pick up mail from mail boxes located
at n different sites.
 An n+1 vertex graph can be used to represent the situation.
 One vertex represents the post office from which the postal van starts
and to which it must return.
 Edge<i,j> is assigned to cost equal to the distance from site i to site j.
 the route taken by the postal van is a tour, and we are interested in
finding a tour of minimum length.
 For simplicity we consider that the tour starts and ends at vertex 1.
 Every tour consists of an edge <1, k> for some k € V-{1} and a path from
vertex k to vertex 1.
 The path from vertex k to vertex 1 goes through each vertex in V-{1, k}
exactly once.
 It is easy to see that if the tour is optimal, then the path from k to 1 must
be shortest k to 1 path going through all vertices in V-{1, k}.
 hence, the principle of optimality holds.
 Let g(i,S) be the length of a shortest path starting at vertex i. going
through all vertices in S, and terminating at vertex 1.
 The function g(1,V-{1}) is the length of an optimal sales person tour.
 From the principal of optimality it follows in
g(i,S) = min {Cij+g(j,.S-{j})}
general. j €
S
The g values can be obtained by using the above formula.
Clearly,

SRET
- 17 -
(Advanced Data structures and Algorithms (ADSA)

g(i,φ) = Ci1, hence we can use the above given formula to

obtain g(i,S) for all S of size 1. Then we can obtain g(i,S) for S with |
S| = 2, and so on.
When |S| < n-1, the values of i and S for which g(i,S) is needed are
such that i ≠ 1, 1¢S, and i¢S.
Example:
Consider the directed graph and the edge lengths are given by matrix are
as follows:
1 1 2
1 2 0
0 5 0
1
5 0 9
3 4 0
Thus,
1 1
6 0
g(2, φ) = C21 = 5 3 2

g(3, φ) = C31 = 6 8 8 9 0

g(4, φ) = C41 = 8
using the given equation we compute g(i,S) with |S| = 2, i ≠ 1, 1¢S, and i¢S,
g(2,{3}) = C23 + g(3, φ) = 15
g(2,{4}) = C24 + g(4, φ) = 18
g(3,{2}) = C32 + g(2, φ) = 18
g(3,{4}) = C34 + g(4, φ) = 20
g(4,{2}) = C42 + g(2, φ) = 13
g(4,{3}) = C43 + g(3, φ) = 15
next we compute g(i,S) with |S| = 2, i ≠ 1, 1¢S, and i¢S.
g(2,{3,4}) = min{ C23 + g(3,{4}) , C24 + g(4,{3}) } = 25
g(3,{2,4}) = min{ C32 + g(2,{4}) , C34 + g(4,{2}) } = 25
g(4,{2,3}) = min{ C42 + g(2,{3}) , C43 + g(3,{2}) } = 23
finally, we ontain,
g(1,{2,3,4}) = min{C12+g(2,{3,4}), C13+g(3,{2,4}), C14+g(4,{2,3})}
= min{ 35, 40, 43 }
= 35.
An optimal tour of the given graph has length “35”.

SRET
- 18 -
(Advanced Data structures and Algorithms (ADSA)

 A tour of this length can be constructed if we retain with each g(i,S) the
value of j that minimizes the right-hand side of the given equation.
 Let j(i,S) be this value. Then, j(1,{2,3,4}) = 2. thus the tour starts from
1 and goes to 2.
 The remaining tour can be obtained from g(2,{3,4}). So j(2,{3,4}) = 4.
thus the next edge is <2,4>.
 The remaining tour is for g(4,{3}). So, j(4,{3}) = 3.
 The optimal tour is 1, 2, 4, 3, 1.

Matrix Chain Multiplication :

Informal problem statement
Given n matrices M1,M2….Mn. what is the fastest way to compute the
product M1.M2….Mn
Issues
 What algorithm to use for multiplying just 2 matrices?
 Matrix multiplication is associative; how do we parenthesize the product?
Lets compute the first issue Multiplying 2 matrices,
How to compute C=AB where,
A:pxq matrix (given)
B:qxr (given)
C:pxr (to be computed)
The second issue concerns Associativity
Let me explain this with specific example involving only 3 matrices.
So For example say n=3.
Now we need to compute the product M1.M2.M3. (M1 times M2 times M3).
There are two ways to do this:
 One possibility is to multiply M2 and M3 first and then multiply that
with M1.
M1.(M2.M3)

SRET
- 19 -
(Advanced Data structures and Algorithms (ADSA)

 Another possibility is to multiply M1 and M2 first and then multiply it

with M3
(M1.M2).M3
Which is better M1.(M2.M3) or (M1.M2).M3?
Let us take a specific example.
Let us assume that M1 is the 3x2 matrix, M2 is the 2 x 4 matrix and M3 is
the 4x1 matrix.
As in the first case,
M1.(M2.M3)
Time[M4 = M2.M3] = 2x4x1 = 8
Time[Ans = M1.M4] = 3x2x1 = 6
Time[M1.(M2.M3)] = 8 + 6 = 14
As in the second case
(M1. M2).M3
Time[M5 = M1.M2] = 3x2x4 = 24
Time[Ans = M5.M3] = 3x4x1 = 12
Time[(M1.M2).M3] = 24 + 12 = 36.
So clearly this first way is better(faster).
The problem now is that we can take our product that we can parenthesize in
M1.(M2.M3) this way or we can parenthasise in (M1. M2).M3 this way.

Another Example:
• Suppose I want to compute A1.A2.A3.A4 .
• Matrix Multiplication is associative, so we can do the multiplication in several
different orders.
• A1 is 10 by 100 matrix
• A2 is 100 by 5 matrix
• A3 is 5 by 50 matrix
• A4 is 50 by 1 matrix
• A1A2A3A4 is a 10 by 1 matrix

SRET
- 20 -
(Advanced Data structures and Algorithms (ADSA)

5 different orderings = 5 different parenthesizations

• (A1(A2(A3A4)))
• ((A1A2)(A3A4))
• (((A1A2)A3)A4)
• ((A1(A2A3))A4)
• (A1((A2A3)A4))
Each parenthesization is a different number of mults
Let Aij = Ai · · ·Aj
• A1 is 10 by 100 matrix, A2 is 100 by 5 matrix, A3 is 5 by 50 matrix, A4 is 50
by 1 matrix, A1A2A3A4 is a 10 by 1 matrix.
• (A1(A2(A3A4)))
– A34 = A3A4 , 250 mults, result is 5 by 1
– A24 = A2A34 , 500 mults, result is 100 by 1
– A14 = A1A24 , 1000 mults, result is 10 by 1
– Total is 1750
• ((A1A2)(A3A4))
– A12 = A1A2 , 5000 mults, result is 10 by 5
– A34 = A3A4 , 250 mults, result is 5 by 1
– A14 = A12A34) , 50 mults, result is 10 by 1
– Total is 5300
• (((A1A2)A3)A4)
– A12 = A1A2 , 5000 mults, result is 10 by 5
– A13 = A12A3 , 2500 mults, result is 10 by 50
– A14 = A13A4 , 500 mults, results is 10 by 1
– Total is 8000
• ((A1(A2A3))A4)
– A23 = A2A3 , 25000 mults, result is 100 by 50
– A13 = A1A23 , 50000 mults, result is 10 by 50
– A14 = A13A4 , 500 mults, results is 10 by
– Total is 75500
• (A1((A2A3)A4))
– A23 = A2A3 , 25000 mults, result is 100 by 50
– A24 = A23A4 , 5000 mults, result is 100 by 1
– A14 = A1A24 , 1000 mults, result is 10 by 1
– Total is 31000
Conclusion Order of operations makes a huge difference. How do we compute
the minimum?

One Approach is :

SRET
- 21 -
(Advanced Data structures and Algorithms (ADSA)

Each parenthesization defines a set of n-1 matrix multiplications. We

justneed to pick the parenthesization that corresponds to the best ordering.
Now the question is how many parenthesizations are there?
Let P(n) ne the number of ways to parenthesize n matrices.

∑ n-1 P(k) P(n-k)

K=1
if n>2

{
P(n) =
1 if n = 1

This recurrence is related to the Catalan numbers, and solves to P(n) =

Ω(4n/n3/2)
The final conclusion is” trying all possible parenthesizations is a bad
idea”. Inorder to overcome that we use Dynamic Programming. The
steps include are,

1. Characterize the structure of an optimal solution

2. Recursively define the value of an optimal solution
3. Compute the value of an optimal solution bottom-up
4. Construct an optimal solution from the computed information

Structure of an optimal solution If the outermost parenthesization is

((A1A2 · · ·Ai)(Ai+1 · · ·An))
then the optimal solution consists of solving A1i and Ai+1,n optimally and
then combining the solutions.

SRET
- 22 -
(Advanced Data structures and Algorithms (ADSA)

Algorithm :
Matrix-Chain-Order(p)
{
n = length[p] -1
for i=1 to n
do m[i,i] = 0
for l=2 to n
do for i=1 to n-l+1
do j = i+l-1
m[i,j] = ∞
for k=i to j-1
do q = m[i,k] + m[k+1,j] + pi-1pkpj
if q< m[i,j]

SRET
- 23 -
(Advanced Data structures and Algorithms (ADSA)

then m[i,j] = q
s[i,j] = k
return m and s
}

BACKTRACKING:

N-QUEENS PROBLEM

The n-queens problem consists of placing n queens on an n-by-n

chessboard so that no two queens attack each other by being in the same row
or in the same column or on the same diagonal. This is a classical
combinatorial problem. Figure 6.3 illustrates n-queens problem for n=1,2,3.

1 2
1 1 1 2 3
1 Q 2 1
n=1 n=2 2
3
n=3
Trivial Solution No Solution No Solution

Let us consider the 4-queens problem and solve it by the backtracking

technique. Each queen has to be placed in its own row; all we need to do
assign a column for each queen on the following board

1 2 3 4
1 <- Queen
1
2 <- Queen
2
3 <- Queen
3
4 <- Queen
4

The 4 by 4 board(the following state space tree) shows a solution for n=4.

SRET
- 24 -
 (Advanced Data structures and Algorithms (ADSA)

All nodes(boards) are numbered from 0 to 16. The numbers within

parentheses indicate columns of the board. For example in board # 4, three
queens are placed in columns 1,4 and 2. Let us place 4 th queen in column 3. If
we place 4th queen in wither row 1, 2, or 3, this will attack other three
queens(already placed queens) row-wise. Also, 4 th row does not permit,
because it arracks diagonally queen at (3,1). Therefore, there is no solution in
this branch of tree (nodes: 1, 3, 4); it backtracks to node 5 and search for
alternate solution. Now, we have two solutions: board#8 and 12.

(Φ)
1 5 9 1
3
Q Q Q Q

(1) (2) (3) (4)

2 3 6 1 1 1
0 4 5
QQ QQ QQ QQ QQ QQ

(1,3) (1,4) (2,4) (3,1) (4,1) (4,2)

4 7 1 1
1 6
QQQ QQQ QQQ QQQ

(1,4,2) (2,4,1) (3,1,4) (4,1,3)

8 1
2
QQQQ QQQQ

A complete backtrack search tree for 4-Queens problem

(2,4,1,3) (3,1,4,2)
SRET
- 25 -
(Advanced Data structures and Algorithms (ADSA)

We will introduce backtracking algorithm which can be used to construct all

solutions for a given n.
 We could continue with the 4-queens example to obtain all possible
solutions for n=4.
 Our goal now is to convert this approach into an explicit algorithm.
 We track the position of the queens by using an array row.
 Row[k] is the row in column k containing a queen.
 The main algorithm is the recursive algorithm rn_queens().
 When rn_queens(k,n) is called, queens have been successfully placed in
columns 1 to k-1.
 The algorithm then attempts to place a queen in column k.
 If it is successful, then
- if k=n, it prints the solution.
- if k<n, it makes the call rn_queens(k+1,n).
- it then returns to the call rn_queens(k-1,n).
 To test for a valid position for a queen in column k, we use an algorithm
position_ok(k,n) which returns true if and only if the queen tentatively
placed in column k does not conflict with the queens in positions 1 to k-1.
 The queens in columns I and k conflict if
- they are in the same row: row[i]=row[k].
- or in the same diagonal: absolute(row[k]-row[i])=k-i.
 The algorithm is invoked with rn_queens(1,n).

Algorithm 6.3(a):rn_queens(k,n)

1. for row[k]=1 to n
2. if position_ok(k,n)=true, then
3. if k=n, then
4. print solution
5. else
6. rn_queens(k+1,n)
7. endif
8. endif
9. endfor.

Algorithm 6.3(b):position_ok(k,n)

1. for i=1 to k-1

2. if(row[k]=row[i] or abs(row[k]-row[i])=k-i), then
3. return false
4. else
5. return true
6. endif
7. endfor.

SRET
- 26 -
(Advanced Data structures and Algorithms (ADSA)

Time Complexity:

We will obtain an upper bound for the running time of the algorithm by
bounding the number of times rn_queens(k,n) is called for each k<n. There
are(n-1)…(n-k+2) ways to place queens in the first k-1 columns in distinct rows.
Ignoring recursive calls, rn_queens(k,n) executes in Ө(n) time for k<n, there is
at most one placement possible for the queen. Also, the loop in rn_queens
executes in Ө(n) time. There are n(n-1)…2 ways for the queens jto have been
placed in the first n-1 columns, so the worst-case running time for
rn_queens(n,m) is:

n[n(n-1)…2]=nxn!
The algorithm runs in O(nxn!).
SUM OF SUBSETS PROBLEM:
The sum of subsets problem consists of finding a subset of a given set X
= {x1,x2…xn} of n distinct positive integers and a positive integer S. find all
subsets of {x1,x2…xn} that sum to S.
For Example,
If X = {1,2,3,4,5,6} and S= 12, there will be more than one subset whose
sum is 12. the subsets are {1,2,3,6}, {1,2,4,5}, {1,5,6}, {2,4,6}, {3,4,5}.
 We will assume a binary state space tree.
 The nodes at depth 1 are for including (yes = 1, no = 0) item 1, the nodes
at depth 2 are for item 2, etc. The left branch includes x i, and the right
branch excludes xi. The nodes contain the sum of the numbers included
so far.
 Backtracking consists of doing a DFS of the state space tree, checking
whether each node is promising and if the node is non-promising
backtracking to the node’s parent.
 We call a node non-promising if it cannot lead to a feasible (or optimal)
solution, otherwise it is promising.
 The state space tree consisting of expanded nodes only is called the
pruned state space tree. The next coming Example shows the pruned
state space tree for the sum of subsets problem.

SRET
- 27 -
(Advanced Data structures and Algorithms (ADSA)

Consider a node at depth I,

sumSoFar = sum of node –that is, sum of numbers included in partial
solution node.
sumLeft = sum of the remaining items i+1 to n(for a node at depth i).

A node at depth I is non-promising:

If ((sumSoFar + sumLeft < S) or (sumSoFar + x[i+1] > S))

To be able to use this “promising function” the x i must be sorted in increasing

order.
The include[1..n] is a Boolean array. If its value is true, then the node is
included in the partial tree, else not included.

The following Algorithm: sumOfSubsets(i, sumSoFar, sumLeft) is invoked with:

i = 0, sumSoFar = 0, and sumLeft = x1+x2+…..+xn.
The algorithm is called with sumOfSubsets(0, 0, 21) for our next coming
Example.

Algorithm: sumOfSubsets(i, sumSoFar, sumLeft)

1. If promising(i, sumSoFar, sumLeft), then // may lead to
solution
2. If(sumSoFar, = S), then
3. print solution
4. Else //expand the node when sumSoFar < S
5. include[i+1] = true //include x[i+1]
6. sumOfSubsets(i+1, sumSoFar + x[i+1], sumLeft – x[i-1])
7. include[i+1] = false //exclude x[i+1]
8. sumOfSubsets(i+1, sumSoFar , sumLeft – x[i+1])
9. Endif
10. Endif.

Algorithm: promising (i, sumSoFar, sumLeft)

1. If(sumSoFar + sumLeft ≥ S) and
2. (sumSoFar = S or sumSoFar + x[i+1] ≤ S), then
3. Return true
4. Else
5. Return false
6. Endif.

SRET
- 28 -
(Advanced Data structures and Algorithms (ADSA)

Example: X = {3,5,6,7} and S = 15. There are only 15 nodes in the pruned
state space tree
In the following fig. The full state space tree has 31 nodes (2 4+1 – 1). Trace of
the Algorithm is given in Table , next to the state space tree. Nodes of the tree
are shown by circled numbers. Notice that the x 1 is included at level 1 of the
tree, x2 at level 2, and son on. We have only one solution with subset {3,5,7}.
This occurs at node 6 in the following Diagram.

1
0
3 0
2 1
1
3 0
5 0 5 0
3 1 1 1
2 5
8 3 5 0
6 0
6 0 6 0
4 5 9
1 11 1 5 1
14 8 9 0 3 3 4

7 0
6 7
Pruned State Space Tree - Example
15 8

SRET
- 29 -
(Advanced Data structures and Algorithms (ADSA)

Trace of sumOfSubsets – Example

nod sumSoF sumLef
i Comment
e ar t
0 1 0 21 Start
1 2 3 18 Include 3
2 3 8 13 Include 5
3 4 14 7 Include 6
3 5 8 7 Exclude 6
4 6 15 0 Include 7
3 5 <-
7 Solution
4 7 8 0 Exclude 7
2 8 3 13 Exclude 5
3 9 9 7 Include 6
3 10 3 7 Exclude 6
1 11 0 18 Exclude 3
2 12 5 13 Include 5
3 13 11 7 Include 6
3 14 5 7 Exclude 6
2 15 0 13 Exclude 5
GRAPH COLORING:

 Graph coloring is a way of coloring the vertices of a graph such that no

two adjacent vertices share the same color.
 The convention of using colors originates from coloring the countries of a
map, where each face is literally colored.
 This was generalized to coloring the faces of a graph embedded in the
plane. By planar duality(x and y coordinates) it became coloring the
vertices, and in this form it generalizes to all graphs.
Consider an undirected graph G=(V,E) and an integer k. Assign one of k
colors to each node, such that adjacent nodes get different colors.
For example, let G=(V,E) where V={1,2,3,4} and E={(1,2),(1,3),(2,3),
(2,4),(3,4)} and suppose that k=3 (Red,Green,Blue). A valid coloring c of G is:
c(1) = R, c(2) = G, c(3) =B,R C(4) = R. G
1 2

R B
4 3
Graph Coloring
SRET
- 30 -
(Advanced Data structures and Algorithms (ADSA)

Suppose that |V|=n. Then,

 (c1,c2,……cn) is a possible coloring of graph where c i is the color of i-
node in the graph.
 Note that there are kn possible colorings.
 A coloring is feasible or valid if no two adjacent nodes are given the
same color, that is, if(i,j) ε E then ci ≠ cj.
Consider a graph shown above with k=3. There are six valid colorings of
graph given in the table as follows:
6 valid colorings
mod (i (ii (iii (iv (v (vi
e ) ) ) ) ) )
1 R R G G B B
2 G B B R R G
3 B G R B G R
4 R R G G B B

Note that all these colorings are sort of equivalent. They all share the following
structure:
 The same color is used for both node 1 and node 4. For colorings (i) and
(ii), it is R, for colorings (iii) and (iv), it is G, and for colorings (v) and (vi), it
is B.
 Nodes 2 and 3 must have distinct colors different from each other and
from the color used for nodes 1 and 4.
By this observation, it follows that two colorings are equivalent if one can
be transformed into another by permuting the k colors.

We will use the following strategy to find all valid colorings of a graph:
 Order nodes arbitrarily
 Assign the first node a color.

SRET
- 31 -
(Advanced Data structures and Algorithms (ADSA)

 Given a partial assignment of colors (c1,c2,……ci-1) to the first i-1 nodes, try
to find a color for the i-th node in the graph.
 If there is no possible color for the i-th node given the previous choices,
backtrack to a previous solution.

Consider a graph shown in figure 6.7 with k=3.

Step 1: Choose a color for node 1. It can be one of: R, B or G. Say we

choose R.
Step 2: Given partial coloring (R), we choose a color for node 2. It can be
one of : G or B. Say we choose G.
Step 3: Given partial coloring (R , G), we choose a color for node 3. It
cannot be either R or G, so it must be B since k=3.
Step 4: Given partial coloring (R ,G, B), we choose a color for node 4. It
cannot be B or G, so it must be this gives the coloring (R,G,B,R)
which is coloring (i).
We have no more choices of colors in step 4, and in step 3. We have
one choice in step 2.
Step 2: Given partial coloring (R), we choose a different color for node 2. we
choose B for node 2.
Step 3: Given partial coloring (R,B), we choose a color for node 3. It cannot
be R or B, so it must be G.
Step 4: Given partial coloring (R,G,B), we choose a color for node 4. It
cannot be B or G, so it must be R. This gives the coloring (R,G,B,R)
which is coloring (ii).
We have no more choices of colors in steps 4,3 and 2. We go back
to step 1.
Step 1: We choose a different color for node 1, say B. This will produce a
branch in the tree equivalent to the first branch where R and B are
switched. Thus, we will get the colorings (v) and (vi).
If we choose G for node 1 then, we will again get a branch equivalent to
the first one with R and G swapped. This will
0 produce the colorings (iii) and (iv).

R G
1 1 1 B

G B R B R G R G
2 2 2 2 2 2 2 2 2 B

G G
B B R B B R G R R G
SRET 3 3 3 3 3 3 3 3 3 3 3 3

- 32 -
R R
4 3 4 G 3 G B 4 B 3

Graph Coloring State Space Tree

(Advanced Data structures and Algorithms (ADSA)

The following Algorithm solves the graph coloring problem. It generates one
solution (only).

C [1....j-1]: a partial coloring for the first j-1 nodes of the graph.
adj [1..n][1..n]: adjacency matrix for n-node graph.
k: Number of colors(labels).
We invoke the algorithm: Coloring(C,j,k,n), where j is the first node(j = 1).

Algorithm: Coloring (C[1..n],j,k,n)

1. If j=n+1, then
2. print C
3. Endif
4. For i=1 to k
5. C[j] = i
6. If Valid(C,j,n) is true, then
7. Coloring(C,j+1,k,n)
8. Endif
9. Endfor.

Algorithm 6.5(b): Valid(C[1..n], j, n)

1. For i=1 to n //for all adjacent nodes v of j with

v<j
2. If adj[j][i] ≠0, then
3. v=i
4. If C[v] = C[j], then
5. Return false
6. Endif
7. Endif
8. Endfor
9. Return true.

SRET
- 33 -
(Advanced Data structures and Algorithms (ADSA)

We now trace the Coloring algorithm by considering 4 node graph in the

following fig.
R G
1 2
1 2 3 4
1 0 1 1 0
R B 2 1 0 1 1
4 3 3 1 1 0 1
Graph Coloring 4 0 1 1 0

Initially, the algorithm: (C,j,k,n) is called with c = {0,0,0,0},j=1,k=3, and

n=4. After first recursive call, C={1,0,0,0} – that is, the node 1 is labeled with
1(red). After second call, C={1,2,0,0} – the node 2 is labeled with 2 (Green).
After third call, C={1,2,3,0} – the node 3 is labeled with 3(Blue). After fourth
call, C={1,2,3,1} – the node 4 is labeled with 1(Red). It prints the labels 1,2,3,1.
After this, the algorithm backtracks, checks two more branches and
terminates. The valid solution is : node 1(label 1), node 2(label 2), node 3(label
3), node 4(label 1). This is one of the six valid colorings shown in Column(i) of
Colorings table above.
The number of internal nodes in the state space tree is given by: ∑ k i
(where 0<i<n-1). Valid routine takes O(kn) time at each internal node to find the
children corresponding to valid colorings. Hence,
∑ ki n = n(k n+1
- 2) / (k-1) = O(nkn). (where 1 < i < n)

AMILTONIAN CYCLES:

A Hamiltonian cycle in a connected graph is a cycle that passes through

every vertex exactly once. The graph in Figure 6.10(a) contains the
Hamiltonian cycle A,B,C,E,F,D,A. The graph in the following Figure contains no
Hamiltonian cycle.

A B A B

C C
D

F E D E
SRET
UnDirected Graphs - 34 -
(Advanced Data structures and Algorithms (ADSA)

There is no known easy way to determine whether a given graph contains

a Hamiltonian cycle. We now look at a backtracking algorithm that finds the
Hamiltonian cycle in a graph. The graph may be directed or undirected. Let us
apply the backtracking technique to find the Hamiltonian cycle in the following
graph (Figure 6.11(a)), and obtain a Hamiltonian cycle as shown in Figure
6.11(b). A

B
Illustration of Hamiltonian

A B C F

D E E
D C

E D F C

F E
F D

A
Example :
Find a Hamiltonian cycle for the directed graph.
Directed Graph and its Adjacency List
3 N 1 2 3
1 → 2
2 → 3 → 4 → 6
2
3 → 4
5
4 4 → 1 → 7
5 → 3
1
6 → 1 → 4 → 7
7 → 5
6 7
Given a directed graph G = (V,E). We store the
graph as an adjacency list (for each vertex v ε {1,2,…,n}, and store a list of the
vertices w such that (v,w) ε E). We store a Hamiltonian cycle as A[1..n], where
the cycle is:
A[n]→ A[n-1] → … → A[2] → A[1] → A[n]
We also set up an array D, the out-degree of each vertex. The values are
tabulated as:

SRET
- 35 -
(Advanced Data structures and Algorithms (ADSA)

D 1 2 3 4 5 6 7

1 3 1 2 1 3 1

Pruning: We keep a Boolean array mark[1..n], where mark[i] is 1 if vertex i is

visited and 0 if vertex is unvisited (prune here).
To solve the Hamiltonian cycle problem, we use arrays N (adjacency lists)
and D (degree) for the input graph, and execute the Algorithm 6.6 (given in
three parts). Assume that n ≥ 2.

Algorithm : Initialization()

1. For i = 1 to n-1
2. mark [i] =0
3. Endfor
4. mark [n] = 1; A[n] = n
5. Hamilton(n-1)

Algorithm : Hamilton(k)

1. If k = 0 , then
2. process(A)
3. Else
4. For j = 1 to D[A[k+1]]
5. w = N[A[k+1]][j]
6. If mark[w] = 0, then
7. mark[w] = 1; A[k] = w
8. Endif
9. Endfor
10. Hamilton(k-1)
11. mark[w]=0
12. Endif.

Algorithm : Process(A)

1. ok = 0
2. For j = 1 to D[A[1]]
3. If N[A[1],j] = A[n], then ok = 1
4. Endfor
5. If ok = 1, then Print(A).
Basic concepts
Some computational problems are complex and difficult to find efficient
algorithms for solving them and it cannot be solved in future also and have non

SRET
- 36 -
(Advanced Data structures and Algorithms (ADSA)

polynomial time algorithm and some problems are there, that can be solved by
a time algorithm, like O(n), O(logn), O(n logn), and O(n 2). Polynomial time
algorithms are being tractable and problems that require non-polynomial time
as being intractable.

Any problem for which answer is either yes or no is called decision

problem. The algorithm for decision problem is called decision algorithm.
Any problem that involves the identification of optimal cost is called
optimization problem. The algorithm for optimization problem is called
optimization algorithm.
Definition of P: Problems that can be solved in polynomial time.(“P” stands
for polynomial).
Examples: searching of key element, sorting of elements, All pair shortest
path.
Definition of NP: It stands for “non-deterministic polynomial time”. Note that
NP does not stand for “non-polynomial”.
Examples: Travelling Sales Person problem, Graph coloring problem,
Knapsack problem, Hamiltonian circuit problems.
The NP class problems can be further categorized into NP-Complete and NP-
Hard problems.
P and NP
 The P versus NP problem is a major unsolved problem in computer
science.
 Informally, it asks whether every problem whose solution can be
efficiently checked by a computer can also be efficiently solved by a
computer.
 In essence, the question P = NP?
 Suppose that solutions to a problem can be verified quickly. Then, can the
solutions themselves also be computed quickly?
 The theoretical notion of quick used here is an algorithm that runs in
polynomial time. The general class of questions for which some algorithm
can provide an answer in polynomial time is called "class P" or just "P".
 For some questions, there is no known way to find an answer quickly, but
if one is provided with information showing what the answer is, it may be

SRET
- 37 -
(Advanced Data structures and Algorithms (ADSA)

possible to verify the answer quickly. The class of questions for which an
answer can be verified in polynomial time is called NP.

NP-complete
 To attack the P = NP question the concept of NP-completeness is very
useful.
 Informally the NP-complete problems are the "toughest" problems in NP
in the sense that they are the ones most likely not to be in P.
 NP-complete problems are a set of problems that any other NP-problem
can be reduced to in polynomial time, but retain the ability to have their
solution verified in polynomial time.
 In comparison, NP-hard problems are those at least as hard as NP-
complete problems, meaning all NP-problems can be reduced to them,
but not all NP-hard problems are in NP, meaning not all of them have
solutions verifiable in polynomial time.

NP-HARD AND NP -COMPLETE PROBLEMS:

 We are concerned with the distinction between

 problems that can be solved by a polynomial time algorithm and

SRET
- 38 -
(Advanced Data structures and Algorithms (ADSA)

 problems for which no polynomial time algorithm is known.

 It is an unexplained phenomenon that for many of the problems we
known and study, the best algorithms for their solutions have computing
times that cluster into two groups.
 The first group consists of problems whose solution times are bounded
by polynomials of small degree. Examples for ordered searching,
which is O(log n), polynomial evaluation which is O(n), sorting which is
O(n long n), and string editing which is O(mn).
 The second group is made up of problems whose best-known
algorithms are nonpolynomial.

Examples we have seen include the traveling salesperson and the

knapsack problems.

 No one has been able to develop a polynomial time algorithm for any
problem in the second group.
 The theory of NP-completeness which we present here does not provide
a method of obtaining polynomial time algorithms for problems in the
second group. Nor does it say that algorithms of this complexity do not
exist. Instead, what we do is shown that many of the problems for which
there are no known polynomial time algorithms are computationally
related.
 In fact, we establish two classes of problems. These are given the names
 NP-hard and
 NP-complete.
 A problems that is NP-complete has the property that it can be solved in
polynomial time if and only if all other NP-complete problems can also be
solved in polynomial time.
 If an NP-hard problem can be solved in polynomial time, then all NP-
complete problems can be solved in polynomial time.

SRET
- 39 -
(Advanced Data structures and Algorithms (ADSA)

 All NP-complete problems are NP-hard, but some NP-hard problems are
not known to be NP-complete.
Nondeterministic Algorithms:
 Up to now the notion of algorithm that we have been using has the
property that the result of every operation is uniquely defined.
Algorithms with this property are termed deterministic algorithms. Such
algorithms agree with the way programs are executed on a computer.
 In a theoretical framework we can remove this restriction on the outcome
of every operation. We can allow algorithms to contain operations whose
outcomes are not uniquely defined but are limited to specified sets
possibilities.
 The machine executing such operations is allowed to choose any one of
these outcomes subject to a termination conditions to be defined later.
 This leads to the concept of a nondeterministic algorithm.
 To specify such algorithms, we introduce three new functions:

1. Choice(S) arbitrarily chooses one of the elements of set S.

2. Failure() signals an unsuccessful completion.
3. Success() signals successful completion.

 The assignment statement x := Choice (1,n) could result in x

being assigned any one of the integers in the range [1,n].
 There is no rule specifying how this choice is to be made.
 The Failure() and Success() signals are used to define a
computation of the algorithm. These statements cannot be used to
effect a return.
 Whenever there is a set of choices that leads to a successful
completion, then one such set of choices is always made and the
algorithm terminates successfully.

SRET
- 40 -
(Advanced Data structures and Algorithms (ADSA)

 A nondeterministic algorithm terminates unsuccessfully if and only

if there exists no set of choices leading to a success signal.
 The computing times for Choice, Success, and Failure are taken to
be O(1).
A machine capable of executing a nondeterministic algorithm in this
way is called a nondeterministic machine.

Examples :

Example 1 Consider the problem for searching for an element x in a

given set of elements A[1 : n], n ≥ 1. We are required to determine an
index j such that A[j] = x or j = 0 if x is not in A.

A nondeterministic algorithm for this is Algorithm 11.1.

1 j := Choice (1,n);
2 if A[j] = x then {write (j); Success();}
3 write (0); Failure();
Algorithm 1: Nondeterministic search

Example 2: Nondeterministic sorting

1 Algorithm NSort(A, n)
2 // Sort n positive integers.
3 {
4 for i := 1 to n do B[i] := 0; // Initialize B[].
5 for i := 1 to n do
6 {
7 j := Choice(1,n);
8 if B[j] ≠ 0 then Failure();
9 B[j] := A[i];
10 }
11 for i := 1 to n – 1 do //Verify order.
12 if B[i] > B[i + 1] then Failure();
13 write (B[1 : n]);
14 Success();
15 }
The classes NP-hard and NP-complete:

SRET
- 41 -
(Advanced Data structures and Algorithms (ADSA)

In measuring the complexity of an algorithm, we use the input length as

the parameter. An algorithm A is of polynomial complexity if there exists a
polynomial p() such that the computing time of A is O(p(n)) for every input of
size n.

Definition : P is the set of all decision problems solvable by deterministic

algorithms in polynomial time. NP is the set of all decision problems solvable by
nondeterministic algorithms in polynomial time.
Since deterministic algorithms are just a special case of nondeterministic
ones, we conclude that
P  NP. What we do not know, and what has become perhaps the most famous
unsolved problem in computer science, is whether P = NP or P ≠ NP.
Is it possible that for all the problems in NP, there exist polynomial time
deterministic algorithms that have remained undiscovered? This seems
unlikely, at least because of the tremendous effort that has already been
expended by so many people on these problems. Nevertheless, a proof that P
≠ NP is just as elusive and seems to require as yet undiscovered techniques.
But as with many famous unsolved problems, they serve to generate other
useful results, and the question of whether NP  P is no exception. Figure 11.1
displays the relationship between P and NP assuming that P ≠ NP.

NP

SRET
- 42 -
(Advanced Data structures and Algorithms (ADSA)

Figure 11.1 Commonly believed relationship between P and NP

S. Cook formulated the following questions: Is there any single problem in NP

such that if we showed it to be in P , then that would imply that P = NP? Cook
answered his own question in the affirmative with the following theorem.

Definition: Let L1 and L2 be problems. Problem L1 reduces to L2 (also written

L1α L2) if and only if there is a way to solve L 1 by a deterministic polynomial
time algorithm using a deterministic algorithm that solves L 2 in polynomial time.

This definition implies that if we have a polynomial time algorithm for L 2,

then we can solve L1 in polynomial time. One can readily verify that α is a
transitive relation (that is, if L1α L2 and L2α L3,
then L1α L3 ).

Definition: A problem L is NP-hard if and only if satisfiability reduces to L

(satisfiability α L). A problem L is NP-complete if and only if L is NP-hard and L 
NP.

It is easy to see that there are NP-hard problems that are not NP-
completer. Only a decision problem can be NP-complete. However, an
optimization problem may be NP-hard. Furthermore if L1 is
A decision problem and L2 an optimization problem, it is quite possible that L 1α
L2. One can trivially show that the knapsack decision problem reduces to the
knapsack optimization problem. For the clique problem one can easily show that
the clique decision problem reduces to the clique optimization problem. In fact,

SRET
- 43 -
(Advanced Data structures and Algorithms (ADSA)

one can also show that these optimization problems reduce to their
corresponding decision problems. Yet, optimization problems cannot be NP-
complete whereas decision problems can. There also exist NP-hard decision
problems that are not NP-complete. Figure 11.2 shows the relationship among
these classes.

SRET
- 44 -