Chapter 8

Dynamic Programming
Copyright 2007 Pearson Addison-Wesley. All rights reserved.
8-1
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Dynamic Programming
Dynamic Programming is a general algorithm design technique
for solving problems defined by or formulated as recurrences
with overlapping subinstances

Invented by American mathematician Richard Bellman in the
1950s to solve optimization problems and later assimilated by CS

Programming here means planning

Main idea:
- set up a recurrence relating a solution to a larger instance
to solutions of some smaller instances
- solve smaller instances once
- record solutions in a table
- extract solution to the initial instance from that table

8-2
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Example: Fibonacci numbers
Recall definition of Fibonacci numbers:

F(n) =F(n-1) +F(n-2)
F(0) =0
F(1) =1
Computing the n
th
Fibonacci number recursively (top-down):

F(n)

F(n-1) + F(n-2)

F(n-2) + F(n-3) F(n-3) + F(n-4)
...
8-3
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Example: Fibonacci numbers (cont.)
Computing the n
th
Fibonacci number using bottom-up iteration and
recording results:

F(0) =0
F(1) =1
F(2) =1+0 = 1

F(n-2) =
F(n-1) =
F(n) = F(n-1) +F(n-2)





Efficiency:
- time
- space

0

1


1

. . .

F(n-2)

F(n-1)

F(n)


n
n
What if we solve
it recursively?
8-4
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Examples of DP algorithms
Computing a binomial coefficient

Longest common subsequence

Warshalls algorithm for transitive closure

Floyds algorithm for all-pairs shortest paths

Constructing an optimal binary search tree

Some instances of difficult discrete optimization problems:
- traveling salesman
- knapsack

8-5
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Computing a binomial coefficient by DP
Binomial coefficients are coefficients of the binomial formula:
(a +b)
n
= C(n,0)a
n
b
0
+ . . . + C(n,k)a
n-k
b
k
+ . . . + C(n,n)a
0
b
n


Recurrence: C(n,k) = C(n-1,k) + C(n-1,k-1) for n >k > 0
C(n,0) = 1, C(n,n) = 1 for n > 0

Value of C(n,k) can be computed by filling a table:
0 1 2 . . . k-1 k
0 1
1 1 1
.
.
.
n-1 C(n-1,k-1) C(n-1,k)
n C(n,k)
8-6
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Computing C(n,k): pseudocode and analysis
Time efficiency: (nk)
Space efficiency: (nk)
8-7
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Knapsack Problem by DP
Given n items of
integer weights: w
1
w
2
w
n
values: v
1
v
2
v
n

a knapsack of integer capacity W
find most valuable subset of the items that fit into the knapsack

Consider instance defined by first i items and capacity j (j s W).
Let V[i,j] be optimal value of such an instance. Then
max {V[i-1,j], v
i
+ V[i-1,j- w
i
]} if j- w
i
> 0

V[i,j] =
V[i-1,j] if j- w
i
< 0

Initial conditions: V[0,j] = 0 and V[i,0] = 0
{
8-8
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Knapsack Problem by DP (example)
Example: Knapsack of capacity W = 5
item weight value
1 2 $12
2 1 $10
3 3 $20
4 2 $15 capacity j
0 1 2 3 4 5
0
w
1
= 2, v
1
=

12 1
w
2
= 1, v
2
=

10 2
w
3
= 3, v
3
=

20 3
w
4
= 2, v
4
=

15 4 ?
0 0 0
0 0 12
0 10 12
22 22 22
0 10 12 22 30 32
0 10 15 25 30 37
Backtracing
finds the actual
optimal subset,
i.e. solution.
8-9
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Knapsack Problem by DP (pseudocode)
Algorithm DPKnapsack(w[1..n], v[1..n], W)
var V[0..n,0..W], P[1..n,1..W]: int
for j :=0 to W do
V[0,j] := 0
for i :=0 to n do
V[i,0] :=0
for i := 1 to n do
for j :=1 to W do
if w[i] s j and v[i] +V[i-1,j-w[i]] >V[i-1,j] then
V[i,j] :=v[i] +V[i-1,j-w[i]]; P[i,j] :=j-w[i]
else
V[i,j] :=V[i-1,j]; P[i,j] :=j
return V[n,W] and the optimal subset by backtracing

Running time and space:
O(nW).
8-10
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Longest Common Subsequence (LCS)
A subsequence of a sequence/string S is obtained by
deleting zero or more symbols from S. For example, the
following are some subsequences of president: pred, sdn,
predent. In other words, the letters of a subsequence of S
appear in order in S, but they are not required to be
consecutive.

The longest common subsequence problem is to find a
maximum length common subsequence between two
sequences.

8-11
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
LCS
For instance,
Sequence 1: president
Sequence 2: providence
Its LCS is priden.

president

providence
8-12
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
LCS
Another example:
Sequence 1: algorithm
Sequence 2: alignment
One of its LCS is algm.
a l g o r i t h m

a l i g n m e n t
8-13
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
How to compute LCS?
Let A=a
1
a
2
a
m
and B=b
1
b
2
b
n
.
len(i, j): the length of an LCS between
a
1
a
2
a
i
and b
1
b
2
b
j
With proper initializations, len(i, j) can be computed as follows.


,
. and 0 , if )) , 1 ( ), 1 , ( max(
and 0 , if 1 ) 1 , 1 (
, 0 or 0 if 0
) , (

= >
= > +
= =
=
j i
j i
b a j i j i len j i len
b a j i j i len
j i
j i len
8-14
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
procedure LCS-Length(A, B)
1. for i 0 to m do len(i,0) = 0
2. for j 1 to n do len(0,j) = 0
3. for i 1 to m do
4. for j 1 to n do
5. if
j i
b a =
then

=
+ =
" " ) , (
1 ) 1 , 1 ( ) , (
j i prev
j i len j i len

6. else if
) 1 , ( ) , 1 ( > j i len j i len

7. then

=
=
" " ) , (
) , 1 ( ) , (
j i prev
j i len j i len

8. else

=
=
" " ) , (
) 1 , ( ) , (
j i prev
j i len j i len

9. return len and prev

8-15
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8


i j 0 1
p
2
r
3
o
4
v
5
i
6
d
7
e
8
n
9
c
10
e
0 0 0 0 0 0 0 0 0 0 0 0
1 p
2
0 1 1 1 1 1 1 1 1 1 1
2 r 0 1 2 2 2 2 2 2 2 2 2
3 e 0 1 2 2 2 2 2 3 3 3 3
4 s 0 1 2 2 2 2 2 3 3 3 3
5 i 0 1 2 2 2 3 3 3 3 3 3
6 d 0 1 2 2 2 3 4 4 4 4 4
7 e 0 1 2 2 2 3 4 5 5 5 5
8 n 0 1 2 2 2 3 4 5 6 6 6
9 t 0 1 2 2 2 3 4 5 6 6 6

Running time and memory: O(mn) and O(mn).
8-16
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8

procedure Output-LCS(A, prev, i, j)
1 if i = 0 or j = 0 then return
2 if prev(i, j)= then

i
a
j i prev A LCS Output
print
) 1 , 1 , , (

3 else if prev(i, j)= then Output-LCS(A, prev, i-1, j)
4 else Output-LCS(A, prev, i, j-1)

The backtracing algorithm
8-17
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8


i j 0 1
p
2
r
3
o
4
v
5
i
6
d
7
e
8
n
9
c
10
e
0 0 0 0 0 0 0 0 0 0 0 0
1 p
2
0 1 1 1 1 1 1 1 1 1 1
2 r 0 1 2 2 2 2 2 2 2 2 2
3 e 0 1 2 2 2 2 2 3 3 3 3
4 s 0 1 2 2 2 2 2 3 3 3 3
5 i 0 1 2 2 2 3 3 3 3 3 3
6 d 0 1 2 2 2 3 4 4 4 4 4
7 e 0 1 2 2 2 3 4 5 5 5 5
8 n 0 1 2 2 2 3 4 5 6 6 6
9 t 0 1 2 2 2 3 4 5 6 6 6

Output: priden
8-18
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Warshalls Algorithm: Transitive Closure
Computes the transitive closure of a relation

Alternatively: existence of all nontrivial paths in a digraph

Example of transitive closure:

3
4
2
1
0 0 1 0
1 0 0 1
0 0 0 0
0 1 0 0
0 0 1 0
1 1 1 1
0 0 0 0
1 1 1 1
3
4
2
1
8-19
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Warshalls Algorithm
Constructs transitive closure T as the last matrix in the sequence
of n-by-n matrices R
(0)
, , R
(k)
, , R
(n)
where
R
(k)
[i,j] = 1 iff there is nontrivial path from i to j with only the
first k vertices allowed as intermediate
Note that R
(0)
= A (adjacency matrix), R
(n)

=T (transitive closure)
3
4
2
1
3
4
2
1
3
4
2
1
3
4
2
1
R
(0)
0 0 1 0
1 0 0 1
0 0 0 0
0 1 0 0
R
(1)
0 0 1 0
1 0 1 1
0 0 0 0
0 1 0 0
R
(2)
0 0 1 0
1 0 1 1
0 0 0 0
1 1 1 1
R
(3)
0 0 1 0
1 0 1 1
0 0 0 0
1 1 1 1
R
(4)
0 0 1 0
1 1 1 1
0 0 0 0
1 1 1 1
3
4
2
1
8-20
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Warshalls Algorithm (recurrence)
On the k-th iteration, the algorithm determines for every pair of
vertices i, j if a path exists from i and j with just vertices 1,,k
allowed as intermediate

R
(k-1)
[i,j] (path using just 1 ,,k-1)
R
(k)
[i,j] = or
R
(k-1)
[i,k] and R
(k-1)
[k,j] (path from i to k
and from k to j
using just 1 ,,k-1)


i
j
k
{
Initial condition?
8-21
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Warshalls Algorithm (matrix generation)
Recurrence relating elements R
(k)
to elements of R
(k-1)
is:

R
(k)
[i,j] = R
(k-1)
[i,j] or (R
(k-1)
[i,k] and R
(k-1)
[k,j])
It implies the following rules for generating R
(k)
from R
(k-1)
:

Rule 1 If an element in row i and column j is 1 in R
(k-1)
,
it remains 1 in R
(k)

Rule 2 If an element in row i and column j is 0 in R
(k-1)
,
it has to be changed to 1 in R
(k)
if and only if
the element in its row i and column k and the element
in its column j and row k are both 1s in R
(k-1)

8-22
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Warshalls Algorithm (example)
3
4
2
1
0 0 1 0
1 0 0 1
0 0 0 0
0 1 0 0
R
(0) =

0 0 1 0
1 0 1 1
0 0 0 0
0 1 0 0
R
(1) =

0 0 1 0
1 0 1 1
0 0 0 0
1 1 1 1
R
(2) =

0 0 1 0
1 0 1 1
0 0 0 0
1 1 1 1
R
(3) =

0 0 1 0
1 1 1 1
0 0 0 0
1 1 1 1
R
(4) =

8-23
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Warshalls Algorithm (pseudocode and analysis)
Time efficiency: (n
3
)
Space efficiency: Matrices can be written over their predecessors
(with some care), so its (n^2).
8-24
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Floyds Algorithm: All pairs shortest paths
Problem: In a weighted (di)graph, find shortest paths between
every pair of vertices

Same idea: construct solution through series of matrices D
(0)
, ,
D
(n)
using increasing subsets of the vertices allowed
as intermediate

Example: 3
4
2
1
4
1
6
1
5
3
0 4
1 0 4 3
0
6 5 1 0
8-25
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Floyds Algorithm (matrix generation)
On the k-th iteration, the algorithm determines shortest paths
between every pair of vertices i, j that use only vertices among
1,,k as intermediate

D
(k)
[i,j] = min {D
(k-1)
[i,j], D
(k-1)
[i,k] + D
(k-1)
[k,j]}

i
j
k
D
(k-1)
[i,j]
D
(k-1)
[i,k]
D
(k-1)
[k,j]
Initial condition?
8-26
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Floyds Algorithm (example)
0 3
2 0
7 0 1
6 0
D
(0) =

0 3
2 0 5
7 0 1
6 9 0
D
(1) =

0 3
2 0 5
9 7 0 1
6 9 0
D
(2) =

0 10 3 4
2 0 5 6
9 7 0 1
6 16 9 0
D
(3) =

0 10 3 4
2 0 5 6
7 7 0 1
6 16 9 0
D
(4) =

3
1
3
2
6 7
4
1 2
8-27
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Floyds Algorithm (pseudocode and analysis)
Time efficiency: (n
3
)
Space efficiency: Matrices can be written over their predecessors
Note: Works on graphs with negative edges but without negative cycles.
Shortest paths themselves can be found, too. How?

If D[i,k] + D[k,j] < D[i,j] then P[i,j] k
Since the superscripts k or k-1 make
no difference to D[i,k] and D[k,j].
8-28
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Optimal Binary Search Trees
Problem: Given n keys a
1
< < a
n
and probabilities p
1
, , p
n

searching for them, find a BST with a minimum
average number of comparisons in successful search.

Since total number of BSTs with n nodes is given by
C(2n,n)/(n+1), which grows exponentially, brute force is hopeless.
Example: What is an optimal BST for keys A, B, C, and D with
search probabilities 0.1, 0.2, 0.4, and 0.3, respectively?

D
A
B
C
Average # of comparisons
= 1*0.4 + 2*(0.2+0.3) + 3*0.1
= 1.7
8-29
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
DP for Optimal BST Problem
Let C[i,j] be minimum average number of comparisons made in
T[i,j], optimal BST for keys a
i
< < a
j
, where 1 i j n.
Consider optimal BST among all BSTs with some a
k
(i k j )
as their root; T[i,j] is the best among them.
a
Optimal
BST for
a , ..., a
Optimal
BST for
a , ..., a
i
k
k-1
k+1 j
C[i,j] =
min {p
k
1 +

p
s
(level a
s
in T[i,k-1] +1) +

p
s
(level a
s
in T[k+1,j] +1)}

i k j
s = i
k-1
s =k+1
j
8-30
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
goal
0
0
C[i,j]
0
1
n+1
0 1 n
p
1
p
2
n
p
i
j
DP for Optimal BST Problem (cont.)
After simplifications, we obtain the recurrence for C[i,j]:
C[i,j] = min {C[i,k-1] + C[k+1,j]} + p
s
for 1 i j n
C[i,i] = p
i
for 1 i j n








s = i
j
i k j
Example: key A B C D
probability 0.1 0.2 0.4 0.3






The tables below are filled diagonal by diagonal: the left one is filled
using the recurrence
C[i,j] = min {C[i,k-1] + C[k+1,j]} + p
s ,
C[i,i] = p
i
;


the right one, for trees roots, records ks values giving the minima
0 1 2 3 4
1 0 .1 .4 1.1 1.7
2 0 .2 .8 1.4
3 0 .4 1.0
4 0 .3
5 0
0 1 2 3 4
1 1 2 3 3
2 2 3 3
3 3 3
4 4
5
i k j s = i
j
optimal BST
B
A
C
D
i

j
i

j
8-32
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Optimal Binary Search Trees
8-33
Copyright 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin Introduction to the Design & Analysis of Algorithms, 2
nd
ed., Ch. 8
Analysis DP for Optimal BST Problem
Time efficiency: (n
3
) but can be reduced to (n
2
) by taking
advantage of monotonicity of entries in the
root table, i.e., R[i,j] is always in the range
between R[i,j-1] and R[i+1,j]
Space efficiency: (n
2
)

Method can be expanded to include unsuccessful searches