Dynamic Programming

1
Dynamic Programming

• Dynamic Programming is a general algorithm
  design technique
• Invented by American mathematician Richard
  Bellman in the 1950s to solve optimization
  problems
• Programming here means planning
• Main idea
• solve several smaller (overlapping) subproblems
• record solutions in a table so that each
  subproblem is only solved once
• final state of the table will be (or contain)
  solution

2
Example Fibonacci numbers

• Recall definition of Fibonacci numbers
• f(0) 0
• f(1) 1
• f(n) f(n-1) f(n-2)
• Compute the nth Fibonacci number recursively
  (top-down)
• f(n)
• f(n-1)
  f(n-2)
• f(n-2) f(n-3) f(n-3)
  f(n-4)
• ...

3
Example Fibonacci numbers (2)

• Compute the nth Fibonacci number using bottom-up
  iteration
• f(0) 0
• f(1) 1
• f(2) 01 1
• f(3) 11 2
• f(4) 12 3
• f(n-2)
• f(n-1)
• f(n) f(n-1) f(n-2)

4
Examples of Dynamic Programming Algorithms

• Computing binomial coefficients
• Optimal chain matrix multiplication
• Constructing an optimal binary search tree
• Warshalls algorithm for transitive closure
• Floyds algorithms for all-pairs shortest paths
• Some instances of difficult discrete optimization
  problems
• travelling salesman
• knapsack

5
Binomial coefficients

• Algorithm based on identity
• Algorithm Binomial(n,k)
• for i ? 0 to n do
• for j ?0 to min(j,k) do
• if j0 or ji then Ci,j ? 1
• else Ci,j?Ci-1,j-1Ci-1,j
• return Cn,k
• Pascals Triangle
• Example
• Space and Time efficiency

6
Warshalls Algorithm Transitive Closure

• Computes the transitive closure of a relation
• Alternatively all paths in a directed graph
• Example of transitive closure

0 0 1 0 1 1 1 1 0 0 0 0 1 1 1 1
0 0 1 0 1 0 0 1 0 0 0 0 0 1 0 0
7
Warshalls Algorithm (2)

• Main idea a path exists between two vertices i,
  j, iff
• there is an edge from i to j or
• there is a path from i to j going through vertex
  1 or
• there is a path from i to j going through vertex
  1 and/or 2 or
• ...
• there is a path from i to j going through any of
  the other vertices

R0 0 0 1 0 1 0 0 1 0 0 0 0 0 1 0
0
R1 0 0 1 0 1 0 1 1 0 0 0 0 0 1 0 0
R2 0 0 1 0 1 0 1 1 0 0 0 0 1 1 1 1
R3 0 0 1 0 1 0 1 1 0 0 0 0 1 1 1 1
R4 0 0 1 0 1 1 1 1 0 0 0 0 1 1 1 1
8
Warshalls Algorithm (3)

• In the kth stage find if a path exists between
  two vertices i, j using just vertices among 1,,k
  
• R(k-1)i,j (path
  using just 1 ,,k-1)
• R(k)i,j or
• (R(k-1)i,k R(k-1)k,j) (path from i
  to k and from
• k to i
  using just 1,,k-1)

k
i
kth stage
j
9
Warshalls Algorithm (4)
Algorithm Warshall(A1..n,1..n) R(0)?A for k?1
to n do for i?1 to n do for j?1 to n do
R(k)i,j? R(k-1)i,j or R(k-1)i,k and
R(k-1)k,j return R(k) Space and Time efficiency
10
Floyds Algorithm All pairs shortest paths

• In a weighted graph, find shortest paths between
  every pair of vertices
• Same idea construct solution through series of
  matrices D(0), D(1), using an initial subset of
  the vertices as intermediaries.
• Example

11
Floyds Algorithm (2)

• Algorithm Floyd(W1..n,1..n)
• D?W
• for k?1 to n do
• for i?1 to n do
• for j?1 to n do
• Di,j? min(Di,j,Di,kDk,j)
• return D
• Space and Time efficiency
• When does it not work?
• Principle of optimality

12
Optimal Binary Search Trees

• Keys are not searched with equal probability
• Suppose keys A, B, C, D with probabilities 0.1,
  0.2, 0.3, 0.4
• What is the average search cost in each possible
  structure?
• How many different structures can be produced
  with n nodes ?
• Catalan number C(n)comb(2n,n)/(n1)

13
Optimal Binary Search Trees (2)

• Ci,j denotes the smallest average search cost
  in a tree including items i through j
• Based on the principle of optimality, the
  optimal search tree is constructed by using the
  recurrence
• Ci,j minCi,k-1Ck1,j Sps
• for 1i j n and i k j
• Ci,i pi

14
Optimal Binary Search Trees (3)
Algorithm OptimalBST(P1..n) for i ? 1 to n do
Ci,i-1?0 Ci,i?Pi, Ri,i?I Cn1,n?0 fo
r d?1 to n-1 do for i?1 to n-d do
j?id minval?8 for k?i to j do if
Ci,k-1Ck1,jltminval minval?
CI,k-1Ck1,j kmin?k Ri,j?kmin
sum?Pi for s?i1 to j do sum?sumPs Ci,
j? minvalsum return C1,n, R
15
The Knapsack problem

• Problem given n items with weight w1,w2, , wn,
  values v1, v2, , vn and capacity W. Find the
  most valuable subset that fit into the knapsack.
• Solution with exhaustive search
• Let Vi,j denote the optimal solution for the
  first i items and capacity j. Purpose to find
  Vn,W
• Recurrence
• maxVi-1,j,viVi-1,j-wi if j-wi0
• Vi,j
• Vi-1,j if j-wilt0

16
The Knapsack problem (2)
Example W5 Space and Time efficiency
i,j 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0 ?
item weight value
1 2 12
2 1 10
3 3 20
4 2 15
17
Memory functions 1
A disadvantage of the dynamic programming
approach is that is solves subproblems not
finally needed. An alternative technique is to
combine a top-down and a bottom-up approach
(recursion plus a table for temporary results)
18
Memory functions 2
Algorithm MFKnapsack(i,j) if Vi,jlt0 if
jltWeightsi value? MFKnapacki-1,j else val
ue?maxMFKnapsack(i-1,j), ValuesiMFKnapsacki
-1,j-Weightsi Vi,j?value return Vi,j
19
Memory functions (3)
Example W5 Space and Time efficiency
i,j 0 1 2 3 4 5
0
1
2
3
4 ?
item weight value
1 2 12
2 1 10
3 3 20
4 2 15