Lecture 7 Dynamic Programming I

1
Lecture 7Dynamic Programming - I

• Jones Pevzner
• Sects. 6.1-6.3

2
A brief review

• We have already been introduced to Dynamic
  Programming (DP) in the second lecture.
• Lets review the Rock Game as an introduction
  to the current chapter.

3
Dynamic Programming

• An extraordinarily powerful approach for some
  classes of problems.
• The trick is You must be able to express the
  answer for your current problem in terms of
  smaller problems for which you already know the
  answer.
• Sometimes the possibility of using dynamic
  programming presents itself when we arrange our
  data in matrix form.
• Lets look at the rock game proposed by the
  authors.

4
The Rock Game

• Rules
• Start with two piles of rocks, n in one pile and
  m in the other.
• Players take turns I start. On each turn, a
  player can either take a single rock (taken from
  either pile), or two rocks (one from each pile).
• Player who takes the last rock wins!
• Goal We want to find an optimal strategy, so
  that given an initial collection of n and m
  rocks, we are sure to win (provided thats
  possible, of course!)

5
Encoding the Rocks game

• We can use a matrix to represent the current
  state of the game being in row i and column j
  represents having i rocks in the first pile and j
  in the second.
• I put a W in each matrix cell if there exists a
  winning strategy for me, given the corresponding
  configuration of rocks, and an L if a competent
  opponent is sure to beat me.

6
Moves in the Rock game

• Each possible action that I can take reduces
  the size of the game, and corresponds to a move
  from my current cell to
• the cell immediately above
• the cell immediately to the left
• the cell on the diagonal, one space to the left
  and one up.
• Suppose that I move, and enter a cell that
  already has a W that symbol means that there
  exists a winning strategy for me, provided that I
  move first. Since it is now my opponents move, I
  have just lost the game!

7
Trying out the Rock Game example

• Lets fill in the matrix, following the lead of
  our authors.
• Note how we easily build on the known solutions
  for smaller problems to progressively fill in the
  matrix.
• Lets go a step further, and plot the possible
  moves that correspond to a winning strategy. Note
  that a particular game will be represented by a
  particular path through our matrix.
• Can we write a rule for expressing the form of a
  winning game, expressed as a path?

8
Revisiting USChange as a DP Problem

• Recall Given a set of coin denominations
  c1,c2,,cd, and a monetary value M to return as
  change, return the smallest number of coins
  possible.
• Heres how to recast the problem by reference to
  smaller problems. We define the best number of
  coins for M in terms of the best number of
  coins for M - ci, for each of the denominations.

9
Recursive Implementation
RecursiveChange(M,c,d) if M 0 return M
bestNumCoins lt- Infinity for i lt- 1 to d
if M gt ci numCoins lt-
RecursiveChange(M-ci,c,d) if numCoins 1
lt numBestCoins numBestCoins lt- numCoins 1
return bestNumCoins
10
Drawbacks of Recursion

• This recursive algorithm suffers the same problem
  of the recursive Fibonacci program it computes
  the same values multiple times.
• For example, if M10, we will generate a call for
  M - 5 5 we will also generate a call for M -
  1 9, which will generate (M - 1) - 1, etc until
  eventually there is another call with argument 5.
  
• We are better off to start from M0, and build up
  the best coin counts for increasing M, always
  referring back to the optimal values already
  recorded. This is the essence of dynamic
  programming.

11
Example

• Suppose we have already determined the optimal
  number of coins to return for the following
  values of M M1 1 (1 cent) M2 2 (2 X 1
  cent) M3 3 ( 3 X 1 cent) M4 4 (4 X 1
  cent) M5 1 (1 nickel)For M6, we
  check Optimum(6 - 1 cent) 1 Optimum(5) 1
  2 (1 nickel 1 cent) Optimum(6 - 1
  nickel) 1 Optimum(1) 1 2 (1 cent 1
  nickel)So, the optimum number of coins in change
  for 6 cents is 2 (1 nickel 1 cent)

12
DP Implementation
DPChange(M,c,d) bestNumCoins0 lt- 0 for m lt- 1
to M bestNumCoinsm lt- Infinity for
i lt- 1 to d if m gt ci if
(bestNumCoinsm-ci 1 lt bestNumCoinsm)
bestNumCoinsm lt- bestNumCoinsm-ci
1 return bestNumCoinsM
13
The Manhattan Tourist Problem
Source
3
2
4
0
1
0
2
4
3
3
2
4
2
4
6
5
2
1
0
7
3
4
4
4
5
2
1
3
3
0
2
Edge weight
5
6
8
5
3
1
3
2
2
Sink
Node/Vertex
Edge
A graph
14
The Greedy Solution
3
2
4
0
3
5
0
9
1
0
2
4
3
3
2
4
2
13
4
6
5
2
1
0
7
3
4
15
19
4
4
5
2
1
3
3
0
2
?
20
5
6
8
5
3
1
3
2
2
23
?
We see immediately that the greedy solution is
not optimal!
15
Approach to Dynamic Programming

• Start with the more general problem - find the
  best path from vertex i to vertex j. Call the
  length of the best path (the sum of the edge
  weights along it) sij.
• With analogy to the DPChange algorithm, we
  discover that we can efficiently find the
  solution by finding the answer for all the
  sub-problems (all possible sink vertices), and
  then picking out the solution we want.

16
Getting started - paths along the edges are
completely constrained!
Column 0
3
2
4
0
0
5
9
3
9
Row 0
1
0
2
4
3
3
2
4
2
1
4
6
5
2
1
0
7
3
4
5
4
4
5
2
1
3
3
0
2
9
5
6
8
5
3
1
3
2
2
14
17
Now we can find s11
Can arrive at (1,1) either from above (N) or the
left (W). We know the best scores for each of
those possible sources, and the weights of the
edges that connect (1,1) to them
3
2
4
0
0
5
9
3
9
1
0
2
4
3
3
2
4
2
4
1
4
6
5
2
1
0
7
3
4
5
4
4
5
2
1
3
3
0
2
9
5
6
8
5
3
1
3
2
2
14
18
and the rest of column 1
3
2
4
0
0
5
9
3
9
1
0
2
4
3
3
2
4
2
4
1
4
6
5
2
1
0
7
3
4
5
10
4
4
5
2
1
3
3
0
2
9
14
5
6
8
5
3
1
3
2
2
14
20
19
and the next column
3
2
4
0
0
5
9
3
9
1
0
2
4
3
3
2
4
2
4
1
7
4
6
5
2
1
0
7
3
4
5
10
17
4
4
5
2
1
3
3
0
2
9
14
22
5
6
8
5
3
1
3
2
2
14
30
20
20
and the next
3
2
4
0
0
5
9
3
9
1
0
2
4
3
3
2
4
2
4
1
13
7
4
6
5
2
1
0
7
3
4
5
20
10
17
4
4
5
2
1
3
3
0
2
22
9
14
22
5
6
8
5
3
1
3
2
2
14
30
32
20
21
and the last
3
2
4
0
0
5
9
3
9
1
0
2
4
3
3
2
4
2
4
1
13
15
7
4
6
5
2
1
0
7
3
4
5
20
10
17
24
4
4
5
2
1
3
3
0
2
22
25
9
14
22
5
6
8
5
3
1
3
2
2
14
30
32
34
20
22
Recurrence relation
23
Manhattan DP Algorithm
ManhattanTouristDP(vert, horiz, n, m) s(0,0) lt-
0 for i lt- 1 to n s(i,0) lt- s(i-1,0)
vert(i,0) for j lt- 1 to m s(0,j) lt-
s(0,j-1) horiz(0,j) for i lt- 1 to n
for j lt- 1 to m s(i,j) lt-
max s(i-1,j) vert(i,j),
s(i,j-1) horiz(i,j) return
s(n,m)
24
What if the grid is not regular?

• Model the street system as a directed graph. The
  intersections are vertices, and all the streets
  are one-way.
• We assume that there are no cycles in the graph
  it is a directed acyclic graph (DAG)
• Represent the graph as a set of vertices and a
  set of weighted edges G (V,E)
• Number vertices from 1 to V with a single
  integer we drop the matrix formalism.

25
Longest Path DAG Problem

• Input A weighted DAG G with source and sink
  vertices.
• Output the longest (optimal) path between source
  and sink.
• Notation
• Write a directed edge as a pair of vertices,
  (u,v).
• The indegree of a vertex is the number of
  incoming edges the outdegree the number that
  exit.
• Vertex u is a predecessor to v if (u,v) is in E
  if v has indegree k that means it has k
  predecessors.

26
Modified recurrence relation
A problem In what order to we visit the
vertices?? By the time vertex v is visited, all
of its predecessors must have already been
visited! An ordering of vertices v1,v2,,vn is
topological if for every edge (vi,vj) in the DAG,
i lt j. If the ordering is topological, all is
well because whenever we visit a vertex, we are
guaranteed to have already visited its
predecessors.