Dynamic programming

1

• LECTURE 11
• Dynamic programming
• - II -

2
Outline

• What do we know about dynamic programming ?
• Application discrete knapsack
• Memory functions (memoization)
• Application optimal multiplication of matrices
• Application transitive closure of a binary
  relation

3
What do we know about dynamic programming ?

• It is a technique for solving problems which can
  be decomposed in overlapping subproblems it
  can be applied to problems having the optimal
  substructure property
• Its particularity is that it solves each smaller
  subproblem only once and it records each result
  in a table from which a solution to the initial
  problem can be constructed.

4
What do we know about dynamic programming ?

• Basic steps
• Analyze the structure of a solution how the
  solution of a problem depends on the solutions of
  subproblems. Usually this step means verifying
  the optimal substructure property.
• Find a recurrence relation which relates a value
  (e.g. the optimization criterion) corresponding
  to the problems solution with the values
  corresponding to subproblems solutions.
• Develop in a bottom up manner the recurrence
  relation and construct a table containing
  information useful to construct the solution.
• Construct the solution based on information
  collected in the previous step.

5
Outline

• What do we know about dynamic programming ?
• Application discrete knapsack
• Memory functions (memoization)
• Application optimal multiplication of matrices
• Application transitive closure of a binary
  relation

6
Application discrete knapsack

• The knapsack problem
• Let us consider a set of n objects. Each object
  is characterized by its weight (or dimension - d)
  and its value (or profit - p). We want to fill in
  a knapsack of capacity C such that the total
  value of the selected objects is maximal.
• Variants
• Continuous variant entire objects or part of
  objects can be selected. The components of the
  solution are from 0,1.
• Discrete variant (0-1) an object either is
  entirely transferred into the knapsack or is not
  transferred. The solution components are from
  0,1

7
Application discrete knapsack
Hypothesis the capacity C and the
dimensions d1,,dn are natural numbers The
problem can be reformulated as find
(s1,s2,,sn) with si in 0,1 such that
s1d1 sndn lt C (constraint)
s1p1 snpn is maximal
(optimization criterion) Remark the greedy
technique can be applied but it does not
guarantee the optimality
8
Application discrete knapsack
Example n3, C5, d11, d22, d33 p16,
p210, p312 Relative profit pr16, pr25,
pr34

• Greedy idea
• Sort decreasingly the set of objects on the
  relative profit (pi/di)
• Select the elements until the knapsack is
  overloaded
• Greedy solution (1,1,0)
• Total value V16

Remark this is not the optimal solution the
solution (0,1,1) is better since V22
9
Application discrete knapsack

• Analyzing the structure of an optimal solution
• Let P(i,j) be the generic problem of selecting
  from the set of objects o1,,oi in order to
  fill in a knapsack of capacity j.
• Remarks
• P(n,C) is the initial problem
• If iltn, jltC then P(i,j) is a subproblem of P(n,C)
• Let s(i,j) be an optimal solution of P(i,j).
  There are two situations
• si1 (the object oi is selected) gt this lead us
  to the subproblem P(i-1,j-di) and if s(i,j) is
  optimal then s(i-1,j-di) should be optimal
• si0 (the object oi is not selected) gt this lead
  us to the subproblem P(i-1,j) and if s(i,j) is
  optimal then s(i-1,j) should be optimal
• Thus the solution s has the optimal
  substructure property

10
Application discrete knapsack
2. Find a recurrence relation Let V(i,j) be the
total value corresponding to an optimal solution
of P(i,j) 0 if i0 or
j0 (the set is empty or the
knapsack has not capacity at
all) V(i,j) V(i-1,j) if digtj
or V(i-1,j)gtV(i-1,j-di) pi
(either the object i doesnt fit the
knapsack or by selecting it we
obtain a worse solution than by not
selecting it)
V(i-1,j-di)pi otherwise
11
Application discrete knapsack

• The recurrence relation can be written also as
• 0
  if i0 or j0
• V(i,j) V(i-1,j)
  if digtj
• maxV(i-1,j), V(i-1,j-di) pi
  if diltj
• Remarks
• for the problem P(n,C) the table V has (n1) rows
  and (C1) columns
• V(n,C) gives us the value corresponding to the
  optimal solution

12
Application discrete knapsack
Example 0
if i0 or j0 V(i,j) V(i-1,j)
if digtj maxV(i-1,j),
V(i-1,j-di) pi if
diltj d 1 2 3 p 6 10 12
V 0 1 2 3 4 5 0
0 0 0 0 0 0 1 0
6 6 6 6 6 2 0 6
10 16 16 16 3 0 6 10
16 18 22
13
Application discrete knapsack
3. Developing the recurrence relation
0 if i0 or
j0 V(i,j) V(i-1,j)
if digtj maxV(i-1,j),
V(i-1,j-di) pi if diltj
Algorithm computeV (p1..n,d1..n,C) FOR
i0,n DO Vi,00 FOR j1,n DO V0,j0
FOR i1,n DO FOR j1,C DO
IF jltdi THEN Vi,jVi-1,j ELSE
Vi,jmax(Vi-1,j,Vi-1,j-dip
i) RETURN V0..n,0..C
14
Application discrete knapsack

• Constructing the solution
• Example
• 0 1 2 3 4 5
• 0 0 0 0 0 0 0
• 1 0 6 6 6 6 6
• 2 0 6 10 16 16 16
• 3 0 6 10 16 18 22
  

• Steps
• Compare V3,5 with V2,5. Since they are
  different it means that the object o3 is selected
• Go to V2,5-d3V2,210 and compare it with
  V1,26. Since they are different it means that
  also o2 is selected
• Go to V1,2-d2V1,00. Since the current
  capacity is 0 we cannot select another object
• Thus the solution is o2,o3 or s(0,1,1)

15
Application discrete knapsack

• Constructing the solution
• Example
• 0 1 2 3 4 5
• 0 0 0 0 0 0 0
• 1 0 6 6 6 6 6
• 2 0 6 10 16 16 16
• 3 0 6 10 16 18 22
  

Algorithm Construct(V0..n,0..C,d1..n) FOR
i1,n DO si0 in jC WHILE jgt0 DO
WHILE (igt1) AND (Vi,jVi-1,j)
DO ii-1 si1 jj-di
ii-1 RETURN s1..n
16
Application discrete knapsack
To compute V3,5 and to construct the solution
only the marked values are needed Thus the
number of computations could be reduced by
computing only the values which are necessary We
can do this by combining the top-down approach
with the idea of storing the computed values in a
table This is the so-called memoization
technique
Remark 0 1 2 3 4
5 0 0 0 0 0 0 0 1
0 6 6 6 6 6 2 0
6 10 16 16 16 3 0 6
10 16 18 22
17
Outline

• What do we know about dynamic programming ?
• Application discrete knapsack
• Memory functions (memoization)
• Application optimal multiplication of matrices
• Application transitive closure of a binary
  relation

18
Memory functions (memoization)

• Goal to solve only the subproblems that are
  necessary and to solve them only once
• Basic idea combine the top-down approach with
  the bottom-up approach since
• The classic top-down approach solves only the
  necessary subproblems but common subproblems are
  solved more than once (this leads us to an
  inefficient algorithm)
• The classic bottom-up approach solves all
  subproblems but even the common ones are solved
  only once

19
Memory functions (memoization)

• Steps in applying the memoization
• Initialize the table with a virtual value (this
  value should be different from any value which
  could be obtained during the computations)
• Compute the value we are searching for (e.g.
  Vn,C) in a recursive manner by storing in the
  same time the computed values in the table and
  using these values any time it is possible

Virtual initialization FOR i0,n DO FOR
j0,C DO Vi,j-1 Recursive
function comp(i,j) IF Vi,jltgt-1 THEN RETURN
Vi,j ELSE IF i0 OR j0 THEN Vi,j0
ELSE IF jltdi THEN Vi,jcomp(i-1,j)
ELSE Vi,j
max(comp(i-1,j),comp(i-1,j-di)pi) RETURN
Vi,j Remark p1..n, d1..n and V0..n,0..C
are global variables
20
Outline

• What do we know about dynamic programming ?
• Application discrete knapsack
• Memory functions (memoization)
• Application optimal multiplication of matrices
• Application transitive closure of a binary
  relation

21
Application optimal multiplication of matrices

• Given n matrices A1, A2, , An to be
  multiplied in this order determine how to group
  the matrices such that the number of scalar
  multiplications is minimized
• Remarks
• The dimensions of matrices are compatible. Let us
  suppose that they are denoted by p0,p1,.pn and
  the matrix Ai has pi-1 rows and pi columns
• Different groupings of factors lead us to the
  same result (since matrices multiplication is
  associative) but they can lead us to different
  values for the number of scalar multiplications

22
Application optimal multiplication of matrices

• Example Let A1, A2 and A3 be three matrices
  having the dimensions (2,20), (20,5) and
  (5,10)
• p02 p120 p25 p310
  
• We consider the following groupings
• (A1A2)A3 - this needs (2205)2510300
  scalar multiplications
• A1(A2A3) this needs (20510)220101400
  scalar multiplications
• Remark for large values of n the number of
  possible groupings can be very large

23
Application optimal multiplication of matrices

• In the general case the grouping process is a
  hierarchical one
• The upper level define the grouping corresponding
  to the last multiplication
• The other levels correspond to groupings of the
  remaining factors
• We identify a grouping by the position of the
  last multiplication. For instance the grouping
• (A1Ak)(Ak1An)
• is specified by the value k
• There are (n-1) possible groupings at the upper
  level (1ltkltn-1) but to each upper level grouping
  correspond a lot of groupings of the two factors
  A1Ak and Ak1An

24
Application optimal multiplication of matrices
The numbers of groupings for a product of n
factors is 1 nlt2 K(n)
K(1)K(n-1) K(i)K(n-i)K(n-1)K(1
) ngt2 Remark K(n)C(n-1) where C(0),C(1)
are the Catalans numbers which satisfy
C(n)Comb(2n, n)/(n1) The
order of K(n) is almost 4n-1/(n-1)3/2
Thus an exhaustive search
is not at all efficient !
25
Application optimal multiplication of matrices

• Analyzing the structure of an optimal solution
• Let us denote by A(i..j) the product AiAi1Aj
  (iltj)
• If the optimal multiplication correspond to a
  grouping at position k (iltkltj) then the
  computation of A(i..k) and A(k1..j) should also
  be optimal (otherwise the computation of A(i..j)
  wouldnt be optimal)
• Thus the property of optimal substructure is
  satisfied

26
Application optimal multiplication of matrices
2. Constructing a recurrence relation Let us
denote by c(i,j) the number of scalar
multiplications necessary to compute A(i..j).
0 if ij c(i,j)
minc(i,k)c(k1,j)pi-1pkpj iltkltj if
iltj
Cost of computing A(i..k)
Cost of computing A(k1..j)
Cost of multiplying A(i..j) with A(k1..j)
All values of k are tried and the best one is
chosen
27
Application optimal multiplication of matrices
3. Developing the recurrence relation
0 if ij c(i,j)
minc(i,k)c(k1,j)
pi-1pkpj iltkltj,
if iltj Example p02 p120 p25 p
310

• Only the upper triangular part of the table will
  be used
• 1 2 3
• 1 0 200 300
• 2 - 0 400
• - - 0
• The elements are computed starting with the
  diagonal (j-i0), followed by the computation of
  elements which satisfy j-i1 and so on

28
Application optimal multiplication of matrices
3. Developing the recurrence relation
0 if ij c(i,j)
minc(i,k)c(k1,j)
pi-1pkpj iltkltj,
if iltj Let qj-i. The table will
be filled in for q varying from 1 to n-1 During
the computation of c the index of grouping is
also stored in a table s. s(i,j) k of the
optimal grouping of A(i..j)
Algorithm Compute(p0..n) FOR i1,n DO
ci,i0 FOR q1,n-1 DO FOR i1,n-q DO
jiq ci,jci,ici1,jpi-1p
ipj si,ji FOR ki1,j-1
DO rci,kck1,jpi-1pkpj
IF ci,jgtr THEN ci,jr
si,jk RETURN
c1..n,1..n,s1..n,1..n
29
Application optimal multiplication of matrices
Complexity analysis Problem size n Dominant
operation multiplication Efficiency class
?(n3)
Algorithm Compute(p0..n) FOR i1,n DO
ci,i0 FOR q1,n-1 DO FOR i1,n-q DO
jiq ci,jci,ici1,jpi-1p
ipj si,ji FOR ki1,j-1
DO rci,kck1,jpi-1pkpj
IF ci,jgtr THEN ci,jr
si,jk RETURN
c1..n,1..n,s1..n,1..n
30
Application optimal multiplication of matrices

• Constructing the solution
• Variants of the problem
• Find out the minimal number of scalar
  multiplications
• Solution this is given by c(1,n)
• Compute A(1..n) in a optimal manner
• Solution recursive algorithm (opt_mul)
• Identify the optimal groupings (placement of
  parentheses)
• Solution recursive algorithm (opt_group)

31
Application optimal multiplication of matrices

• Computation of A(1..n) in a optimal manner
• Hypothesis Let us suppose that
• A1..n is an global array of matrices (Ai is
  Ai)
• s1..n,1..n is a global variable and
  classic_mul is a function for computing the
  product of two matrices.
• opt_mul(i,j)
• IF ij THEN RETURN Ai
• ELSE
• X opt_mul(i,si,j)
• Yopt_mul(si,j1,j)
• Zclassic_mul(X,Y)
• RETURN Z

32
Application optimal multiplication of matrices
Printing the optimal grouping (the positions
where the product is split opt_group(i,j) IF
iltgtj THEN opt_group(i,si,j)
WRITE si,j
opt_group(si,j1,j)
33
Outline

• What do we know about dynamic programming ?
• Application discrete knapsack
• Memory functions (memoization)
• Application optimal multiplication of matrices
• Application transitive closure of a binary
  relation

34
Application transitive closure of a binary
relation

• Let R? 1,2,,nx1,2,,n a binary relation.
  Its transitive closure is the smallest (in the
  sense of set inclusion) relation R which is
  transitive and includes R
• R has the following property
• if i and j are from 1,,n and there exists
  i1,i2,.im such that
• i1Ri2, ., im-1Rim
• i1i and imj
• then iRj
• Examples R(1,2),(2,3)
  R(1,2),(2,3),(1,3)
• R(1,2),(2,3),(3,1)
• R(1,2),(2,3),(3,1),(1,3),(1,
  1), (2,1),(2,2),(3,2),(3,3)

35
Application transitive closure of a binary
relation
Even if this is not an optimization problem it
can be solved by using the idea of dynamic
programming of deriving a recurrence
relation. R is successively constructed
starting from R0R and using R1, R2,RnR The
intermediate relations Rk (k1..n) are defined as
follows i Rk j lt gt i
Rk-1 j or i Rk-1 k and k Rk-1 j Example R(1,2
),(2,3)
R1R R2(1,2),(2,3),(1,3) RR3
(1,2),(2,3),(1,3)
36
Application transitive closure of a binary
relation
Representation of binary relations Let us
consider that a binary relation is represented
using a nn matrix whose elements are defined as
follows 1 if iRj r(i,j)
0 if not iRj Example R
(1,2),(2,3) 0 1 0 r 0 0 1
0 0 0
37
Application transitive closure of a binary
relation
Recurrence relation for the matrices
1 if rk-1(i,j)1 OR (rk-1(i,k)1
AND rk-1(k,j)1) rk(i,j)
0 otherwise Example 0 1 0
0 1 0 0 1 1
0 1 1 r 0 0 1 r1 0 0 1
r2 0 0 1 r3 0 0 1 0 0 0
0 0 0 0 0 0
0 0 0
38
Application transitive closure of a binary
relation
Warshalls algorithm It develops the recurrence
relationship on matrices by using two matrices r1
and r2
Closure(r1..n,1..n) r21..n,1..nr1..n,1..n
FOR k1,n DO r11..n,1..nr21..n,1..n
FOR i1,n DO FOR j1,n DO IF
r1i,j0 OR r1i,k1 AND r1k,j1 THEN
r2i,j1 ELSE r2i,j0 RETURN
r21..n,1..n
39
Next lecture will be on

• Backtracking
• and its applications