Agenda for This Week

1
Agenda for This Week
Monday, April 18 Dynamic Programming
Wednesday, April 20 Dynamic Programming Case 3 Due
Friday, April 22 Dynamic Programming
Monday, April 25 AHP
2
Chapter 18

• Dynamic Programming Part 1

3
Dynamic Programming

• Dynamic Programming (DP) is a model that deals
  with decisions where several decisions need to be
  made in sequence usually over time called
  dynamic, multistage or sequential decision
  problems
• Also useful for decomposing multi-variable,
  single-time problems into several single-variable
  problems at different times

4
Dynamic Programming

• DP is a broad problem structure rather than a
  specific method
• The essential characteristic is that there is a
  series of decisions distributed in time, where
  there is an interrelationship between decisions
  thus, the current decision cannot be made
  independently of future ones.

5
Dynamic Programming Concepts

• Stages each point in the problem where a
  decision must be made (can be actual or
  artificial points in time
• States represents what the status of the system
  is when entering the next stage (e.g., how many
  resources left still to allocate)
• Policy decision making rule that transforms the
  state at a given stage into a state associated
  with an earlier stage
• Optimal Policy policy that optimizes the value
  of the objective function

6
Dynamic Programming

• General relationship between components

Decision, Di
Decision, Di-1
Decision, Di-2
State Si-3
State Si
Stage i
Stage i-1
Stage i-2
State Si-1
State Si-2
(Ti)
(Ti-1)
(Ti-2)
Return, Ri
Return, Ri-1
Return, Ri-2
7
Dynamic Programming

• At any stage i, the next state (moving backward),
  Si-1 is a function only of state Si (entering
  state) and the decision made, Di, at that stage

Decision, D1
State S1
Stage 1
State S0
(T1)
Return, R1
8
Dynamic Programming

• Ti is state transformation function
• Ti transforms state Si to Si-1 given the
  decision Di (the form of T depends on the
  specific problem)

Si-1 Ti (Si, Di)
9
DP Road Trip Example

• Suppose you are traveling from the West Coast to
  the East Coast and you plan on taking 4 days for
  your trip. Each night you have a choice of
  cities as to where you will stay, but the
  decision each night has an impact on the
  distances that will need to be traveled in the
  future.
• The diagram on the next slide shows the options
  you have and the distances between cities.

10
DP Road Trip Example
C11
7
4
C31
C21
6
9
4
8
5
7
8
3
C12
12
6
5
3
7
C41
C32
C22
C01
4
9
6
8
10
5
9
3
C13
6
10
6
C33
C23
6
8
C14
Stage 4
Stage 3
Stage 2
Stage 1
11
DP Road Trip Example
Naïve approach would say to look for the shortest
route from left to right
C11
7
4
C31
C21
6
9
4
8
5
7
8
3
C12
12
6
5
3
7
C41
C32
C22
C01
4
9
6
8
10
5
9
3
C13
6
10
6
C33
C23
6
8
C14
3 3 6 12 24
12
DP Road Trip Example
Stage 1 (From C1 to C0)
States
Total Distance
Current city
Distance
Travel to
C11 C12 C13 C14
C01 C01 C01 C01
9 12 8 10
9 12 8 10
13
DP Road Trip Example
Stage 2 (From C2 to C1)
Total Distance
Current city
Distance
Travel to

7 6 8 8
C21 C22
C11 C12 C13 C14
9 12 8 10
7 6 8 8
16 18 16 18


C11 C12 C13 C14
6 7 9 10
9 12 8 10
6 7 9 10
15 19 17 20
14
DP Road Trip Example
Stage 2 (From C2 to C1)
Total Distance
Current city
Distance
Travel to
9 6 6 8
C11 C12 C13 C14
C23
9 12 8 10
9 6 6 8
18 18 14 18

15
DP Road Trip Example
Stage 3 (From C3 to C2)
Total Distance
Current city
Distance
Travel to
C31 C32
C21 C22 C23
4 4 7
16 15 14
4 4 7
20 19 21

16 15 14
3 5 4
3 5 4
19 20 18
C21 C22 C23

16
DP Road Trip Example
Stage 3 (From C3 to C2)
Total Distance
Current city
Distance
Travel to
5 3 6
C33
C21 C22 C23
16 15 14
5 3 6
21 18 20

17
DP Road Trip Example
Stage 4 (From C4 to C3)
Total Distance
Current city
Distance
Travel to
5 3 6
C41
C31 C32 C33
19 18 18
5 3 6
24 21 24

Optimal Path C41 C32 C23
C13 C01
21 Days
18
For Next Class

• Read Balance of Chapter
• Complete HW 1
• Case 3 Due