ESI 6912: Dynamic Programming

1
ESI 6912Dynamic Programming

• Production and Inventory Control

2
Production andInventory Control

• Many production and inventory planning problems
  can be efficiently solved using Dynamic
  Programming.
• We start with a generalization of some of the
  examples from the first week.

3
Production andInventory Control

• Finite horizon periods 1,,T
• Deterministic (and integral) demands for a single
  product d1,,dT
• Production cost functions ct(x) the cost of
  producing x units in period t
• Holding cost functions ht(I) the cost of
  holding I units in inventory at the end of period
  t

4
Production andInventory Control

• Decision variables
• xt production in period t
• It inventory at the end of period t

5
Dynamic Programming formulation

• State
• (t,I) (current period, starting inventory)
• Initial state (T1,0)
• Ending state (1,0)
• Decision
• Quantity to produce in period t

6
Dynamic Programming formulation

• Optimal value function
• f(t,I) the minimum cost of satisfying demand in
  periods t,,T with starting inventory I
• We wish to find f(1,0)
• Boundary condition f(T1,0) 0

7
Dynamic Programming formulation

• Recurrence relation
• for t1,,T I0,1,2,
• Running time
• The network has O(TD) nodes and O(TD2) arcs
  (where D is total demand)
• The costs of each arc can be determined in
  constant time
• The running time is O(TD2)

8
Pros and cons

• Note that
• the cost functions can be arbitrary
• production and inventory capacities can easily be
  incorporated
• However
• in general, the problem is not polynomially
  solvable using this dynamic programming
  formulation, due to the dependence of the running
  time on D

9
Concave costs

• In the following, we will restrict ourselves to
  the case where the cost functions are concave.
• Note that if a function g is defined on the set
  of integers only, concavity means that

10
Problem formulations

• As an alternative to the
• mathematical programming formulation or the
• dynamic programming formulation
• the production and inventory control problem can
  be formulated as a
• minimum cost network flow problem.

11
Network flow formulation
Cost function c5(x5)
0
x1
1
2
3
4
5
I1
Cost function h3(I3)
12
Concave costs uncapacitated case

• Under the assumption that
• the cost functions are concave
• there exists an extreme point optimal solution
  to the problem.
• Under the assumption that
• there are no production or inventory capacities
• the extreme points of the mathematical
  programming formulation correspond to (spanning)
  trees in the network.

13
Extreme point solutions

• Example

0
1
2
3
4
5
14
Extreme point solutions

• Extreme point solutions have the so-called zero
  inventory ordering (ZIO) property
• It-1xt 0 for all t
• Put differently
• each demand node is supplied through a unique
  path or
• Each production period satisfies the demand of a
  consecutive set of periods (including the current)

15
Alternative formulation

• This structure can be used to find an alternative
  formulation of the problem, using the
  regeneration point approach.
• In this alternative formulation, the state is the
  first period for which to satisfy demand (with no
  initial inventory).

16
Dynamic Programming formulation

• State
• (t) (current period)
• Initial states (T1)
• Ending state (1)
• Decision
• Periods for which to produce in period t

17
Dynamic Programming formulation

• Optimal value function
• f(t) the minimum cost of satisfying demand in
  periods t,,T with starting inventory 0
• We wish to find f(1)
• Boundary condition
• f(T1) 0

18
Dynamic Programming formulation

• Recurrence relation
• for t1,,T
• Running time
• The network has O(T) nodes and O(T2) arcs
• The cost of all arcs can be determined in O(T2)
  time
• The running time is O(T2)

19
Dynamic Programming formulation

• Note that the running time for this special case
  with
• concave costs
• no capacities
• is polynomial in T.
• We next extend the problem by allowing for
  backlogging.

20
Production and Inventory Control with Backlogging

• Decision variables
• xt production in period t
• It inventory at the end of period t
• Bt amount backlogged to period t
• Backlogging cost functions bt(B) the cost of
  backlogging B units to period t.

21
Production and Inventory Control with Backlogging
22
Network flow formulation
Cost function c5(x5)
0
x1
1
2
3
4
5
I1
B1
Cost function h3(I3)
Cost function b2(B2)
23
Extreme point solutions

• Example

0
1
2
3
4
5
24
Extreme point solutions

• A generalization of the ZIO property
• At most one of It-1, xt, and Bt is positive
• As in the case without backlogging
• each demand node is supplied through a unique
  path or
• Each production period satisfies the demand of a
  consecutive set of periods (including the current)

25
Dynamic Programming formulation

• We can formulate this problem as a dynamic
  programming formulation with the same state
  variable.
• In the case without backlogging production any
  decision consists of
• Producing in the current period
• In the case with backlogging
• We need to choose the production period

26
Dynamic Programming formulation

• Recurrence relation
• for t1,,T
• Here

27
Dynamic Programming formulation

• Running time
• The network has O(T) nodes and O(T2) arcs
• The cost of all arcs can be determined in O(T3)
  time
• The running time is O(T3)

28
Production capacities

• We return to the production and inventory control
  problem
• without backlogging
• concave costs
• but we allow for production capacities

29
Network flow formulation
Cost function c5(x5)
Capacity b1
0
x1
1
2
3
4
5
I1
Cost function h3(I3)
30
Concave costs andproduction capacities

• As in the uncapacitated case, under the
  assumption that
• the cost functions are concave
• there exists an extreme point optimal solution
  to the problem.
• However,
• when there are production capacities
• the extreme points of the mathematical
  programming formulation do no longer correspond
  to (spanning) trees in the network

31
Extreme point solutions

• However, there is still a generalization of the
  ZIO property
• If It-10,Itgt0,,Isgt0,Is10, then among xt,,xs
  at most one satisfies 0ltxrltbr (t?r?s).

32
Extreme point solutions

• Example

0
1
2
3
4
5
33
Dynamic Programming formulation

• We can formulate this problem as a dynamic
  programming formulation with the same state
  variable as in the uncapacitated case.
• Recurrence relation

34
Dynamic Programming formulation

• But how do we compute Ct??
• It is the optimal cost of satisfying demand in
  periods t,,? using only a single period in which
  production is nonzero and below capacity.
• In general, these costs cannot be found in
  polynomial time.

35
Constant production capacities

• Special case
• All production capacities are equal b1bTb
• Now recall that we need to find the minimal cost
  of satisfying demand in periods t,,? using only
  a single period in which production is nonzero
  and below b.
• The total demand to be satisfied is

36
Constant production capacities

• In this case we can determine exactly
• The number of periods in which we produce to
  capacity, b, namely
• The quantity produced in the remaining production
  period, namely

37
Constant production capacities

• We can find the minimal cost of satisfying demand
  in periods t,,? using only a single period in
  which production is nonzero and below b
• using dynamic programming
• in polynomial time

38
Dynamic Programming formulation

• State
• (s,p) (current period, quantity produced so
  far)
• Initial state (t,0)
• Ending state

39
Dynamic Programming formulation

• Decision
• Quantity to produce in period s
• Note that the production quantity can only be
• Note that the cumulative production can only be
  0,kb,kb? for some k.

40
Dynamic Programming formulation

• Optimal value function
• f(s,p) the minimum cost of satisfying demand in
  periods s,,? when p units have been produced so
  far
• We wish to find f(t,0)
• Boundary condition

41
Dynamic Programming formulation

• Recurrence relation
• for st,,?-1 and all p0,kb,kb? for some k
• Note that
• we can only produce ? if we havent done so
  before
• we can only produce b if we havent produced all
  demand yet

42
Dynamic Programming formulation

• Running time (for computing the costs of a single
  arc)
• The network has O(T2) nodes and O(T2) arcs
• The cost of each arc can be computed in constant
  time
• The running time is O(T2)
• Running time (for solving the entire constant
  capacity problem)
• The network has O(T) nodes and O(T2) arcs
• The cost of each arc can be computed in O(T2)
  time
• The running time is O(T4)

43
Multi-level problems

• We return to the production and inventory control
  problem
• without backlogging
• concave costs
• no capacities
• We extend the supply chain to include multiple
  levels.

44
Network representation
Production Source Supply ?t dt
Production arcs Cost pt(xt)
Inventory holding arcs Flow cost ht?(It ?)
Transportation arcs Cost ct?(zt?)
4 period example
Demands, dt
45
Multi-level problems

• Under the assumption that
• the cost functions are concave
• there exists an extreme point optimal solution
  to the problem.
• Under the assumption that
• there are no production or inventory capacities
• the extreme points of the mathematical
  programming formulation correspond to (spanning)
  trees in the network.

46
Extreme point solutions

• The generalization of the zero inventory ordering
  (ZIO) property to the multi-level case is
• Put differently
• each demand node is supplied through a unique
  path or
• Each production period satisfies the demand of a
  consecutive set of periods
• Each arc carries the demand of a consecutive set
  of periods

47
Extreme point solutions

• Example

48
Dynamic Programming formulation

• State
• (t,?,s1,s2) (current period,current level,first
  periods demand to satisfy,last periods demand
  to satisfy)
• Initial state (T,L,T,T)
• Ending state (1,0,1,T)
• Decision
• How to split up the incoming shipment into
  inventory and transportation quantities

49
Dynamic Programming formulation

• Optimal value function
• C(t,?,s1,s2) the minimum cost of satisfying
  demand in periods s1,,s2 from period t at level
  ?
• We wish to find C(1,0,1,T)
• Boundary condition C(T,L,T,T)0

50
Dynamic Programming formulation

• Recurrence relations
• At the retailer level
• At the producer level
• At the warehouse levels

51
Dynamic Programming formulation

• Recurrence relations
• Note that at the last warehouse level the
  incoming quantity must satisfy the demand of a
  consecutive number of periods including the
  current period.

52
Dynamic Programming formulation

• Running time
• We have O(LT3) states
• The total number of operations to perform the
  necessary minimizations is O(T3(L-2)T4)
• The running time is
• O(T3) if L2
• O(LT4) if Lgt2