ESI 6912: Dynamic Programming

1
ESI 6912Dynamic Programming

• Equipment Replacement problems

2
Equipment replacement

• Consider a type of machine that deteriorates with
  age, and the decision to replace it.
• We have a need for such a machine during each of
  the next N time periods.
• The problem is to decide when (or if) to replace
  an existing machine by a new one so as to
  minimize the total costs.

3
Equipment replacement

• For simplicity, we assume that the costs are
  time-invariant.
• Costs
• ci cost of operating a machine that is i
  periods old at the start of a period for 1 period
  
• p price of a new machine
• ti trade-in value received when a machine that
  is of age i at the beginning of a period is
  traded for a new machine
• si salvage value received for a machine that
  has just turned age i at the end of period N
• a age of machine at start of 1st year

4
Equipment replacement states and decisions

• State
• (k,i) (current period, age of machine)
• k1,,N1 i1,,k-1,ak-1
• k is also the stage variable
• Initial state (1,a)
• Ending states (N1,i), i1,,N,aN
• Decisions
• buy or keep

5
Equipment replacement optimal value function

• Optimal value function
• f(k,i) the minimum cost of owning a machine in
  periods k,,N, starting year k with a machine
  that is i periods old
• We wish to find f(1,a)
• Boundary conditions
• f(N1,i) -si, for i1,,N,aN

6
Equipment replacement recurrence relation

• Recurrence relation

7
Equipment replacement network

• Network representation
• Current year ?
• Current age ?
• Keep
• Buy
• Initial age 2

8
Running time

• The acyclic network contains O(N2) arcs.
• The cost of each arc can be computed in O(1)
  time.
• The running time of the recursive fixing
  algorithm is thus O(N2).

9
Equipment replacement regeneration point approach

• Note that all paths but one pass through a state
  of the form (k,1).
• This corresponds to the observation that (unless
  the current machine is kept for the entire time
  horizon) it is at some point replaced by a new
  one.
• We can use this to develop an alternative dynamic
  programming formulation of the problem.

10
Equipment replacement regeneration point approach

• State
• k purchase period k1,,N1
• Decision
• Number of periods that you will keep a new
  machine
• Optimal value function
• S(k) minimum cost given that we purchase a new
  machine at the start of period k

11
Equipment replacement regeneration point approach

• Boundary condition
• S(N1) 0
• Recurrence relation

12
Equipment replacement regeneration point approach

• Solution to the problem

13
Equipment replacement alternative

• Network representation
• Running time
• The network has O(N2) arcs
• The cost of each arc can be computed in O(N) time
• But the cost of all arcs can be computed in
  O(N2) time
• Thus the running time of the algorithm is O(N2)

14
Equipment replacement extension 1

• Suppose now that a machine is not functional
  anymore when it reaches age M.
• In addition, suppose that we can trade-in a used
  machine on a machine of any age between 0 (new)
  and M-1.
• The cost of replacing an i year old machine by
  one of age j? is uij, where
• ui0 p ti and uii 0

15
Equipment replacement states and decisions

• State
• (k,i) (current period, age of machine)
• k1,,N1 i1,,M
• k is also the stage variable
• Initial state (1,a)
• Ending states (N1,i), i1,,M
• Decision
• Switch to a machine of age j (j0,,M-1).

16
Equipment replacement optimal value function

• Optimal value function
• f(k,i) the minimum cost of owning a machine in
  periods k,,N, starting year k with a machine
  that is i periods old
• We wish to find f(1,a)
• Boundary conditions
• f(N1,i) -si, for i1,,M

17
Equipment replacement recurrence relation

• Recurrence relation
• Running time
• The network has O(NM) nodes and O(NM2) arcs
• The cost of each arc can be determined in
  constant time
• The running time is thus O(NM2)

18
Doubling up

• For some problems, an alternative formulation
  using a technique called doubling-up can yield
  substantial savings in running time.
• Instead of increasing by 1 the duration of the
  problem solved in each iteration, we double the
  duration in each iteration.

19
Equipment replacement doubling-up

• Let the number of periods be N2L.
• Optimal value function
• S(k,i,j) the minimum cost of getting from an i
  year old machine to a j year old one in k periods
• Boundary conditions
• S(1,i,j) ui,j-1cj-1
• for i1,,M j1,,M

20
Equipment replacement doubling-up

• Recurrence relation
• for k1,,L
• This procedure has running time
• O(LM3) O( ln(N)M3 )
• Compare this with O(NM2) for the conventional
  formulation.

21
Equipment replacement doubling-up

• For durations that are not powers of 2, this
  approach may or may not be useful.
• In general, we have
• which we can use to obtain the solution for
  arbitrary durations.

22
Doubling-up

• In general, a doubling-up procedure can only be
  used if the costs are time-invariant.
• Although we have made that assumption throughout,
  the more conventional formulations can easily be
  applied to cases where the costs are time-
  (stage-) dependent.

23
Equipment replacement extension 2

• Let us return to the basic problem in which our
  decisions were to keep or replace the current
  machine.
• In addition, there now is a third alternative,
  namely to overhaul the machine.
• An overhauled machine is better than a used one,
  but not as good as a new one.
• The performance of a machine depends on its age
  as well as the number of periods since the last
  overhaul.

24
Equipment replacement

• Costs
• ekij cost of exchanging a machine of age i,
  last overhauled at age j, for a new machine at
  the start of period k
• ckij cost of operating a machine of age i, last
  overhauled at age j, during period k
• oki cost of overhauling a machine of age i at
  the start of period k
• sij salvage value received at the end of period
  N for a machine that has just turned age i and
  was last overhauled at age j
• Convention if j0, then the machine has never
  been overhauled.

25
Equipment replacement states and decisions

• State
• (k,i,j) (current period, age of machine, age at
  last overhaul)
• k1,,N1 i1,,k-1j0,,i-1
• k is also the stage variable
• Initial state (1,0,0)
• Ending states (N1,i,j), i1,,N, j0,,i-1
• Decisions
• Buy, keep, or overhaul

26
Equipment replacement optimal value function

• Optimal value function
• f(k,i,j) the minimum cost of owning a machine
  in periods k,,N, starting year k with a machine
  that is i periods old and was last overhauled at
  age j
• We wish to find f(1,0,0)
• Boundary conditions
• f(N1,i,j) -sij, for i1,,N, j0,,i-1

27
Equipment replacement recurrence relation

• Recurrence relation

28
Equipment replacement running time

• Running time
• The network has O(N3) nodes and arcs
• The cost of each arc can be computed in constant
  time
• The running time of the algorithm is O(N3)