Resource Allocation problem

1
Resource Allocation problem
A company has 3 million to invest, and three
investments are available. If dj million dollars
are invested in investment j, then a net present
value of rj (dj ) million is obtained, where the
rj (dj ) s are given as follows,
Why not LP? 1. Non negative resources allocated
(OK) 2. Return on each investment is proportional
to the resources invested (?) 3 Benefits of
individual activities are summed up (linearly
related) to obtain the total benefit (OK)
2
Resource Allocation problem

• Limited amount of resources available
• There are several activities
• Each activity produce different benefits
• Optimally allocate the resources that maximizes
  the total benefit

3
DP for Resource Allocation problem
Remember Stages gt number of investments States
gt Resources available
4
Resource Allocation problem

• Knapsack problem one-dimensional version of bin
  packing problem (find a partition of a set of
  objects such that a constraint is satisfied)
• Linear programming can be used to solve this kind
  of RAP , but it holds some assumptions (as also
  mentioned before)
• Amount of a resource assigned to an activity may
  be any nonnegative number (OK)
• The benefit obtained from the activity is
  proportional to the amount of resources assigned
  (NOT OK)
• The benefit obtained from more than one activity
  is the sum of the benefits obtained from the
  individuals (OK)

5
Equipment replacement problems

• How long a machine should be utilised before it
  is traded in for a new one
• Consider
• Cost of purchasing a new equipment
• Cost of maintaining the existing equipment which
  increases with time period
• If we trade an equipment, we will get a salvage
  value.
• Determine a replacement and trade-in policy that
  minimises the total net cost

6
Equipment replacement problems
The equipment can be traded at any given time as
follows
Remember Stages gt time steps States gt period
used
7
Equipment replacement problems
0
N-2 N-1 N
Similar formulation to Shortest path problem
8
Stochastic Dynamic programming

• In deterministic DP, given a state and the
  control variables, both cost and the next states
  are known.
• If cost or the next state is not known and given
  by a probability function then we should go for
  the stochastic DP.
• Here, we consider the expected values instead of
  deterministic values.
• Algorithm is same as deterministic case but we
  consider all possible scenarios.

9
Inventory Control example with uncertain demand
Demand is uncertain with known probability
distribution
Determine the production schedule that will meet
all demands on time and will minimize the total
cost incurred during the production period
10
Stochastic DP
11
Maximize the probability of Favourable event
occurring

• In Tennis you can have two types of serves a
  hard serve (H) and soft serve (S). The
  probability that the H and S land in bounds are
  pH and pS respectively. If the serve lands in
  bound, the probability of winning a point are wH
  and wS for H and S respectively. Select an
  optimal serving strategy that maximizes the
  probability of winning a point for a sequence of
  2 serves.

12
(No Transcript)
13
Maximize the probability of favourable event
occurring
(WHY?)
If the optimal strategy were H first and S second
14
DP to Maximize the probability of Favourable
event occurring
Stages gt number of serves States gt Type of
serve
H
H
S
S
15
DP to Maximize the probability of Favourable
event occurring
If the optimal strategy were H first and S second
16
Approximate DP

• DP solves the problems more efficiently than
  other classical methods.
• However, the required computation exponentially
  increases as the problem size increases.
• This is called curse of dimensionality
• We need some kind of approximations to make the
  problem that is tractable computationally.

17
Vehicle routing

• Consider m vehicles that move along the arcs of a
  given graph.
• Each node of the graph has a given value.
• The first vehicle that will pass through the node
  will collect the value and the vehicle pass
  subsequently will not collect the value.
• Each vehicle starts at a given node and has a
  limited number of moves.
• Find a route for each vehicle that maximizes the
  total value collected by all the vehicles

18
Vehicle routing

• This can be solved by the traditional DP
  algorithm.
• We should consider
• Current positions of the vehicles
• List of nodes that have been traversed by the
  vehicles
• List of nodes have lost their values
• Unfortunately number of states increases
  exponentially with number of nodes and number of
  vehicles.
• Approximation is carried out by one step-look
  ahead policy

19
Approximate DP for vehicle routing

• Approximation multi vehicle ? single vehicle
• We fix the order of vehicles and calculate a path
  for the first vehicle, assuming the rest of the
  vehicles are not moving
• We may use DP or heuristic solve this single
  vehicle routing
• Then calculate the path for the second vehicle in
  the same manner but consider the first vehicle
  route
• At the end we know the sub-optimal solution for
  the particular order of the vehicles
• We can consider other possible orders and find
  the routes for each vehicle that maximises the
  total value.

20
Quantizing the states and controls
21
Quantizing the states and controls

• DP can solve this problem by considering all
  possible control values at each possible state
  values.
• The number of states and the controls are large.
  Therefore, it is computationally not feasible to
  solve.
• To make it feasible, the possible state and
  control values must be quantized.

22
Quantizing the states and controls
23
Quantising the states and controls

• It can be seen from the state-stage diagram that
  all possible controls at time k for a given state
  drive to new state at k1 which is one of the
  quantized values.
• If we get any new state which does not belong to
  the set of quantised states then we need to
  calculate the optimal value by interpolation.
• Degree of approximation depends on the separation
  of the quantised values.
• Finer quantisation means greater accuracy but
  also increased computation

24
The breakthrough Problem

• Consider a binary tree with N stages. Stage k of
  the tree has 2 k nodes.
• There are two types of tree arcs free and
  blocked
• A free arc can be traversed in the direction from
  the root to the leaves.
• Objective is to break through the graph with the
  sequence of free arcs starting from root and
  ending at one of the leaves.

25
The breakthrough Problem

• DP may be used to solve this problem. However,
  amount of computation increases exponentially
  with the number of stages. O(N2N)
• Use a sub-optimal greedy method which starts
  from the root node and selects a free outgoing
  arc, if one of the outgoing arcs is free.
• If both of them are free, selects one with some
  fixed rule. O(N)

success
failed
26
Using Rollout Algorithm to breakthrough problem

• Select a sub-optimal method (base heuristic)
  which has feasible computational cost.
• At each stage, selects each node and use the
  sub-optimal method.
• Rollout the same rule until the last stage.

Greedy algorithm is selected as a Sub-optimal
method