Inventory Routing for Dynamic Waste Collection from Underground Containers

1
Inventory Routing for DynamicWaste Collection
fromUnderground Containers

• Martijn MesDepartment of Operational Methods for
  Production and LogisticsUniversity of TwenteThe
  Netherlands

Monday, November 14, 2011INFORMS Annual Meeting
2011, Charlotte, NC
2
OUTLINE

• Case introduction
• The company
• The underground container project
• Dynamic collection policies
• The Inventory Routing Problem
• Heuristic approach
• Optimization approach
• Conclusions

3
THE COMPANY

• Twente Milieu a waste collection company located
  in the Netherlands.
• Main activity collection and processing of
  waste.
• But also cleaning of streets and sewers, mowing
  of verges, road ice control, and the control of
  plague animals.
• One of the largest waste collectors in the
  Netherlands when it comes to the households
  connected to their network.
• Yearly collection of around 225,000,000 kg of
  waste from a population of around 400,000
  inhabitants.

4
(No Transcript)
5
TYPE OF CONTAINERS
Mini containers
Block containers
One per household have to be put along the side
of the road on pre-defined days.
One for multiple households mostly located at
apartment buildings freely accessible.
6
UNDERGROUND CONTAINERS
7
(No Transcript)
8
ADVANTAGES UNDERGROUND CONTAINERS

• Can be used at all places apartments, houses,
  business parks, within the city centre etc. (?
  mini containers)
• Dont have to be emptied on pre-defined days (?
  mini containers)
• Much larger then the block containers (typically
  5m3 which is 5 times the volume of a block
  container)
• Only accessible with a personal card
• Avoids illegal waste deposits (? block
  containers)
• Enables the introduction of Diftar charging
  waste disposal at different rates per kg
  depending on the type of garbage
• Less odour nuisance due to solid locking (? block
  containers)
• Contributes to an attractive environment (? block
  containers)

9
(No Transcript)
10
USING THE UNDERGROUND CONTAINERS

• Between 2009 and 2011, around 700 underground
  containers have been installed 800 new
  containers will be added soon.
• Containers are equipped with a motion sensor the
  number of lid openings are communicated to Twente
  Milieu.
• There is a static cyclic schedule that states
  which containers have to be emptied on what day.
  For example container X has to be emptied every
  Tuesday and container Y has to be emptied on
  Friday once in the two weeks.
• Every workday, a planning employee assigns trucks
  and drivers to the pre-defined containers. On
  Fridays, the planner uses the sensor information
  to include some additional urgent containers,
  thereby slightly deviating from the static cyclic
  schedule.
• Why not using this sensor information for the
  whole selection process?

11
DYNAMIC WASTE COLLECTION

• Dynamic planning methodology each day, select
  the containers to be emptied based on their
  estimated fill levels (using sensor information).
  
• Research objective
• To asses in what way and up to what degree a
  dynamic planning methodology can be used by
  Twente Milieu to increase efficiency in the
  emptying process of underground containers in
  terms of logistical costs, customer satisfaction,
  and CO2 emissions.

12
INVENTORY ROUTING PROBLEM

• In the literature, our problem is known as a
  Inventory Routing Problem (IRP) which combines
• The vehicle routing problem (VRP)
• Inventory Management \ Vendor Managed Inventory
  (VMI)
• Trade-off decisions
• When to deliver a customer?
• How much to deliver a customer?
• Which delivery routes to use?
• The current cyclic planning approach relates to
  the Periodic Vehicle Routing Problem (PVRP)
• A multi-period VRP where customers have to be
  visited a given number of times within a given
  planning horizon (decision on visit combinations
  and routes).

13
ILLUSTRATION OF THE IRP

• Basic question for IRPs which customers to serve
  today and how to route our trucks?

Enough empty space left
Depot
Empty space needs to be delivered soon
Parking
14
SOLUTION METHODOLOGIES FOR IRPs

• ILP\SDP\MDP\Heuristics
• Federgruen and Zipkin (1984), A Combined Vehicle
  Routing and Inventory Allocation Problem.
• Campbell et al. (1997), The Inventory Routing
  Problem.
• Bard et al. (1998), A Decomposition Approach to
  the Inventory Routing Problem with Satellite
  Facilities.
• Chan et al. (1998), Probabilistic Analyses and
  Practical Algorithms for Inventory-Routing
  Models.
• Berman et al. (2001), Deliveries in an
  inventory/routing problem using stochastic
  dynamic programming.
• Kleywegt et al. (2002), The Stochastic inventory
  routing problem with direct deliveries.
• Adelman (2004), A Price-Directed Approach to
  Stochastic Inventory/Routing.
• Campbell et al. (2004), A decomposition approach
  for the inventory-routing problem.
• Kleywegt et al. (2004), Dynamic programming
  approximations for a stochastic inventory routing
  problem.
• Archetti et al. (2007), A branch-and-cut
  algorithm for a vendor-managed inventory-routing
  problem.
• Bard et al. (2009), The integrated
  productioninventorydistributionrouting problem.

15
OUR SOLUTION METHODOLOGY

• Some characteristics of our problem
• Multi-vehicle up to 7 trucks.
• Multi-depot 2 parking areas and 1 waste
  processing center.
• Large-scale expanding to 1500 customers
  (containers), which requires gt 300 visits per
  day.
• Long planning horizon a short-term planning
  approach will postpone deliveries to the next
  period.
• Dynamic environment stochastic travel times and
  waste disposals ? we have to be able to do
  replanning.
• Changing environment seasonal patters and
  special days.
• To cope with these characteristics, we use a fast
  heuristic.
• To anticipate changes in waste disposal, we equip
  our heuristic with a number of tunable parameters
  and optimize over these parameters.

16
BASIC IDEA OF THE HEURISTIC

• Create initial routes based on MustGos (seed
  customers and workload balancing) and extend
  these routes with MayGos.

MayGo
MustGo
Depot
Parking
17
BASIC IDEA OF THE HEURISTIC

• Create initial routes based on MustGos (seed
  customers and workload balancing) and extend
  these routes with MayGos.

Seed
Depot
Parking
18
BASIC IDEA OF THE HEURISTIC

• Create initial routes based on MustGos (seed
  customers and workload balancing) and extend
  these routes with MayGos.

Depot
Parking
19
BASIC IDEA OF THE HEURISTIC

• Create initial routes based on MustGos (seed
  customers and workload balancing) and extend
  these routes with MayGos.

Depot
Parking
20
BASIC IDEA OF THE HEURISTIC

• Create initial routes based on MustGos (seed
  customers and workload balancing) and extend
  these routes with MayGos.

Depot
Parking
21
BASIC IDEA OF THE HEURISTIC

• Create initial routes based on MustGos (seed
  customers and workload balancing) and extend
  these routes with MayGos.

Depot
Parking
22
BASIC IDEA OF THE HEURISTIC

• Create initial routes based on MustGos (seed
  customers and workload balancing) and extend
  these routes with MayGos.

Depot
Parking
23
BASIC IDEA OF THE HEURISTIC

• Create initial routes based on MustGos (seed
  customers and workload balancing) and extend
  these routes with MayGos.

Depot
Parking
Extended with MayGos
24
ALGORITHM OUTLINE
1. Start

1. Initial planning in the morning and replanning
   during the day.
2. Empty schedules in a non-preemtive way and keep
   them feasible.
3. Estimate the days left MustGos (days left lt
   MustGoDay) optional workload balancing (to avoid
   peaks on Mondays and Fridays) trucks to use
   lower bound on the number of routes to use.
4. One seed per truck to (i) spread trucks across
   the area, (ii) realize container insertions both
   close and far from the depot, and (iii) balance
   the workload per route to anticipate later MayGo
   insertions seeds based on largest minimum
   distance from the depot and other seeds Assign
   routes to trucks.
5. Optionally, assign MustGos to trucks or routes
   in a balanced way (in anticipation of MayGo
   insertions).
6. Plan all remaining MustGos based on cheapest
   insertion costs.
7. Play MayGos see next sheet.
8. Execute planning and perform replanning when
   needed.

2. Initialize schedules
3. Initial computations
4. Plan seeds
5. Balance workload
6. Plan MustGos
7. Plan MayGos
8. End
25
ADDING MAYGO CONTAINERS

• MayGos days left lt MustGoDayMayGoDay.
• Planning extremes
• Wait first MayGoDay0
• Drive first MayGoDay8
• The best option would be somewhere in between.
• Selection of MayGos depend on the additional
  travel time (insertion costs) as well as the
  inventory (volume garbage).
• Options
• Ratio insertion costs / inventory.
• Relative improvement of this ratio compared to a
  smoothed historical ratio. A large positive value
  indicates an opportunity we should take.
• Use (optional) limit on the number of MayGos.

26
WILL IT WORK? A SIMULATION STUDY

• Benchmark the current way of working and gain
  insight in the performance of our heuristic

27
NUMERICAL RESULTS

• Based on current deposit volumes and truck
  capacity, savings of 14.6 can be achieved, which
  consists of 40 reduction of penalty costs and
  18 less travel distance.
• Savings increase with decreasing truck
  capacities.

28
OBSERVATIONS

• Performance heavily depends on the parameter
  settings
• MustGoDay
• MayGoDay
• MaxPerDay (to limit MayGos)
• NrTrucks
• Slack capacity in trucks (to avoid replanning)
• Etc.
• Moreover, the right settings for these
  parameters heavily depend on the day of the week.
• We could learn these parameters
• Through experimentation in practice (online
  learning)
• Through simulation experiments (offline learning)

29
STOCHASTIC SEARCH

• Where is the min\max of some multi-dimensional
  function when the surface is measured with noise?
• In our case at least a 10 dimensional function
  (using only the parameters MustGoDay and MayGoDay
  for 5 workdays).

30
SIMULATION OPTIMIZATION

• The optimization problem
• Simulation optimization
• The measurements follow from a simulation run.
• Hence, these measurements are expensive.
• Hence, we aim to reduce the required number of
  measurements.
• Approaches Heuristic methods (genetic
  algorithms, simulated annealing, tabu search
  etc.) Response Surface Methods (RSM) Stochastic
  Approximation (SA) methods Bayesian Global
  Optimization (BGO).

Vector or parameters to be adjusted (MustGoDay,
MayGoDay, NrTrucks, etc., for all working days)

• Unknown function (no closed-form formulation)
• We can measure it
• Measurement will not be exact (we measure with
  noise yf(x)e)

Set of all parameter combinations
31
BAYESIAN GLOBAL OPTIMIZATION

• Bayesian optimization involves three stages
• Designing the prior distribution (belief about f)
• Updating this distribution using Bayes' rule
• Deciding what values to sample next
• Often, the belief about f conforms to a Gaussian
  process.
• A Gaussian process is a collection of random
  variables yx1, yx2, for which any finite
  subset has a joint multivariate Gaussian (Normal)
  distribution

Measurements
Kernel function (covariance between two variables)
Mean
32
MORE INFORMATION ON BGO

• Daniel Lizotte (2008)Practical Bayesian
  Optimization, PhD Thesis.
• Eric Brochu, Mike Cora and Nando de Freitas
  (2009)A Tutorial on Bayesian Optimization of
  Expensive Cost Functions, with Application to
  Active User Modeling and Hierarchical
  Reinforcement Learning.
• INFORMS Tutorial by Peter Frazier today from
  1630-1800Bayesian Methods for Global and
  Simulation Optimization.

33
OPTIMIZATION POLICIES WE CONSIDER

• Sequential Kriging Optimization (SKO) by Huang et
  al. (2006) which is an extension of Efficient
  global optimization (EGO) by Jones et al. (1998)
  for noisy measurements. EGO new points to be
  measured are selected based on expected
  improvement which strikes a balance between
  exploitation and exploration.
• Knowledge Gradient for Correlated Beliefs (KGCB)
  by Frazier et al. (2009). KG best we can do
  given we if there is only one measurement left to
  make.
• Hierarchical Knowledge Gradient (HKG) by Mes et
  al. (2011). HKG hierarchical aggregation
  technique that uses the common features shared by
  alternatives to learn about many alternatives
  from even a single measurement.

34
ILLUSTRATION OF EGO N2
Source Brochu et al. (2009)
35
ILLUSTRATION OF EGO N3
Source Brochu et al. (2009)
36
ILLUSTRATION OF EGO N4
Source Brochu et al. (2009)
37
ILLUSTRATION OF EGO N5
Source Brochu et al. (2009)
38
ILLUSTRATION OF HKG EXCEL DEMO
39
APPLICABILITY OF THESE POLICIES

•  

40
EXPERIMENTS WITH SKO

• Experiment 1 378 containers with 3 trucks
• with a maximum of 113 emptying's per day.
• Experiment 2 700 containers, 50 higher deposit
  volumes and 2 trucks
• with a maximum of 672 emptyings per day.
• Results are counterintuitive at first sight.
  Still, they result in additional savings of
  around 10.

Mon Tue Wed Thu Fri
MustGoDay 4.0 0.0 0.0 1.2 0.0
MayGoDay 4.0 X X 3.5 X
Mon Tue Wed Thu Fri
MustGoDay 1.0 1.1 1.5 2.7 2.1
MayGoDay 0.0 0.0 4.0 4.0 4.0
41
CONCLUSIONS

• We proposed a fast heuristic suitable for
  Inventory Routing Problems involving a large
  number of customers.
• Application of this heuristic to the waste
  collection problem is expected to result in a
  reduction of 18 in travel costs and 40 in
  penalty costs (due to waste overflow).
• An optimization approach is preferred to
  anticipate changes in waste disposals. To enable
  this, we equipped our heuristic with several
  tunable parameters.
• To optimize over these parameters we used
  techniques from Simulation Optimization and
  Bayesian Global Optimization (SKO, KGCB, HKG).
• For our waste collection problem, this will
  result in additional savings of 10 in total
  costs (travel costs and penalty costs).

42
QUESTIONS?

• Martijn Mes
• Assistant professor
• University of Twente
• School of Management and Governance
• Operational Methods for Production and Logistics
• The Netherlands
• Contact
• Phone [phone]
• Email m.r.k.mes_at_utwente.nl
• Web http//www.utwente.nl/mb/ompl/staff/Mes/