Operations Research: Optimization, Heuristics, Simulation, and Forecasting

1
Operations ResearchOptimization, Heuristics,
Simulation, and Forecasting
Judy Pastor Continental Airlines
2
Operations Research

• application of mathematical techniques, models,
  and tools to a problem within a system to yield
  the optimal solution
• Phases of an OR Project
• formulate the problem
• develop math model to represent the system
• solve and derive solution from model
• test/validate model and solution
• establish controls over the solution
• put the solution to work

3
Linear Programs - Major Tool of OR

• Linear Programs (LPs) are a special type of
  mathematical model where all relationships
  between parts of the system being modeled can be
  represented linearly (a straight line).
• Not always realistic, but we know how to solve
  LPs.
• May need to approximate a relationship that is
  slightly non-linear with a linear one.
• When to use if a problem has too many
  dimensions and alternative solutions to evaluate
  all manually, use an LP to evaluate.

4
Linear Programs - Major Tool of OR

• LPs can evaluate thousands, millions, etc. of
  different alternatives to find the one that best
  meets the objective of the business problem.
• Fleet Assignment Model - assign aircraft to
  flight legs to minimize cost and maximize revenue
• Revenue Management - set bid prices to maximize
  revenue and/or minimize spill
• Crew Scheduling - schedule crew members to
  minimize number of crew needed and maximize
  utilization

5
Linear Program Formulation

• Understand the system and environment to which
  the problem belongs
• Understand the problem and the objective to be
  achieved
• State the model - clear idea of problem and what
  can and can not be included in the model
• Collect Data - get data/parameters/constraints
  and boundaries of system and interrelationships
• Determine decisions - define decision variables -
  what do we need the model to tell us?
• Formulate and solve model

6
Example RM Network LP

• Problem - how many passengers of each itinerary
  and fare class should be accepted on each flight
  to achieve the maximum revenue for the flight
  network?
• Statement - the model should tell us the above
• Data - demand by itinerary/fare class, aircraft
  capacity, overbooking levels, expected revenue by
  itinerary/fare class
• Decisions - how many passengers of each
  itinerary/fare class to accept on each flight leg

7
Example RM Network LPData Collection

• Two Flights SFO-IAH, IAH-AUS
• Two fare classes Y-high fare, Q-low fare
• Three itineraries SFOIAH, IAHAUS, SFOAUS
• Six fares Y Q
• SFOIAH 400 300
• IAHAUS 250 100
• SFOAUS 450 350
• Flight capacity SFO-IAH 124, IAH-AUS 94
• No overbooking

8
Example RM Network LPData Collection
(continued)

• Demand Y Q
• SFOIAH 30 90
• IAHAUS 50 30
• SFOAUS 20 50

9
Example RM Network LPFormulate Model

• Definition of Data
• F set of flights SFOIAH, IAHAUS
• f index of F (1,2)
• CAPf capacity of flightf 124, 94
• I set of itineraries SFOIAH, IAHAUS, SFOAUS
• i index of I (1,2,3)
• IFf set of itineraries over flight f
• IF1SFOIAH,SFOAUS IF2IAHAUS,SFOAUS
• C set of classes Y, Q
• c index of C (1,2)
• DMDi,c demand for itinerary i and class c
• FAREi,c fare for itinerary i and class c

10
Example RM Network LP Formulate Model

• Now that we have defined all the data that we
  know about the model, we now must define what we
  want the model to tell us.
• Problem - how many passengers of each itinerary
  and fare class should be accepted on each flight
  to achieve the maximum revenue for the flight
  network?
• Define decision variables
• Xi,c pax accepted for itinerary i and class c
• There are 3 itineraries and 2 classes so there
  are a total of 6 decision variables.

11
Example RM Network LP Formulate Model

• Many sets of values (collectively called
  solutions) for the six Xi,c variables exist which
  could satisfy the constraints (formulation
  coming) of aircraft capacity and maximum demand.
  These are feasible solutions.
• Which solution do we want?
• Problem - how many passengers of each itinerary
  and fare class should be accepted on each flight
  to achieve the maximum revenue for the flight
  network?
• The feasible solution for this is optimal.

12
Example RM Network LP Objective Function
13
Example RM Network LP Objective Function
Constraints

• The Objective Function is an expression that
  defines the optimal solution, out of the many
  feasible solutions. We can either
• MAXimize - usually used with revenue or profit or
• MINimize - usually used with costs
• Feasible solutions must satisfy the constraints
  of the problem. LPs are used to allocate scarce
  resources in the best possible manner.
  Constraints define the scarcity.
• The scarcity in this problem involves a fixed
  number of seats and scarce high paying customers.

14
Example RM Network LP Constraints
15
Example RM Network LP Constraints

• Rules for Constraints
• must be a linear expression
• decision variables can be summed together but not
  multiplied or divided by each other
• have relational operators of , lt, or gt
• must be continuous
• Constraints define the feasible region - all
  points within the feasible region satisfy the
  constraints.
• The feasible region is convex.
• The optimal solution lies at an extreme point of
  the feasible region.

16
Example RM Network LPCplex Input File

• MAX
• 400 X_SFOIAH_Y 300 X_SFOIAH_Q
• 250 X_IAHAUS_Y 100 X_IAHAUS_Q
• 450 X_SFOAUS_Y 320 X_SFOAUS_Q
• ST
• CAPY_SFOIAH X_SFOIAH_Y X_SFOIAH_Q
  X_SFOAUS_Y X_SFOAUS_Q lt 124
• CAPY_IAHAUS X_SFOAUS_Y X_SFOAUS_Q
  X_IAHAUS_Y X_IAHAUS_Q lt94
• DMD_SFOIAH_Y X_SFOIAH_Y lt 30
• DMD_SFOIAH_Q X_SFOIAH_Q lt 90
• DMD_IAHAUS_Y X_IAHAUS_Y lt 50
• DMD_IAHAUS_Q X_IAHAUS_Q lt 30
• DMD_SFOAUS_Y X_SFOAUS_Y lt 20
• DMD_SFOAUS_Q X_SFOAUS_Q lt 50
• END

17
Example RM Network LPSolution - Constraints

• SECTION 1 - ROWS
• NUMBER ......ROW....... AT ...ACTIVITY...
  SLACK ACTIVITY ..LOWER LIMIT. ..UPPER LIMIT.
  .DUAL ACTIVITY
• 1 obj BS 58100
  -58100 NONE NONE
  1
• 2 CAPY_SFOIAH UL 124
  0 NONE 124
  -300
• 3 CAPY_IAHAUS UL 94
  0 NONE 94
  -100
• 4 DMD_SFOIAH_Y UL 30
  0 NONE 30
  -100
• 5 DMD_SFOIAH_Q BS 74
  16 NONE 90
  0
• 6 DMD_IAHAUS_Y UL 50
  0 NONE 50
  -150
• 7 DMD_IAHAUS_Q BS 24
  6 NONE 30
  0
• 8 DMD_SFOAUS_Y UL 20
  0 NONE 20
  -50
• 9 DMD_SFOAUS_Q BS 0
  50 NONE 50
  0
• obj is the objective function value - total
  revenue from the small network of flights
• CAPY_SFOIAH and CAPY_IAHAUS are the capacity
  constraints. Both are at UL - upper limit with
  activities of 124 and 94, respectively (i.e.
  both flight legs are full).
• Dual Activity on each capacity constraint is
  also known as the Shadow Price of the flight.
  The SP of SFOIAH is 300 and the SP of IAHAUS is
  100. In RM terms, this means that the value of
  one more seat on SFOIAH is 300 and the value of
  one more seat on IAHAUS is 100. Alternately,
  300 and 100 also define the lowest fare that
  should be accepted on each leg.
• DMD_SFOIAH,IAHAUS_Y,Q are the demand
  constraints. SFOIAH_Y, IAHAUS_Y, and SFOAUS_Y
  are at upper level (i.e. accept all Y
  passengers). Reject some/all of Q.

18
Example RM Network LPSolution - Decision
Variables

• SECTION 2 - COLUMNS
• NUMBER .....COLUMN..... AT ...ACTIVITY...
  ..INPUT COST.. ..LOWER LIMIT. ..UPPER LIMIT.
  .REDUCED COST.
• 10 X_SFOIAH_Y BS 30
  400 0 NONE
  0
• 11 X_SFOIAH_Q BS 74
  300 0 NONE
  0
• 12 X_IAHAUS_Y BS 50
  250 0 NONE
  0
• 13 X_IAHAUS_Q BS 24
  100 0 NONE
  0
• 14 X_SFOAUS_Y BS 20
  450 0 NONE
  0
• 15 X_SFOAUS_Q LL 0
  320 0 NONE
  -80
• This section of the solution report shows the
  values for the decision variables at the optimal
  solution.
• The LP tells us to accept 30 SFOIAH Y, 74 SFOIAH
  Q (reject 16), accept 50 IAHAUS Y, accept 24
  IAHAUS Q (reject 6), accept 20 SFOAUS Y, and
  accept no SFOAUS Q.
• Note that the LP cut off all SFOAUS Q booking
  requests because their fare of 320 is less than
  the sum of the shadow prices of the two flights
  (300100 400 gt 320).

19
Example RM Network LPSolving LPs

• A problem that sounds small, like our example,
  can balloon out into many decision variables and
  constraints.
• Computer software is available to solve linear
  programs.
• Cost of programs depends on size of problems to
  be solved.
• Excel has an Add-in to solve small LPs.
• CPLEX is state of the art, but more expensive.
• LPs with 100,000s row and columns can be solved.

20
Example RM Network LPSolving LPs

• The first method of solving LPs was invented
  during WWII by George Dantzig. The algorithm is
  called SIMPLEX. It is based on convexity theory
  and that the optimal solution will occur at an
  extreme point of the solution space
• Newer state of the art algorithms are based on
  steepest descent gradient methods and are called
  interior point methods
• Interior point methods can be extremely fast
  (much faster than SIMPLEX) for certain structures
  of problems

21
Degeneracy

• When an LP has more than one unique way to reach
  an optimal objective function value, we say that
  the problem is degenerate
• LP solvers can detect degeneracy but only report
  one solution
• It would be nice to see all possible solutions
• Different solvers can land on different
  solutions of a degenerate problem, depending on
  solution strategy
• The RM Network problem is usually degenerate

22
Other Types of Linear Optimizations

• MIP (Mixed Integer Programming)
• is similar to LP but at least one decision
  variable is required to be a integer value
• violates the LP rule that decision variables be
  continuous
• is solved by branch and bound - solving a
  series of LPs that fix the integer decision
  variables to various integer values and comparing
  the resulting objective function values
• is done in a smart way to avoid enumerating all
  possibilities
• is useful, since you can not have .3 of an
  aircraft

23
Other Types of Linear Optimization

• Network problem
• is a special form of LP which turns out to be
  naturally integer
• can be solved faster than an LP, using a special
  network optimization algorithm
• is very restrictive on types of constraints that
  can be present in the problem
• Shortest Path
• finds the shortest path from the source (start)
  to sink (end) nodes, along connecting arcs, each
  having a cost associated with them
• is used in many applications

24
Other Optimization Models

• Quadratic Program
• has a quadratic objective function with linear
  constraints
• can be applied to revenue management, because it
  allows fare to rise with demand within a problem
• price(OD) 50 5numpax(OD)
• max revenue price numpax

25
Other Optimization Models

• Non-linear Program (NLP)
• can have either non-linear objective function or
  non-linear constraints or both
• feasible region is generally not convex
• much more difficult to solve
• but it is worth our time to learn to solve them
  since world is actually non-linear most of the
  time
• some non-linear programs can be solved with LPs
  or MIPs using piecewise linear functions

26
Deterministic versus Stochastic

• Two broad categories of optimization models exist
• deterministic
• parameters/data known with certainty
• stochastic
• parameters/data know with uncertainty
• Deterministic models are easier to solve. Our RM
  LP is deterministic (we pretend we know the
  demand with certainty).
• Stochastic model are difficult to solve. In
  reality, we know a distribution about our demand.
  We get around this in real life by re-optimizing.

27
Deterministic versus Stochastic

• Deterministic optimization ignores risk of being
  wrong about parameter/data estimates.
• No commercial software packages are currently
  available to do generalized, stochastic
  optimization.

28
Heuristics

• Definition - educated guess
• When you use a heuristic to solve a problem, you
  have a gut feeling that it is a pretty good
  solution, but can not prove it mathematically
• You can not prove that there is not a better
  solution out there
• To qualify as an optimal solution, there must
  be a mathematical proof to say that no better
  solution exists

29
Types of Heuristics

• Greedy Algorithms - also called myopic -
  nearsighted solutions.
• Example in our RM Network LP, the greedy
  solution would be to take the highest fare
  passengers possible on each leg, without looking
  at the consequences of doing so on the connecting
  leg. So the greedy solution is to take the
  SFOAUS Q passengers at a fare of 320. But the
  optimal solution looks at displacement and says
  do not take any SFOAUS Q passengers.

30
Heuristics

• Combinatorial problems can grow exponentially
  when the number of decisions needed to be made
  grows linearly. Heuristics can be used in these
  cases to get a good solution in a reasonable
  amount of time.
• TSP - Travelling Salesman Problem is a good
  example of this.
• EMSR is a heuristic. It is provably optimal for
  two fare classes, but not more. However, it
  gives a good answer in a finite amount of time
  and takes probability into account.

31
Simulation

• When a problem is too complicated to be put into
  an LP or a solvable non-linear optimization, one
  way to study the problem is to simulate it under
  different conditions.
• PODS (Passenger O D Simulation) is one.
• Simulation can tell us something about a set of
  parameters (i.e. total revenue, load factor), but
  does not point us in the direction of an
  improvement.

32
Forecasting

• Types of forecasting techniques used in RM
• pick-up
• booking regression
• exponential smoothing
• Pick-up - adds average future bookings from
  historical observations to bookings on hand.
• Booking Regression - computes best fit for
  history of bookings on hand (independent) to
  final booked (dependent)
• final booked a b(bookings on hand)

33
Forecasting

• Exponential Smoothing - similar to pick-up except
  the average is weighted. The most recent
  historical observations are weighted most
  heavily, decreasing for earlier observations.
• recursive relationship
• Avg Pick Up a (pick upt-1) (1-a)2 (pick
  upt-2) ...
• boils down to
• Avg Pick Up a (pick upt-1) (1-a) (last
  fcst pick up)
• Problem is how to estimate a. 0ltalt1
• How much weight should be on most recent
  observation?