HFE 730: Advanced Modeling Techniques

1
HFE 730Advanced Modeling Techniques
2
Course Details

• Objectives of Course
• Course Policies
• Grading Components
• Textbook (none)
• Readings

3
Recap of Modeling

• Why do we create models?
• What are usual models employed?
• Optimization models
• Statistical models
• Simulation models
• Physical models (not really going to be our
  focus)
• What is our goal in using these models?

4
Modeling Defined

• A simplified representation of a system or
  phenomena (ref Websters)
• To simulate commonly with the aid of a computer
  (ref Websters)
• Set of mathematical relationships and logical
  assumptions implemented in a computer as a
  representation of some real-world decision
  problem or phenomenon (Ref Ragsdale)

5
Current Modeling

• Mathematical modeling
• Capture the system or phenomena in some set of
  algebraic structures
• Represent the resulting model in a computer via
  some modeling language
• GAMS, X-Press, Excel
• Find a solution to the model
• Typically codes for linear, nonlinear and
  integer
• Very much a prescriptive decision aiding
  technique

6
Current Modeling

• Statistical modeling
• Capture some data and build a mathematical model
  that accurately represents the data
• Regression is popular and well known method
• Many others
• Time series and smoothing models
• Neural networks
• Multi-variate data analysis techniques
• Very much a predictive technique
• Not going to spend much time on these techniques

7
Current Modeling

• Simulation modeling
• A computer program depicting the interactions
  among entities over time
• Monte Carlo sampling techniques and risk analysis
• Discrete event simulation
• Object-oriented simulation
• Very much a descriptive technique
• Can be thought of as applied statistics

8
Whats Wrong with Current Methods

• Are there limitations to our current suite of
  modeling tools and techniques?
• Hopefully you will answer YES.
• Are there emerging techniques to help overcome
  some of these limitations?
• Again, the answer is YES
• The intent of this course is to cover some of
  these
• Do these new techniques bring along new sets of
  challenges?

9
Limitations of Current Methods

• Mathematical modeling limitations include
• Desire to model more complex problems
• Ability to find multiple good solutions
• Ability to learn through the process
• Ability to accommodate particularly nasty
  computational forms and search nasty landscapes
• Ability to obtain solutions to both tractable and
  less tractable problems in a reasonable amount of
  time
• Increased computing capability driving some of
  these concerns/limitations

10
New Mathematical Methods

• Heuristic optimization methods have long history,
  although approaches have changed
• Newer heuristic methods resemble computer
  programs for optimization search
• Handle more complex problems
• Find lots of good solutions
• Overcome classical method limitations in terms of
  local optima, multiple optima, nasty problem
  formulations, and solution response time

11
Methods Covered

• Simulated Annealing
• Search method motivated by annealing process of
  metals
• Explicit use of randomization in the search
  process
• Genetic Algorithms
• Search method motivated by biological systems and
  survival of the fittest
• Very generic approach to solving problems

12
Methods Covered

• Tabu Search
• Search method motivated by human problem solving
• Explicit use of memory in the search process
• Ant Colony Algorithms
• Search method motivated by biological systems and
  how ants find a shortest path
• Very useful approach to solving routing problems

13
Methods Covered

• Scatter Search
• Search method motivated by simplex search
• Explicit use of memory in the search process
• Method found in commercial package, OptQuest
• Memetic Algorithms
• Genetic algorithms except learning is passed on
• Add a local search on new members of the
  population

14
Limitations of Current Methods

• Simulation modeling limitations include
• Desire to develop more complex models but at the
  same time retain model tractability
• Ability to model humans and their behaviors
• Want to provide better range of outputs to get a
  better grip on the potentials in the future
• Too many models are overly complex and thus
  provide outputs that can be hard to interpret

15
New Simulation Methods

• Object-oriented simulation
• Distributed simulation
• Interactive simulation
• Complex adaptive simulations/systems
• Agent-based simulation
• Simulation-based optimization
• Try and wrap a method around a simulation to
  guide the search for a best set of parameters

16
Mathematical Modeling Concepts
17
Mathematical Modeling

• Describe system with set of algebraic equations
• Capture key relationships within the system
• Capture key behaviors in system
• Decisions for which insight needed are decision
  variables
• Goal embedded within the objective function
• Limitations/restrictions in constraints
• Physical constraints
• Logical constraints

18
Simple Example Model

• Objective Determine production mix that
  maximizes the profit under the raw material
  constraint and other production requirements
  (detailed next).
• Maximize 50D 30C 6 MSubject to 7D 3C
  1.5M lt 2000 (raw steel) D gt 100 (contract
  ) C lt 500 (cushions available) D, C, M
  gt 0 (Non-negativity) D and C are integers

19
Linear Programming

• Assumptions of the linear programming model
• The parameter values are known with certainty.
• The objective function and constraints exhibit
  constant returns to scale.
• There are no interactions between the decision
  variables (the additivity assumption).
• The Continuity assumption Variables can take on
  any value within a given feasible range.

20
Galaxy Industries(Lawrence and Pasternak)
21
Feasible Region of LP
X2
1000
Profit 4360
700
500
X1
500
22
Integer Programming

• Many real life problems call for at least one
  integer decision variable.
• There are three types of Integer models
• Pure integer (AILP)
• Mixed integer (MILP)
• Binary (BILP)
• Unfortunately, these get quite hard to solve

23
Descriptive Constraints(Lawrence and Pasternack)
24
Salem City Example(Lawrence and Pasternack)
25
Solution Methods

• Linear models Simplex algorithm
• Fast in practice
• Lots of good sensitivity analysis information
• Integer models Branch and bound
• Can take very long time
• No sensitivity information
• Can be sped up with specific knowledge
• Nonlinear models
• Variety of solution methods

26
Multidimensional Knapsack Problem (MKP)
27
Neighborhood Search

• Understanding Some Common Features

28
Purpose of This Portion

• Tie together basic concepts of numerical search
  (NS)
• Heuristics are simply numerical search methods
• Provide an understanding of the basic local
  search concepts
• Set the stage for the hands-on portion
• We will use a simple optimization example to
  demonstrate the concepts and then extend these
  concepts with further discussion

29
Commonalities of NS

• Iterative improvement
• Neighborhoods of solutions
• Improving directions
• Space assumptions

30
Iterative Improvement

• NS methods move from solution to solution
• Population methods look to improve the population
• Goal is to continually improve solutions until
  local optima found
• Amount of improvement can vary each iteration
• Stop when improvement slows
• Either within some bound, some range, or time
• In general number of iterations required unknown

31
Solution Neighborhoods

• A key concept in all NS methods
• For an LP, adjacent corner points
• For NLP, some continuous region about the current
  point
• For heuristics, points within a move distance
• Since moves are generic, neighborhoods change
• In general, all points attainable via some
  prescribed solution characteristic change
  comprise the current solution neighborhood

32
Improving Directions

• Goal is to converge to local optima
• Simplex directions move along some edge whose net
  benefit is largest
• NLP directions vary but generally seek one of
  descent (ascent)
• Gradient-based, actual or estimated
• Zero-order through second-order information
• Heuristicsgenerally zero order methods
• Include non-improving moves
• In general, we want an aggressive search, one
  that improves the solution

33
Differences among NS

• Space assumptions
• Use of past knowledge
• Generality of method

34
Solution Space Assumptions

• LPs have convexity due to strict linear nature of
  objective function and constraints
• NLP primary assumptions can be
• Continuity and differentiable in local region
• Reasonableness of Taylor series estimates
• Heuristics make very few space assumptions
• Recognize and accommodate multiple local optima
• Avoid having to explicitly re-start a search
  method

35
Use of Past Knowledge

• LPs really do not need this
• Convexity ensures improvement to optimal
• NLPs use little
• Some in conjugate gradient methods
• Some in estimates of gradients or Hessians
• An explicit part of heuristics
• Provides a level of adaptation not available in
  classic methods
• By using past knowledge can we infer new and
  improved numerical search methods?

36
Generality of Method

• Classic methods have few tunable parameters
• Usually convergence parameters or step size
• Heuristic methods represent class of problems
• Tuning period provides improved performance on
  general class of problems
• Neighborhoods can be defined in a variety of
  fashions
• Homework assignments will cover these topics
• Most articles will discuss parameter tuning

37
Heuristic Characteristics

• Iterative numerical search techniques
• Employ some evaluation function
• Neighborhood structure local exploration
• More flexibility with respect to feasibility
• A requirement to parameterize the search
• Local optimality escape mechanism
• No guarantee of global optimality
• Although one can point to convergence results for
  some guarantee

38
Questions?
39
Methods Covered

• Agent-based optimization
• This is an area that will actually cross the
  fields of optimization and of simulation
• Agent-based modeling
• Agent-based simulation
• Complex adaptive systems simulation