Principles for Successful Simulation

1
Principles for Successful Simulation

• Jerry Banks
• Atlanta, GA

2
A man went into a restaurant
3
Definition of Simulation

• Simulation is the operation of a real-world
  process or system over time.
• Simulation involves the generation of an
  artificial history of the system, and the
  observation of that artificial history to draw
  inferences concerning the operating
  characteristics of the real system that is
  represented.

4
Definition of Simulation

• One of the top 3 technologies
• Simulation is no longer the technique of last
  resort
• It is an indispensable problem-solving methodology

5
Definition of Simulation

• We use simulation to
• Describe and analyze the behavior of a system
• Ask what if questions about the real system
• Aid in the design of real systems
• Existing and conceptual systems
• We model both kinds

6
Ad Hoc Simulation

• Mechanics arrive for service between one and ten
  minutes apart in time
• Mechanics are served in a time between one and
  six minutes
• Integer values
• Simulate for twenty mechanics

7
Ad Hoc Simulation Table
Mechanic Time Arrival Service Service Time Time I
dle Time between Time Time begins Service in Time
in Arrivals Ends System Queue ______________
__________________________________________________
__________________________________________________
_________1 - 0 2 0 2 2 0 0 2 5 5 2 5 7 2 3 0 3 1 6
6 7 13 7 0 1 20 7 98 1 98 99 1 4 0 72
34 7
8
Performance Measures

• Avg time in system 72/20 3.6 minutes
• idle time (34/99)100 34
• Avg waiting time/mechanic 7/20 0.35 minutes
• Fraction having to wait 3/20 0.15
• Avg waiting time of those that waited 7/3
  2.33 minutes

9
Steps in the simulation process

• 1. Problem formulation
• Prepare a statement of what is to be
  accomplished, the goals of the project, the
  questions to be answered, and the scenarios to be
  studied

10
Steps in the simulation process

• 2. Model conceptualization
• Abstract the system into a model described by the
  interaction between elements of the system

11
Steps in the simulation process

• 3. Data collection
• Gather the data that support the modeling
  objectives

12
Steps in the simulation process

• 4. Model translation
• Capture the essence of the conceptual model in a
  simulation language

13
Steps in the simulation process

• 5. Verification and validation
• Have we solved the problem right?
• Have we solved the right problem?

14
Steps in the simulation process

• 6. Analyze the output
• Draw inferences and make recommendations for
  solving the problem

15
Steps in the simulation process

• 7. Documentation and reporting
• Supply evidence that the simulation study was
  conducted appropriately

16
Steps in the simulation process

• 8. Implementation
• Present the findings to management

17
1. Problem formulation

• Guidelines
• First impression
• Avoid a g error
• Manage expectations
• Avoid judgment
• Communicate
• Get ahead of yourself

18
First impression

• A strong finish begins with a successful
  beginning
• Meet with the customers
• Introduce the simulation team
• Explain why the simulation is being conducted
• Respond to questions
• Mitigate their concerns
• Control the flow
• Prepare an agenda
• Have respect for the customers time

19
Avoid a g error

• Fuzzy at the outset, unclear at the finish
• Make sure that the objectives are clear at the
  beginning
• Objectives should be precise, reasonable,
  understandable, and measurable

20
Manage expectations

• Expectations can be tempered now, but, beliefs
  are hard to change later
• Allowing expectations to go unchecked can result
  in an off course project later
• Make sure that the customers understand what is
  in ad what is out of the study
• Dont agree too readilyput your foot down!

21
Ask the right questions

• To get the right answers, ask the right questions
• Prior preparation is useful
• Select your questions before you arrive
• Force new thinking
• What if the incoming freight arrives late? Early?
  
• What if storage space is increased? Decreased?

22
Ask the right questions

• Avoid yes/no questions
• Poor Is the system functioning OK?
• Better What aspects of the system need
  attention and why?

23
Ask the right questions

• Avoid putting the customer on the defensive?
• Poor Why did you store that freight in the
  AS/RS when it could have been placed in the
  racks?
• Better How do you decide where to store
  incoming freight?

24
Avoid judgment

• Dont solve the problem before learning what it
  is
• Good simulationists are good modelers
• But, they are great listeners
• They give the customer a chance to explain the
  system
• They solve the customers problem, not theirs
• Quickly solving the problem and leaping to a
  conclusion can backfire

25
Communicate

• Cooptate!
• Put everything in writing
• Get the customers agreement
• Gain the customers support
• Make the customer a part of the team
• Have frequent deliverable
• If there is a problem, tell the customer about it

26
Get ahead of yourself

• Ask the customer to do a quick and dirty
• Gets the thought process started
• Arouses the customers interest
• Helps in understanding the customers thinking
  process

27
2. Model conceptualization

• Guidelines
• Provide direction
• Take a chance

28
Provide direction
Set objectives
Identify and prioritize key questions
Define model outputs needed
Bound scope and level of detail
Specify model inputs
29
Set objectives

• Get an overview of the system
• Discuss base system and future system
• Review current operating procedures

30
Identify and prioritize key questions

• List possible questions
• Rank them
• Select key questions
• Determine benefits from each key question

31
Define model outputs needed

• Determine information needed
• Review standard reports
• Specify additional reports

32
Bound scope and level of detail

• Look for logical boundaries
• Look at scarce resources
• Identify simplifying assumptions

33
Specify model inputs

• Determine data needed to obtain desired outputs

34
Take a chance

• Nothing ventured, nothing gained
• Push the envelope
• Use part of the simulation language with which
  you arent familiar
• Use a new analysis technique
• Failure is a learning experience

35
3. Data collection

• Guidelines
• Challenge the data
• Make an educated guess

36
Challenge the data

• Dont take the data for granted
• Does the data make sense?
• Is it at the appropriate level of detail?
• Is it grouped improperly?
• Example Truck arrivals are confounded instead
  of by specific firm or type of truck
• A model cannot handle questions beyond the limits
  of the data

37
Make an educated guess

• You may need to lay it on the line
• Rather than stop the modeling process, you may
  have to take a chance, but not a dumb one
• Previous exampletruck arrivals are confounded
• May have to apportion the data
• .667 l
• .333 l

38
4. Model Translation

• Guidelines
• Focus on the problem
• Start simply
• Not too complex
• Keep it moving!
• What have we done?

39
Focus on the problem

• Forget the model!
• The objective is solving the problem, not
  building the model
• Spend more time experimenting and less time
  refining
• Use the model to generate new ideas

40
Start simply

• Grow the model
• If you start with too much detail, youll never
  be able to verify and validate the model
• Start with the basics
• Void of complex flows, decision logic, and
  disruptions
• Verify and validate
• Then, expand the model
• Cyclic process

41
Not too complex

• Keep it simple, but not too simple complex
  enough only to answer the questions asked
• Watch for model creep
• A model can become more sophisticated than the
  system being modeled!
• Example Route selection in the model that uses
  algorithms not in the real system
• Dont build a model more complex than the
  customers ability to implement

42
Keep it moving!

• The way to eat an elephant is one fork full at a
  time
• Have lots of small milestones, not just one
  deadline
• Model specs, prototype demos, sample animations,
  training tools,
• Time these over the life of the project
• Keep the customer involved

43
What have we done?

• Stop, in the name of love!
• Review what has been done to date
• Is a mid-course correction needed?
• How is our time and budget going versus
  completion?
• How much more do we have to do?

44
5. Verification and validation

• Guidelines
• Plan for VV
• Be in control

45
Plan for V/V

• Its not an afterthought
• Budget for V/V at the beginning
• Dont say, well do it if theres time
• 20 to 30 of the effort
• Do lots of V/V
• Dont stop too soon

46
Be in control

• Changes are inevitable
• Play it smart
• Dont accept a change just because it is small
• Or, because the project is going well
• Or, because the customer wants it
• Accept changes only if necessary
• Changes cause other changes
• Consider holding changes for later
• First, we finish the baseline model

47
Be in control

• Document change requests
• And the decision made to accept or not to accept
  (and why or why not)

48
6. Analyze the output

• Guidelines
• Work the model
• Question the output
• Understand the models limits
• Know when to stop
• Present some alternatives

49
Work the model

• Let the model do the work
• Exercise the model
• Draw insights
• Test the sensitivity of the system
• Understand the underlying behavior of the system
• Working the model can lead to discoveries about
  cause effect relationships

50
Question the output

• If it doesnt make sense, its probably wrong
• Are the outputs rational?
• Can they be explained?
• Are the assumptions accurate?
• How do the results compare to the customers
  quick and dirty?
• Reverify and revalidate as needed

51
Understand the models limits

• The customer decides, not the model
• The model is only an abstraction
• Interpretation of the output is needed
• Dont stray beyond what is reasonable
• Example Dont extrapolate beyond the range of
  the input dataif the model is regional, dont
  assume that it applies nationallyit may, or it
  may not
• Models dont replace human thought
• They support decision making they dont replace
  it

52
Know when to stop

• The perfect model is unaffordable
• More can always be done
• The list is endless
• More boundary testing
• More training
• Better documentation
• More rigorous statistical evaluation
• Know when to say when

53
Present some alternatives

• Dont put the customer in a box
• The customer asks for a solution, but really
  wants a choice
• If your one solution is rejected, you are stuck
• Present a range of possibilities

54
8. Implementation

• Guidelines
• Inspire trust
• Structure presentations

55
Inspire trust

• Say what you do, do what you say
• The customer must have trust in the project team
  to insure success
• The project team has to continue to prove itself
  to achieve this trust
• Achieved through high principles
• Achieved through keeping your word
• If trust is lost, its hard to get it back

56
Structure presentations

• The medium is the message
• Make a persuasive presentation
• Present the message clearly
• Organize the presentation in a clear manner
• Anticipate questions
• Edit for punch
• Focus on qualitative insights rather than
  quantitative results
• Get to the point!

57
References
Banks, J., Plan for Success, 1998, IIE Solutions,
Institute of Industrial Engineers, Norcross,
Georgia, USA.   Law, A. and M.G. McComas, 2001,
How to build valid and credible simulation
models, Proceedings of the 2001 Winter Simulation
Conference, B.A. Peters, J.S. Smith, D.J.
Medeiros, and M.W. Rohrer, eds., Piscataway, New
Jersey, USA Institute of Electrical and
Electronics Engineers, pp. 22-29. Musselman,
K.J., 1998, Chapter 22 in Handbook of Simulation,
J. Banks, ed., New York, New York, USA John
Wiley Sons.   Sadowski, D.A. and M.R. Grabau.
2000, Tips for successful practice of simulation,
Proceedings of the 2000 Winter Simulation
Conference, J.A. Joines, R.R. Barton, K. Kang,
and P.A. Fishwick, eds., Piscataway, New Jersey,
USA Institute of Electrical and Electronics
Engineers, pp. 26-31.
58
Video clips
59
Questions