Steps of a sound simulation study

1
Steps of a sound simulation study

• 1. Formulate the problem and plan the study.
• Problem of interest is stated precisely
• Arrange a meeting of the team of simulation
  analysts and determine
• the objectives of the simulation study
• Specific questions to answer
• Performance measure used to evaluate the
  efficiency of different system configurations
• Scope of the model
• System configurations to be modeled
• Software to be used
• Time frame for the study

2
2. Collect data and define a model

• Collect information on the system layout and
  operating procedures
• Collect data (if possible) to specify the model
  parameter and input probability distributions
• Collect data on the performance of the existing
  system (if possible)
• Build the model according to
• Project objectives
• Performance measure
• Data availability
• Credibility concerns
• Computer constraints
• Opinions of subject matter experts
• Time and memory constraints

3
3. Validate the model

• perform a structural walk-through of the
  conceptual model, this
• Helps ensure that the model assumptions are
  correct and complete
• Promotes ownership of the model
• Takes place before programming begins
• 4. Construct a computer program and Verify
• Decide which computer program to use
• Construct the computer program and make sure it
  is verified.

4
5. Make pilot runs

• Use for validating the model
• Gives you how much more runs you need
• 6. Is the program model valid?
• Use the pilot runs to check whether the results
  obtained are consistent with the real world or
  not.
• 7. Design experiments. Decide how much simulation
  runs to be made for each alternative system

5

• Perform the productive simulation runs that are
  used to estimate the performance measure
• Analyze the output data.
• Use statistical techniques to analyze the output
  data obtained by the production runs (will
  discuss later)
• Document, present, and use results

6
Monte Carlo Simulation

• Solving deterministic problems using stochastic
  models.
• Example estimate
• It is efficient in solving multi dimensional
  integrals.

7
Monte Carlo Simulation

• To illustrate, consider a known region R with
  area A and R1 subset of R whose area A1 in
  unknown.
• To estimate the area of R1 we can through random
  points in the region R. The ratio of points in
  the region R1 over the points in R approximately
  equals the ratio of A1/A.

R
R1
8
Monte Carlo Simulation

• To estimate the integral I. one can estimate the
  area under the curve of g.
• Suppose that M max g(x) on a,b

1. Select random numbers X1, X2, ,Xn in
a,b And Y1, Y2, ,Yn in 0,M 2. Count how
many points (Xi,Yi) in R1, say C1 3. The estimate
of I is then C1M(b-a)/n
M
R
R1
a
b
9
Advantages of Simulation

• Most complex, real-world systems with stochastic
  elements that cannot be described by mathematical
  models. Simulation is often the only
  investigation possible
• Simulation allow us to estimate the performance
  of an existing system under proposed operating
  conditions.
• Alternative proposed system designs can be
  compared with the existing system
• We can maintain much better control over the
  experiments than with the system itself
• Study the system with a long time frame

10
Disadvantages of Simulation

• Simulation produces only estimates of performance
  under a particular set of parameters
• Expensive and time consuming to develop
• The Large volume of numbers and the impact of the
  realistic animation often create high level of
  confidence than is justified.

11
Pitfalls of Simulation

• Failure to have a well defined set of objectives
  at the beginning of the study
• Inappropriate level of model details
• Failure to communicate with manager during the
  course of simulation
• Treating a simulation study as if it is a
  complicated exercise in computer programming
• Failure to have well trained people familiar with
  operations research and statistical analysis
• Using commercial software that may contain errors

12
Pitfalls of Simulation cont.

• Reliance on simulator that make simulation
  accessible to anyone
• Misuse of animation
• Failure to account correctly for sources of
  randomness in the actual system
• Using arbitrary probability distributions as
  input of the simulation
• Do output analysis un correctly
• Making a single replication and treating the
  output as true answers
• Comparing alternative designs based on one
  replication of each design
• Using wrong measure of performance