Output Analysis for a Single Model

1
Lecture 9

• Output Analysis for a Single Model

2
Output Analysis for a Single Model

• Output analysis is the examination of data
  generated by a simulation.
• Its purpose is to predict the performance of a
  system or to compare the performance of two or
  more alternative system designs.
• This lecture deals with the analysis of a single
  system, while next lecture deals with the
  comparison of two or more systems.

3
Type of Simulation with respect to Output Analysis

• When analyzing simulation output data, a
  distinction is made between terminating or
  transient simulation and steady-state simulation.
• A terminating simulation is one that runs for
  some duration of time TE, where E is a specified
  event (or set of events) which stops the
  simulation.
• Example 11.1 Shady Grove Bank operates 830
  1630, then
• TE 480min.
• Example 11.3 A communication system consists of
  several components. Consider the system over a
  period of time, TE , until the system fails. E
  A fails, or D fails, or (B and C both fail)

4
Terminating Simulation

• When simulating a terminating system, the initial
  conditions of the system at time 0 must be
  specified, and the stopping time TE, or
  alternatively, the stopping event E, must be well
  defined.
• Whether a simulation is considered to be
  terminating or not depends on both the objectives
  of the simulation study and the nature of the
  system.
• Examples 11.1 and 11.3 are considered the
  terminating systems because
• Ex. 11.1 the objective of interest is one days
  operation
• Ex. 11.3 short-run behavior, from time 0 until
  the first system failure.

5
Steady-state Simulation

• A nonterminating system is a system that runs
  continuously, or at least over a very long period
  of time.
• For example, assembly lines which shut down
  infrequently, continuous production systems of
  many different types, telephone systems and other
  communications systems such as the Internet,
  hospital emergency rooms, fire departments, etc.
• A steady-state simulation is a simulation whose
  objective is to study long-run, or steady-state,
  behavior of a nonterminating system.
• The stopping time, TE, is determined not by the
  nature of the problem but rather by the
  simulation analyst, either arbitrarily or with a
  certain statistical precision in mind.

6
Stochastic Nature of Output Data

• Consider one run of a simulation model over a
  period of time 0, T . Since the model is an
  input-output transformation, and since some of
  the model input variables are random variable, it
  follows that the model output variables are
  random variables.
• The stochastic (or probability) nature of output
  variables will be observed.
• Example 2.2 (Able-Baker carhop problem)
• Input randomness of arrival time and service
  time
• Output randomness of utilization and time spent
  in the system per customer.

7
Output Analysis for Terminating Simulations

• Consider the estimation of a performance
  parameter, ? (or ?), of a simulated system.
• The simulation output data is of the form Y1,
  Y2, , Yn (discrete-time data) for estimating
  ?.
• E.g. the delay of customer i, total cost in week
  i.
• The simulation output data is of the form Y(t),
  0 ? t ? TE (continuous-time data) for estimating
  ?.
• E.g. the queue length at time t, the number of
  backlogged orders at time t.
• Point Estimation

8
Output Analysis for Terminating Simulations
(cont)

• By the Central Limited Theorem (CLT), for n ? 30,
• where
• Interval Estimation
• An approximate 100(1 - ?) confidence interval
  for ? is given by

9
Output Analysis for Terminating Simulations
(Example)

• Example 11.10 (Able Baker Carhop Problem)

10
Number of Replications

• PRECISION LEVEL
• Suppose that an error criterion ? is specified
  in other words, it is desired to estimate ? by
  to within
• with high probability, say at least 1 ?.

11
Number of Replications(Example)

• Example 11.12 (Able Baker Carhop Problem)
• Suppose that it is desired to estimate Ables
  utilization in Example 11.7 to within with
  probability 0.95. An initial sample size R04 is
  taken.
• Step 1
• Step 2

R 15 Additional replications R R0 15 4
11
12
Output Analysis for Steady-State Simulations

• Prior to beginning analysis of output data, the
  modeler must take every effort to ensure that the
  output represents an accurate estimate of the
  true system values.
• One useful technique for improving the
  reliability of output results from steady-state
  simulation is to provide an initialization period
  for which statistics are not kept.
• A steady-state condition implies that a
  simulation has reached a point in time where the
  state of the model is independent of the initial
  start-up conditions.

13
Output Analysis for Steady-State Simulations
(cont)

• The amount of time required to achieve
  steady-state conditions is referred to as a
  warm-up period.
• Data collection begins after a warm-up period is
  completed.
• Determining the length of this period can be
  accomplished by utilizing moving averages
  calculated from the output produced by multiple
  model replications.

14
Warm-up Period

• Determine A Warm-up Period in a Steady-state
  Simulation

15
Moving Average
d 12
16
Output Analysis for Steady-State Simulations
(Example)

• Observed cost during i-th period and j-th
  replication

17
Output Analysis for Steady-State Simulations
(Example)

• Confident Interval for a Steady-State Simulation