ESI 6529 Digital Simulation Techniques

1
ESI 6529 Digital Simulation Techniques

• Lecture 9

2
OUTPUT ANALYSIS

• Common tasks of output analysis include
• Determining appropriate number of simulation runs
• Selection of appropriate simulation run length
• Interpretation of simulation results
• Analysis of the difference between runs

3
The observations from a particular replication
(row) are not IID in general. However, the
observations in the ith column, y1i, y2i, ...,
yni, are IID observations of random variable Yi
(i1,2, ..., m).
4
Terminating and Non-terminating Systems

• A terminating simulation model is one for which
  there is a natural event E that specifies the
  length of each run. The event E often occurs at a
  time instant beyond which no useful information
  is obtained or at a time point where the system
  is cleaned-out.
• EXAMPLE A bank branch is open from 9 AM to 5
  PM. All customers enter the bank before 5 PM will
  be served. The simulation objective is to
  estimate the average waiting time of customers.
  In this case, Eat least 8 hours of simulated
  time has been elapsed and the system is empty.

5
Terminating and Non-terminating Systems

• A non-terminating simulation model is one for
  which there is no natural event E to specify the
  length of a run. If the output process Y1, Y2,
  ... has a steady-state distribution, we might be
  interested in estimating the steady-state
  statistics of some performance measures.

6
Statistical Analysis for Terminating Simulations
For random variable X, we have simulated IID data
points X1, X2, ..., Xn The sample mean is an
unbiased point estimator for the mean of
X, Approximately, the 100(1-?) percent (0lt ?
lt1) confidence interval for the mean is given by
where S2(n) is the sample variance which is
given by
7
Example
We perform 10 simulation runs of a bank model and
list the output results in the following table.

8
Example
We want to obtain a point estimate and
approximate 90 confidence interval for the
expected average delay in queue for a customer.
Using the simulation data, we obtained

9
Precision of the Point Estimate
Let h be the half-width of the confidence
interval of the point estimate,
In order to ensure the accuracy of the
estimation, we would like to have
where ? is a given parameter, 0lt? lt1. In
practice, usually ?0.1.
10
Number of Simulation Runs

• Run the simulation model n1 times initially
  (usually n110).
• Calculate the sample average x-bar and the
  confidence interval half width h1.
• If h1 lt 0.1x-bar, stop.
• If h1 gt 0.1x-bar, let h2 0.1x-bar.

The total number of simulation runs needed is
then
11
For the bank problem above, suppose that we
would like to have the confidence interval half
width less than or equal to 10 of the sample
mean of customer waiting time. Determine the
total number of simulation runs needed. For this
ease, we have n1 10, h1 0.32, and h2 0.203.
The total number of runs is
Therefore, additional 15 simulation runs are
needed to get the desired accuracy of the
confidence interval.
12
Statistical Analysis for Non-terminating
Simulations

• Replication / Deletion Approach for Means
• Make n1 pilot simulation runs
• Estimate the warm-up time for the system
• Cut off the warm-up period from each replication
• Compute sample mean and confidence interval
• If half width of the confidence interval h1 is
  less than 10 of the sample mean, stop.
• Otherwise, determine the total number of
  replications needed
• Run more replications and compute the sample mean
  again

13
The Method of Batch Means
Run one replication of the simulation and cut off
the warm-up period. Partition the remaining
steady-state data points into smaller batches and
treat each batch as a separate simulation run.
We can determine the warm-up period using the
moving average graph, and determine the number of
batches using the same formula for number of
simulation replications for terminating
systems. We will use the following example to
illustrate the technique.
14
THE SIMAN OUTPUT PROCESSOR

• It assists in analyzing output data by
• Displaying data graphically
• Analyzing data statistically
• Interacting with data files
• Graphics Commands
• Statistical Commands

15
THE SIMAN OUTPUT PROCESSOR
Data Transfer Commands
Entering the Output Processor gt
C\ARENA\arena gt Click on the TOOLS/OUTPUT
ANALYZER menu
16
EXAMPLE Machine Shop Revisit
For illustration purpose, we simulate again the
single machine with inspection model in Example
4. We output the average flowtime from each
replication to a data file, flowtime.dat, using
OUTPUTS TAVG(Flowtime), Flowtime.OUT
Determine the flowtime. Simulate the system for
a sufficient number of days so that the average
flowtime falls within ?10 percent of the sample
mean with 95 percent confidence. Then determine
the likelihood that the flowtime will be at least
15 percent less than the calculated average, and
at least 20 percent greater than the calculated
average.
17
Table and Barchart of Average Flowtimes

• C\ARENA\Arena
• Click on the TOOLS/OUTPUT ANALYZER menu and
  choose the NEW option
• Once the NEW DATA GROUP window is open, open the
  FILES menu and pick the ADD option
• Add the FLOWTIME.OUT file by double clicking on
  the name of the file. Then, click on OPEN
• Go to the GRAPH menu and pick TABLES
• Click on LUMPRED
• Fill in the necessary information and OK

18
Table and Barchart of Average Flowtimes
TABLE FLOWTIMES
RUN NO TAVG(FLOWTIME)
1.00 71.0 2.00
82.3 3.00 116.
4.00 108. 5.00
133. 6.00 101.
7.00 116. 8.00
68.1 9.00 130.
10.0 59.2
19

• Go to the GRAPH menu and pick the BARCHART option
  
• Fill in the information you might want to have on
  your graph and OK

20
Confidence Interval for Average Flowtimes

• Go to the ANALYZE menu and click on the INTERVALS
  option
• Fill in the necessary information, action and OK

INTERVALS FLOWTIMES IDENTIFIER
AVERAGE STANDARD 0.950 C.I.
MINIMUM MAXIMUM NUMBER
DEVIATION HALF-WIDTH
VALUE VALUE OF OBS.
--------------------------------------------------
--------------------------------------------------
-------------------------- AVG FLOWTIME 98.5
26.6 19.1
59.2 133.
10 OBSERVATION INTERVALS FLOWTIMES




59.2
98.5
133 AVG FLOWTIME
X

79.5
118


MINIMUM
LOWER 95 CL X AVERAGE UPPER
95 CL MAXIMUM
21
Determining the Number of Additional Runs
In this case, we have n1 10, h1 19.1, and
?0.1. Then the total number of simulation runs
is estimated as The number of runs is
rounded up to 40. Therefore, we need to make
another 30 replications in order to obtain the
desired statistics.
22
Append the Production Data to the Pilot Data

• Run the program again after making the following
  changes to the experiment file
• OUTPUTS TAVG (Flowtime), More.OUT
• SEEDS 1,123 !Stream 1
• 2,456 !Stream 2
• 3,987 !Stream 3
• 4,654 !stream 4
• 10,321 Stream l0 (Branch Block)
• REPLICATE,30
• Do steps (1) through (2)
• Go to FILE/DATA FILES menu and click on the
  APPEND option
• Select the FLOWTIME.OUT and MORE.OUT file and
  click on the OK button

23
Table of Average Flowtimes

• Repeat Steps (3) and (4)
• Make sure that your FLOWTIME.OUT file show LUMP
  under REP CHOICES, otherwise, click on the LUMP
  button
• Do steps (5) and (6)

TABLE FLOWTIMES
RUN NO TAVG(FLOWTIME) 1.00
71.0 2.00 82.3
3.00 116.
........................... 38.0
141. 39.0 121.
40.0 89.3
24
Barchart of Average Flowtimes

• Do (7) and (8)

25
Confidence Interval for Average Flowtimes

• Do (10) and (11)

OBSERVATION INTERVALS FLOWTIMES




59.2 98.0
190
AVG FLOWTIME
X
89.7
106




MINIMUM LOWER 95 CL X
AVERAGE UPPER 95 CL MAXIMUM
26
Histogram of Average Flowtimes

• Go to the GRAPH menu and choose the HISTOGRAM
  option
• Fill the necessary information, and OK (default
  values are used generated by SIMAN if you dont
  put anything)

27
Histogram of Average Flowtimes
HISTOGRAM AVG FLOWTIME
CELL LIMITS ABSOLUTE FREQ.
RELATIVE FREQ. CELL NO. ----------------------
--------------------------------------------------
------------ FROM TO
CELL CUMUL. CELL
CUMUL.
0 -INFINITY 65.000
2.0 2.0 0.0500 0.0500 1
65.000 70.000 2.0
4.0 0.0500 0.1000 2 70.000
75.000 5.0 9.0
0.1250 0.2250 3 75.000
80.000 4.0 13.0 0.1000
0.3250 4 80.000 85.000
2.0 15.0 0.0500 0.3750
5 85.000 90.000 2.0
17.0 0.0500 0.4250 6
90.000 95.000 4.0 21.0
0.1000 0.5250 7 95.000
100.00 3.0 24.0 0.0750
0.6000 8 100.00 105.00
3.0 27.0 0.0750 0.6750
9 105.00 110.00
2.0 29.0 0.0500 0.7250 10
110.00 115.00 1.0 30.0
0.0250 0.7500 11 115.00
120.00 3.0 33.0 0.0750
0.8250 12 120.00 125.00
2.0 35.0 0.0500 0.8750
13 125.00 130.00 0.0
35.0 0.0000 0.8750 14
130.00 135.00 2.0 37.0
0.0500 0.9250 15 135.00
140.00 0.0 37.0 0.0000
0.9250 16 140.00 INFINITY
3.0 40.0 0.0750 1.0000
28
Estimates of System Flowtimes
Using the smoothed curve shown below, 1.
Expected value 98.0 minutes 2. Likelihood
that the flowtime X will be at least 15 below
the expected value Prob(X lt 83.3 minutes)
39 3. Likelihood that the flowtime will be
atleast 20 above the expected value
Prob(X gt 117.6 minutes) (100-82) 18
29
Non-Terminating Systems
Business has been very good at the two-machine
jobshop. They are now running three shifts, five
days a week. All work in progress ceases at the
end of the third shift on Saturday morning and
resumes where it left off on the following
Monday. Thus, the system is never empty.
Estimate the steady-state behavior of the
system. Simulate the system long enough for the
true average flowtime to fall within ?10 percent
of the sample mean with 95 percent confidence.
Now we have a non-terminating system. We first
make a pilot run of 21 days (30,000 minutes) and
output a data file Flow2.OUT.
30
Selection of Warm-Up Period

• Go to OUTPUT ANALYZER menu and choose NEW
• Add the Flow2.OUT file into the DATA GROUP
  window by using the ADD option
• Click on the MOVAVERAGE option from the GRAPH
  menu
• Enter MOVING in the TYPE field and 200 in the
  VALUE field and OK

Select 15,000 minutes for the warm-up period.
31
Remove the Warm-Up Period and Determine the
Batch Size

• Open the ANALYZE menu and click on the
  BATCH/TRUNCATE
• Fill the different fields as follows
• TITLE FLOWTIMES
• TYPE OF TRUNCATION TIM
• INIT. OBS/TIME TRUNCATED 15000
• TYPE OF BATCHING OBS
• OBS OR TIME PER GROUP 1
• SAVE VALUES FILE FLOWTIME . FILT
• Choose OK

32
FILTER
SUMMARY FLOWTIMES
--------------------------------------------------
-------------------------------
INITIAL TIME TRUNCATED 1.5000E04
NO. OF OBSERVATIONS PER BATCH
1 NUMBER OF BATCHES
754 NO. OF
TRAILING OBSERVATIONS TRUNC. 0
EST. OF COV. BETWEEN BATCHES 0.5420
----------------------------------
-----------------------------------------------
COV.EQ.0 REJECTED IN FAVOR OF
COV.GT.0 AT .05 LEVEL COV.EQ.0
REJECTED IN FAVOR OF COV.NE.0 AT .05 LEVEL
33

• Click on the FLOWTIME.FILT file. Go to the
  ANALYZE menu and pick the CORRELOGRAM command
• Fill in the necessary information and ACCEPT

Lag length corresponding to zero correlation
42 Batch size 10 x 42 420 observations
34
Estimate of Production Run Length
Warm-up period 15,000 min. Batch
Length Average interarrival time 20 min. 420
observations/batch x 20 min./ observation 8400
min./batch Total run length Assume 20 batches,
15,000 20 x 8400 183,000 min. Choose
200,000 min.
35
Batch the Truncated Production Data

• Re-run your program, but this time send your data
  to a file called Productn.OUT and use the
  warm-up and running times, we found.
• Add the PRODUCTN.OUT file to a NEW DATA GROUP
  screen
• Open the ANALYZE menu and click on the
  BATCH/TRUNCATE
• Fill the different fields as follows
• TITLE FLOWTIMES
• TYPE OF TRUNCATION
• INIT. OBS/TIME TRUNCATED
• INIT. OBS/TIME TRUNCATED OBS
• OBS OR TIME PER GROUP 420
• SAVE VALUES FILE PRODUCTN.FILT
• OK

36
Batch the Truncated Production Data
FILTER SUMMARY FLOWTIMES
-----------------------------------------
-----------------------------------------
NO. OF INITIAL OBSERVATIONS TRUNC.
0 NO. OF OBSERVATIONS PER
BATCH 420 NUMBER OF
BATCHES 22
NO. OF TRAILING OBSERVATIONS TRUNC. 4
EST. OF COV. BETWEEN BATCHES
3.9271E-02 --------------------
--------------------------------------------------
-------------
Too many batches. We would like to have 20
batches. Repeat with somewhat larger batch size.
37
Confidence interval for Batched Production Data

• Use the steps you learn to create the confidence
  interval for the data stored in the file
  PRODUCTN. FILT

INTERVALS FLOWTIMES IDENTIFIER
AVERAGE STANDARD 0.950 C.I.
MINIMUM MAXIMUM NUMBER
DEVIATION HALF-WIDTH
VALUE VALUE OF OBS.
--------------------------------------------------
--------------------------------------------------
-------------------------- AVG FLOWTIME 146
30.2 14.1
108 217 20
38
Results of the Production Run

• (200,000 minutes, less a 15,000 - minute warm-up
  period)
• Average flowtime 146 minutes
• Calculated 95 confidence interval 132, 161
  
• The confidence interval half width is within 10
  of the calculated mean. Hence, additional
  simulation is not required.