Variance

1

• Variance
• Reduction
• Techniques

2
Outline

• Importance of Variance Reduction
• Types of Variance Reduction Techniques
• Common Random Numbers
• Example Common Random Numbers
• Implementing Common Random Numbers in Arena
• Antithetic Variates
• Implementing Antithetic Variates in Arena
• Control Variates

3
Reasons for Importance

• Associated with an estimate of a performance
  measure (e.g., mean time in the emergency room
  for a patient).
• The variance is a measure of the amount of
  uncertainty in an estimate -- the smaller the
  variance, the less the uncertainty.
• The smaller the variance, the easier it is to
  distinguish between/among (i.e., rank, or select
  the best) alternative systems/policies.
• To reduce the variance of an estimate, one
  could
• 1. Make more replications.
• 2. Use one of several variance reduction
  techniques.
  
• (The latter is the preferred approach,
  especially if computational time is a constraint).

4
Types of Variance Reduction Techniques

• 1. Common Random Numbers (CRN)
• 2. Antithetic Variates (AV)
• 3. Control Variates (CV)
• 4. Others Indirect Estimation, Conditioning

5
Common Random Numbers

• Applies only when comparing two or more
  alternative systems
• Most Popular VRT
• Basic idea compare alternative system
  configurations using similar experimental
  conditions (i.e., the same set of random
  numbers)
• A form of blocking
• Also called correlated sampling or matched
  streams
• Synchronization of random numbers across
  different alternatives is important
• A pilot study may be appropriate, since
  backfiring is a possibility
• Formal statistical analysis can be made
  complicated with CRN

6
Justification for the Use of Common Random Numbers

• Let Xij be the observation for the jth
  replication of the ith alternative (i1, 2
  j1, 2, ..., n).
• If the two sets of replications are done
  independently, then Cov(X1j, X2j)0, but if
  positive correlation is induced with CRN, then
  Cov(X1j, X2j) gt0.

7
Common Random Numbers

• Synchronization is an important aspect in the
  use of CRN. For example, you would want to
  dedicate particular streams of random numbers for
  particular purposes (e.g., for interarrival
  times) for each alternative.

8
ExampleCommon Random Numbers

• Suppose we have a queuing situation in which two
  alternative system configurations are being
  considered
• Alternative 1 1 server, with a processing time
  that is exponentially distributed, with a mean of
  .9 minute.
• Alternative 2 2 servers, each with a processing
  time that is exponentially distributed, with a
  mean of 1.8 minute.
• Assume that the interarrival time for customers
  is exponentially distributed, with a mean of 27
  seconds, and the queuing discipline is FIFO.
• Suppose that the performance measure of interest
  is the average time in the queue of the first 100
  customers.
• (This model will be called the ATM model).

9
Example Common Random Numbers (continued)
Running each of these models for 10 replications
with no variance reduction technique gave the
following results for average time in the queue
for the first 100 customers. j X1j X2j Zj
X1j - X2j _______________________________________
______________________ 1 5.04 1.02
4.02 2 6.10 3.03 3.07 3 6.26 1.85
4 1.77 2.81 5 2.43 3.40 6 3.20 1.25 7
11.96 2.63 8 4.00 2.87 9
12.58 3.71 10
11.97 2.18 Sample Mean
6.53 2.48 4.06 Sample Variance 17.2 .78
16.56 95 CI Half-Width 2.97 .63 2.91
10
Example Common Random Numbers (continued)
Now, implementing Common Random Numbers, through
the use of the SEEDS module (from the Elements
panel), we obtain the following output for
average time in the queue for the first 100
customers. (See the following pages for how
this implementation was accomplished).
j X1j X2j Zj X1j - X2j ____________________
_________________________________________ 1 5.71
5.17 .54 2 2.70 2.29 .41 3 .53
.41 .12 4 5.78 5.64 .14 5 8.77 7.82
6 2.45 2.01 7 4.18 3.46 8 .83
.65 9 5.17 5.00 10 1.65 1.38
Sample Mean 3.78 3.38 .395 Sample Variance
6.81 6.00 .076 95 CI Half-Width 1.87 1.75 .197
11
Example Common Random Numbers

• Note how the sample variance for the difference
  was decreased from 16.56 to 0.076 ( a decrease of
  99.5) and how the half-width for the 95
  confidence interval for the difference between
  the two alternatives was decreased from 2.91 to
  0.197 ( a decrease of 93).
• In summary, we have
• 95 CI without CRN (1.15, 6.97)
• 95 CI with CRN (0.319, 0.592)

12
Implementing Common Random Numbers in Arena

• By default, Arena uses stream 10 to generate all
  of its random numbers for a series of simulation
  replications. This is not what you want for CRN.
• Use the SEEDS module from the Elements panel to
  implement CRN. This allows us to name and
  dedicate different streams for different purposes
  in a model
• Using the Common selection for the Initialize
  Option spaces the seeds for each replication
  within each stream 100,000 random numbers apart,
  so that there will be no overlap of random number
  usage within a stream across replications.
• Generally, do not use stream 10 when using this
  approach.
• The SEEDS module allows you to dedicate the
  streams 1,2, in series.

13
Implementing Common Random Numbers in Arena

• Each location in the model which involves
  generation of random variates needs to be
  modified. For example, instead of
• EXPO(5)
• for interarrival time in a Arrive module, one
  would use
• EXPO(5, Stream Interarrivals)
• where Stream Interarrivals is defined as a
  particular stream in the SEEDS module.
• To implement CRN, the SEEDS module should be
  employed in all alternative model runs.

14
Example Implementing CRN in the Arena model

• For each of the two alternative models,
• 1) attach the ELEMENTS panel, and place the SEEDS
  module in the model window.
• 2) Name and dedicate streams for Interarrival
  Times and Processing Times, by adding the
  elements
• i) for stream 1
• Identifier Interarrival Times Stream
• Seed Value (default)
• Initialize Option Common
• ii) for stream 2
• Identifier Processing Times Stream
• Seed Value (default)
• Initialize Option Common
• 3) In the Arrive module, replace the EXPO(1) in
  the Time Between field with EXPO(1, Interarrival
  Time Stream)

15
Example Implementing CRN in the Arena model

• 4) In the Server module, replace the Process Time
  in the single server model with EXPO(0.9,
  Processing Time Stream) and in the two server
  model with EXPO(1.8, Processing Time Stream).
• 5) Save the observations for the average value of
  the time in queue for each model to one file for
  the single server model and to another file for
  the two-server model, by modifying the Outputs
  section of the Statistics module.

16
Example Implementing CRN in the Arena model

• After running each of the models, we can use the
  output analyzer to compare the alternatives
• 1) Select Tools/Output Analyzer.
• 2) Select File/New.
• 3) Add Data from Files
• 4) Select Analyze/Compare Means.
• 5) Select
• Data File A Replications Lumped
• Data File B
• Replications Lumped
• We obtain the following results for a 95
  paired-t confidence interval
• Std Dev 0.275
• CI 0.198, 0.592
• Reject Ho means are not equal
  at the 0.05 level.

17
Antithetic Variates (AV)

• Useful in reducing the variance of an estimate
  for a single alternative.
• Basic idea make pairs of runs for a single
  alternative so that a small observation for one
  run is offset by a large observation for the
  second run of the pair.
• Operationally,
• rns for run 1 of a pair r1, r2, r3,
• rns for run 2 of a pair 1-r1, 1-r2, 1-r3,
• Note that both streams are i.i.d., u(0,1) rns,
  hence everything is valid with respect to each
  run.
• Synchronization must be used.

18
Implementing Antithetic Variates in Arena

• This is much like implementing CRN in Arena, the
  only difference is that you need to specify
  Antithetic in the Initialize Option field if the
  SEEDS module. This will give you Antithetic pairs
  in your replications.
• For example, implementing the antithetic option
  in the single-server ATM model used for the CRN
  example would give us the following results for
  average queuing time (the AV model is contained
  in the file av1).

19
Implementing Antithetic Variates in Arena

• Replication No Variance Reduction Antithetic
  Variates
• 1 5.04 6.83
• 2 6.10 3.15
• 3 6.26 7.16
• 4 1.77 2.51
• 5 2.43 3.41
• 6 3.20 2.68
• 7 11.96 0.58
• 8 4.00 7.25
• 9 12.58 2.43
• 10 11.97 1.99
• Sample Mean 6.53 3.8
• Sample Variance 17.2 5.7
• 95 CI Half Width 2.97 1.71
• Note how the sample variance was reduced by 67
  (from 17.2 to 5.7) and how the confidence
  interval half-length was reduced by 42 (from
  2.97 to 1.71) by using antithetic variates.

20
Control Variates

• The basic idea is to take advantage of
  correlation between certain random variates, in
  order to reduce the variance of the output. For
  example, suppose we have a queuing system with
• -- Interarrival Time Mean (specified) 3 min.
• -- Interarrival Time Mean (actual, from model
  run) 3.21 min.
• -- Mean Time in system(estimated, from model)
  3.4 min.
• Since the mean interarrival time from the model
  (3.21) is greater than it should be (3.), one
  might expect that the sample mean time in the
  system (3.4 minutes) is less than it should be.
  We should adjust this time (3.4 minutes) upwards
  the question is, by how much?

21
Choosing Control Variates

• Important factors to consider include
• The correlation between the control variate and x
  should be high,
• The control variate itself should have a low
  variance.