Part 8: Input Modeling

1
Part 8 Input Modeling
2
Agenda

1. Purpose Overview
2. Data Collection
3. Identifying Distribution
4. Parameter Estimation
5. Goodness-of-Fit Tests
6. Multivariate and Time-Series Input Models

3
1. Purpose Overview

• Input models provide the driving force for a
  simulation model.
• We will discuss the 4 steps of input model
  development
• Collect data from the real system
• Identify a probability distribution to represent
  the input process
• Choose parameters for the distribution
• Evaluate the chosen distribution and parameters
  for goodness of fit.

2
4
2. Data Collection

• One of the biggest tasks in solving a real
  problem. GIGO garbage-in-garbage-out
• Suggestions that may enhance and facilitate data
  collection
• Plan ahead begin by a practice or pre-observing
  session, watch for unusual circumstances
• Analyze the data as it is being collected check
  adequacy
• Combine homogeneous data sets, e.g. successive
  time periods, during the same time period on
  successive days
• Be aware of data censoring the quantity is not
  observed in its entirety, danger of leaving out
  long process times
• Check for relationship between variables, e.g.
  build scatter diagram
• Check for autocorrelation
• Collect input data, not performance data

3
5
3. Identifying the Distribution (1) Histograms
(1)

• A frequency distribution or histogram is useful
  in determining the shape of a distribution
• The number of class intervals depends on
• The number of observations
• The dispersion of the data
• Suggested the number intervals ? the square root
  of the sample size works well in practice
• If the interval is too wide, the histogram will
  be coarse or blocky and its shape and other
  details will not show well
• If the intervals are too narrow, the histograms
  will be ragged and will not smooth the data

4
6
3. Identifying the Distribution (1) Histograms
(2)

• For continuous data
• Corresponds to the probability density function
  of a theoretical distribution
• A line drawn through the center of each class
  interval frequency should results in a shape like
  that of pdf
• For discrete data
• Corresponds to the probability mass function
• If few data points are available combine
  adjacent cells to eliminate the ragged appearance
  of the histogram

Same data with different interval sizes
5
7
3. Identifying the Distribution (1) Histograms
(3)

• Vehicle Arrival Example of vehicles arriving
  at an intersection between 7 am and 705 am was
  monitored for 100 random workdays.
• There are ample data, so the histogram may have a
  cell for each possible value in the data range

6
8
3. Identifying the Distribution (2) Selecting
the Family of Distributions (1)

• A family of distributions is selected based on
• The context of the input variable
• Shape of the histogram
• The purpose of preparing a histogram is to infer
  a known pdf or pmf
• Frequently encountered distributions
• Easier to analyze exponential, normal and
  Poisson
• Harder to analyze beta, gamma and Weibull

7
9
3. Identifying the Distribution (2) Selecting
the Family of Distributions (2)

• Use the physical basis of the distribution as a
  guide, for example
• Binomial of successes in n trials
• Poisson of independent events that occur in a
  fixed amount of time or space
• Normal distn of a process that is the sum of a
  number of component processes
• Exponential time between independent events, or
  a process time that is memoryless
• Weibull time to failure for components
• Discrete or continuous uniform models complete
  uncertainty. All outcomes are equally likely.
• Triangular a process for which only the minimum,
  most likely, and maximum values are known.
  Improvement over uniform.
• Empirical resamples from the actual data
  collected

8
10
3. Identifying the Distribution (2) Selecting
the Family of Distributions (3)

• Do not ignore the physical characteristics of the
  process
• Is the process naturally discrete or continuous
  valued?
• Is it bounded or is there no natural bound?
• No true distribution for any stochastic input
  process
• Goal obtain a good approximation that yields
  useful results from the simulation experiment.

9
11
3. Identifying the Distribution (3)
Quantile-Quantile Plots (1)

• Q-Q plot is a useful tool for evaluating
  distribution fit
• If X is a random variable with cdf F, then the
  q-quantile of X is the g such that
• When F has an inverse, g F-1(q)
• Let xi, i 1,2, ., n be a sample of data from
  X and yj, j 1,2, , n be the observations in
  ascending order. The Q-Q plot is based on the
  fact that yj is an estimate of the (j-0.5)/n
  quantile of X.
• where j is the ranking or order number

By a quantile, we mean the fraction (or percent)
of points below the given value
percentile 100-quantiles deciles
10-quantiles quintiles 5-quantiles quartiles
4-quantiles
10
12
3. Identifying the Distribution (3)
Quantile-Quantile Plots (2)

• The plot of yj versus F-1( (j-0.5)/n) is
• Approximately a straight line if F is a member of
  an appropriate family of distributions
• The line has slope 1 if F is a member of an
  appropriate family of distributions with
  appropriate parameter values
• If the assumed distribution is inappropriate, the
  points will deviate from a straight line
• The decision about whether to reject some
  hypothesized model is subjective!!

11
13
3. Identifying the Distribution (3)
Quantile-Quantile Plots (3)

• Example Check whether the door installation
  times given below follows a normal distribution.
• The observations are now ordered from smallest to
  largest
• yj are plotted versus F-1( (j-0.5)/n) where F has
  a normal distribution with the sample mean (99.99
  sec) and sample variance (0.28322 sec2)

12
14
3. Identifying the Distribution (3)
Quantile-Quantile Plots (4)

• Example (continued) Check whether the door
  installation times follow a normal distribution.

Straight line, supporting the hypothesis of a
normal distribution
Superimposed density function of the normal
distribution
13
15
3. Identifying the Distribution (3)
Quantile-Quantile Plots (5)

• Consider the following while evaluating the
  linearity of a Q-Q plot
• The observed values never fall exactly on a
  straight line
• The ordered values are ranked and hence not
  independent, unlikely for the points to be
  scattered about the line
• Variance of the extremes is higher than the
  middle. Linearity of the points in the middle of
  the plot is more important.
• Q-Q plot can also be used to check homogeneity
• Check whether a single distribution can represent
  two sample sets
• Plotting the order values of the two data samples
  against each other. A straight line shows both
  sample sets are represented by the same
  distribution

14
16
4. Parameter Estimation (1)

• Next step after selecting a family of
  distributions
• If observations in a sample of size n are X1, X2,
  , Xn (discrete or continuous), the sample mean
  and variance are defined as
• If the data are discrete and have been grouped in
  a frequency distribution
• where fj is the observed frequency of value Xj

15
17
4. Parameter Estimation (2)

• When raw data are unavailable (data are grouped
  into class intervals), the approximate sample
  mean and variance are
• where fj is the observed frequency of in the jth
  class interval mj is the midpoint of the jth
  interval, and c is the number of class intervals
• A parameter is an unknown constant, but an
  estimator is a statistic.

16
18
4. Parameter Estimation (3) Suggested Estimators
Distribution Parameters Suggested Estimator
Poisson ?
Exponential ?
Normal ?,?2
17
19
4. Parameter Estimation (4)

• Vehicle Arrival Example (continued) Table in the
  histogram example on slide 7 (Table 9.1 in book)
  can be analyzed to obtain
• The sample mean and variance are
• The histogram suggests X to have a Possion
  distribution
• However, note that sample mean is not equal to
  sample variance.
• Reason each estimator is a random variable, is
  not perfect.

18
20
5. Goodness-of-Fit Tests (1)

• Conduct hypothesis testing on input data
  distribution using
• Kolmogorov-Smirnov test
• Chi-square test
• Goodness-of-fit tests provide helpful guidance
  for evaluating the suitability of a potential
  input model
• No single correct distribution in a real
  application exists.
• If very little data are available, it is unlikely
  to reject any candidate distributions
• If a lot of data are available, it is likely to
  reject all candidate distributions

19
21
5. Goodness-of-Fit Tests (2) Chi-Square test (1)

• Intuition comparing the histogram of the data to
  the shape of the candidate density or mass
  function
• Valid for large sample sizes when parameters are
  estimated by maximum likelihood
• By arranging the n observations into a set of k
  class intervals or cells, the test statistics is
• which approximately follows the chi-square
  distribution with k-s-1 degrees of freedom, where
  s of parameters of the hypothesized
  distribution estimated by the sample statistics.

Expected Frequency Ei npi where pi is the
theoretical prob. of the ith interval. Suggested
Minimum 5
Observed Frequency
20
22
5. Goodness-of-Fit Tests (2) Chi-Square test (2)

• The hypothesis of a chi-square test is
• H0 The random variable, X, conforms to the
  distributional assumption with the parameter(s)
  given by the estimate(s).
• H1 The random variable X does not conform.
• If the distribution tested is discrete and
  combining adjacent cell is not required (so that
  Ei gt minimum requirement)
• Each value of the random variable should be a
  class interval, unless combining is necessary, and

21
23
5. Goodness-of-Fit Tests (2) Chi-Square test (3)

• If the distribution tested is continuous
• where ai-1 and ai are the endpoints of the ith
  class interval
• and f(x) is the assumed pdf, F(x) is the assumed
  cdf.
• Recommended number of class intervals (k)
• Caution Different grouping of data (i.e., k) can
  affect the hypothesis testing result.

22
24
5. Goodness-of-Fit Tests (2) Chi-Square test (4)

• Vehicle Arrival Example (continued) (See Slides 7
  and 19)
• The histogram on slide 7 appears to be Poisson
• From Slide 19, we find the estimated mean to be
  3.64
• Using Poisson pmf
• For ?3.64, the probabilities are
• p(0)0.026 p(6)0.085
• p(1)0.096 p(7)0.044
• p(2)0.174 p(8)0.020
• p(3)0.211 p(9)0.008
• p(4)0.192 p(10)0.003
• p(5)0.140 p(?11)0.001

23
25
5. Goodness-of-Fit Tests (2) Chi-Square test (5)

• Vehicle Arrival Example (continued)
• H0 the random variable is Poisson
  distributed.
• H1 the random variable is not Poisson
  distributed.
• Degree of freedom is k-s-1 7-1-1 5, hence,
  the hypothesis is rejected at the 0.05 level of
  significance.

Combined because of min Ei
24
26
5. Goodness-of-Fit Tests (2) Chi-Square test (5)

• Chi-square test can accommodate estimation of
  parameters
• Chi-square test requires data be placed in
  intervals
• Changing the number of classes and the interval
  width affects the value of the calculated and
  tabulated chi-sqaure
• A hypothesis could be accepted if the data
  grouped one way and rejected another way
• Distribution of the chi-square test static is
  known only approximately. So we need other tests

25
27
5. Goodness-of-Fit Tests (3) Kolmogorov-Smirnov
Test

• Intuition formalize the idea behind examining a
  q-q plot
• Recall from Chapter 7.4.1
• The test compares the continuous cdf, F(x), of
  the hypothesized distribution with the empirical
  cdf, SN(x), of the N sample observations.
• Based on the maximum difference statistics
  (Tabulated in A.8)
• D max F(x) - SN(x)
• A more powerful test, particularly useful when
• Sample sizes are small,
• No parameters have been estimated from the data.
• When parameter estimates have been made
• Critical values in Table A.8 are biased, too
  large.
• More conservative.

26
28
5. Goodness-of-Fit Tests (3) p-Values and Best
Fits (1)

• p-value for the test statistics
• The significance level at which one would just
  reject H0 for the given test statistic value.
• A measure of fit, the larger the better
• Large p-value good fit
• Small p-value poor fit
• Vehicle Arrival Example (cont.)
• H0 data is Possion
• Test statistics , with 5
  degrees of freedom
• p-value 0.00004, meaning we would reject H0
  with 0.00004 significance level, hence Poisson is
  a poor fit.

27
29
5. Goodness-of-Fit Tests (3) p-Values and Best
Fits (2)

• Many software use p-value as the ranking measure
  to automatically determine the best fit.
• Software could fit every distribution at our
  disposal, compute the test statistic for each fit
  and choose the distribution that yields largest
  p-value.
• Things to be cautious about
• Software may not know about the physical basis of
  the data, distribution families it suggests may
  be inappropriate.
• Close conformance to the data does not always
  lead to the most appropriate input model.
• p-value does not say much about where the lack of
  fit occurs
• Recommended always inspect the automatic
  selection using graphical methods.

28
30
6. Multivariate and Time-Series Input Models (1)

• Multivariate
• For example, lead time and annual demand for an
  inventory model, increase in demand results in
  lead time increase, hence variables are
  dependent.
• Time-series
• For example, time between arrivals of orders to
  buy and sell stocks, buy and sell orders tend to
  arrive in bursts, hence, times between arrivals
  are dependent.

Co-variance and Correlation are measures of
the linear dependence of random variables
29
31
6. Multivariate and Time-Series Input Models (2)
Covariance and Correlation (1)

• Consider the model that describes relationship
  between X1 and X2
• b 0, X1 and X2 are statistically independent
• b gt 0, X1 and X2 tend to be above or below their
  means together
• b lt 0, X1 and X2 tend to be on opposite sides of
  their means
• Covariance between X1 and X2
• 0, 0
• where cov(X1, X2) lt 0, then b lt 0
• gt 0, gt 0
• Co-variance can take any value between -? to ?

e is a random variable with mean 0 and is
independent of X2
30
32
6. Multivariate and Time-Series Input Models (2)
Covariance and Correlation (2)

• Correlation normalizes the co-variance to -1 and
  1.
• Correlation between X1 and X2 (values between -1
  and 1)
• 0, 0
• where corr(X1, X2) lt 0, then b lt 0
• gt 0, gt 0
• The closer r is to -1 or 1, the stronger the
  linear relationship is between X1 and X2.

31
33
6. Multivariate and Time-Series Input Models (3)
Auto Covariance and Correlation

• A time series is a sequence of random variables
  X1, X2, X3, , are identically distributed (same
  mean and variance) but dependent.
• Consider the random variables Xt, Xth
• cov(Xt, Xth) is called the lag-h autocovariance
• corr(Xt, Xth) is called the lag-h
  autocorrelation
• If the autocovariance value depends only on h and
  not on t, the time series is covariance stationary

32
34
6. Multivariate and Time-Series Input Models (4)
Multivariate Input Models (1)

• If X1 and X2 are normally distributed, dependence
  between them can be modeled by the bi-variate
  normal distribution with m1, m2, s12, s22 and
  correlation r
• To Estimate m1, m2, s12, s22, see Parameter
  Estimation (Section 9.3.2 in book)
• To Estimate r, suppose we have n independent and
  identically distributed pairs (X11, X21), (X12,
  X22), (X1n, X2n), then

Sample deviation
33
35
6. Multivariate and Time-Series Input Models (4)
Multivariate Input Models (2)

• Algorithm to generate bi-variate normal random
  variables
• Generate Z1 and Z2, two independent standard
  normal random variables (see Slides 38 and 39 of
  Chapter 8)
• Set X1 m1 s1Z1
• Set X2 m2 s2(?Z1 Z2 )
• Bi-variate is not appropriate for all
  multivariate-input modeling problems
• It can be generalized to the k-variate normal
  distribution to model the dependence among more
  than two random variables

34
36
6. Multivariate and Time-Series Input Models (4)
Multivariate Input Models (3)

• Example X1 is the average lead time to deliver
  in months and X2 is the annual demand for
  industrial robots.
• Data for this in the last 10 years is shown

Lead time Demand
6.5 103
4.3 83
6.9 116
6.0 97
6.9 112
6.9 104
5.8 106
7.3 109
4.5 92
6.3 96
35
37
6. Multivariate and Time-Series Input Models (4)
Multivariate Input Models (4)

• From this data we can calculate
• Correlation is estaimted as

36
38
6. Multivariate and Time-Series Input Models (5)
Time-Series Input Models (1)

• If X1, X2, X3, is a sequence of identically
  distributed, but dependent and covariance-stationa
  ry random variables, then we can represent the
  process as follows
• Autoregressive order-1 model, AR(1)
• Exponential autoregressive order-1 model, EAR(1)
• Both have the characteristics that
• Lag-h autocorrelation decreases geometrically as
  the lag increases, hence, observations far apart
  in time are nearly independent

37
39
6. Multivariate and Time-Series Input Models (5)
Time-Series Input Models (2)AR(1) Time-Series
Input Models (1)

• Consider the time-series model
• If X1 is chosen appropriately, then
• X1, X2, are normally distributed with mean m,
  and variance se2/(1-f2)
• Autocorrelation rh fh
• To estimate f, m, se2

38
40
6. Multivariate and Time-Series Input Models (5)
Time-Series Input Models (2)AR(1) Time-Series
Input Models (2)

• Algorithm to generate AR(1) time series
• Generate X1 from Normal distribution with mean
  m, and variance se2/(1-f2). Set t2
• Generate ?t from Normal distribution with mean 0
  and variance se2
• Set Xt??(Xt-1- ?) ?t
• Set tt1 and go to step 2

39
41
6. Multivariate and Time-Series Input Models (5)
Time-Series Input Models (3)EAR(1) Time-Series
Input Models (1)

• Consider the time-series model
• If X1 is chosen appropriately, then
• X1, X2, are exponentially distributed with mean
  1/l
• Autocorrelation rh fh , and only positive
  correlation is allowed.
• To estimate f, l

40
42
6. Multivariate and Time-Series Input Models (5)
Time-Series Input Models (3)EAR(1) Time-Series
Input Models (2)

• Algorithm to generate EAR(1) time series
• Generate X1 from exponential distribution with
  mean 1/?. Set t2
• Generate U from Uniform distribution 0,1.
• If U? ?, then set Xt ? Xt-1.
• Otherwise generate ?t from the exponential
  distribtuion with mean 1/? and set Xt??(Xt-1-
  ?) ?t
• Set tt1 and go to step 2

41
43
6. Multivariate and Time-Series Input Models (5)
Time-Series Input Models (3)EAR(1) Time-Series
Input Models (3)

• Example The stock broker would typically have a
  large sample of data, but suppose that the
  following twenty time gaps between customer buy
  and sell orders had been recorded (in seconds)
  1.95, 1.75, 1.58, 1.42, 1.28, 1.15, 1.04, 0.93,
  0.84, 0.75, 0.68, 0.61, 11.98, 10.79, 9.71,
  14.02, 12.62, 11.36, 10.22, 9.20. Standard
  calculations give
• To estimate the lag-1autocorrelation we need
• Thus, cov924.1-(20-1)(5,2)2/(20-1)21.6 and
• Inter-arrivals are modeled as EAR(1) process with
  mean 1/5.20.192 and ?0.8 provided that
  exponential distribution is a good model for the
  individual gaps

42
44
6. Multivariate and Time-Series Input Models (6)
Normal-to-Anything Transformation (NORTA)

• Z is a Normal random variable with cdf ?(z)
• We know R?(z) is uniform U(0,1)
• To generate any random variable X that has CDF
  F(x), we use the variate method
• XF-1(R)F-1(?(z))
• To generate bi-variate non-normal
• Generate bi-variate normal RVs (Z1,Z2)
• Use the above transformation
• Numerical approximations are needed to inverse

43