Monte Carlo

1
Monte Carlo A Car, A Place, A . . . ?
Six Sigma Alliance, LLC
2
Interactive Presentation Topics

• How Is Monte Carlo Simulation Used To Solve
  Business Problems?
• Whatre The Steps In A Good Simulation Process
• How Does Crystal Ball Software Make the Above
  Easy

3
Predicting Variability
How (Today) Can You Predict the Variation in the
Gap Dimension?
Gap
Specifications for Assembly Gap - 0.0000,
0.0100
Next Question Why Would We Care About the
Variation in the GAP?
4
Some Variation Challenges

• Design Can We Produce This Thing?
• Design - Will This Design Meet Customer
  Requirements (i.e. Upper/Lower Specifications)?
• Finance What Are the Uncertainties in Our
  Financial Predictions?
• Business Process Design What Queues Will Build
  Up, Wait Times for Customers, Average and
  Variation in Service Time, Cost?
• Improvement Projects If We Buy This Lower
  (Higher) Cost Part, What Impact Will It Have on
  Our Quality?

5
Simulation Models

• What is a Simulation Model?
• A representation of a real system (e.g. computer,
  mockup, etc.)
• What are Simulation Models used for?
• Evaluate the trade-offs between performance and
  resource requirements and to determine the
  optimal design
• When are Simulation Models used?
• When the process or system is complex
• When the risk of failure of the real system is
  high

6
Monte Carlo Simulation

• Special Type of Discrete Event Simulation
• The Effect (Y) Can be Described as a Function of
  the Factors (Xs)
• Some or All of the Causes/Factors Can be
  Described Via Probability Distributions (e.g.
  Normal, Weibull, Poisson)
• Monte Carlos Goal
• Describe the Distribution of the Effect (Y)
• Monte Carlo Method
• Repeatedly Sample the Input Factors (Xs) and
  Compute/Record the Effect (Y)

7
Crystal Ball Monte Carlo Simulator

• Features, Functions Limitations
• Add-on to Microsoft Excel
• Can Propagate Variability In Xs To Variability
  In Ys
• Includes A Wide Variety Of Probability
  Distributions
• Requires That An Explicit Mathematical
  Relationship Be Defined (Y FXs)
• Treats Each Simulation Run As An Independent
  Event Doesnt (Easily) Model Queuing Or Other
  Throughput Factors
• (With OptQuest) Will Optimize The Y as a Function
  of The Xs

8
Typical Crystal Ball Applications

• Predict Variability In Overall Component Length
  As A Function Of Parts Variability
• Predict Variability In Revenues, Profit, ROI As A
  Function Of Variability In Sales, Market
  Conditions, Expenses, etc.
• Predict Reliability Of A Mechanical Device As A
  Function Of Variability In Stresses Imposed On
  The Device And The Strength Of Devices
  Materials
• In General, Predict Probability Of A Desired
  Outcome As A Function Of Variability In Inputs
  (i.e., Desired Profit, Sales, Volumes, Risk)

9
Modeling/Simulation Process

• Understand Problem To Be Studied And Objective Of
  Doing Simulation. Develop A Project Plan And
  Define Roles Responsibilities
• Describe Model Based On Expert Interviews And
  Observation Of Process
• Collect Data Needed To Define Process Properties
• Prepare Software Model (Using Crystal Ball, Other
  Method)
• Determine That Computer Model Executes Properly
  Compare Model Output With Real Process (If
  Exists)
• Establish The Experimental Options (Scenarios) To
  Be Simulated
• Execute Options (Scenarios) And Collect
  Performance Measures
• Analyze Simulation Results
• Make Recommendations

1. Specify
2. Develop
3. Quantify
4. Implement
5. Verify Validate
6. Plan
7. Conduct
8. Analyze
9. Recommend
10
1. Specify the Problem

• Clarify The Problem/Question
• Have Clearly Stated And Accepted Objectives
• Get Input From Users, Customers And Stakeholders
• Make Certain All Agendas Are Understood
• The Customers, Process Owners, Staff, As Well As
  Management
• Clarify Roles And Responsibilities
• Customer - Define And Refine Study Goals, Take
  Action Based On Study Results
• Process SMEs - Provide Information On
  Product/Process Workings, Support Data Collection
• Analyst - Build Accurate Model Of Product/Process
  To Support Study Goals
• Statistician - Perform Input And Output Data
  Analysis And Evaluation
• Develop A Project Plan (e.g. Gantt Chart) Of
  Modeling/Simulation Project Activities (Integrate
  With Design or Improvement Project Plan)

11
Crystal Ball Problem Assembly Gap
A Mechanical Assembly Consists of Four Parts
Fitted Together as Shown Below
Problem Statement Given the Current Design
Dimensions and Manufacturing Tolerances, What is
the Likelihood of an Assembly Being Produced with
Excessive Interference?
12
2. Develop the Model

• Define Output Variables (Ys) Needed To Support
  Study Objectives
• e.g. Physical Quantity, Cycle Times, DPMO
• Study Real-World Product or Process
• e.g. Interviews, Documentation, Observations
• Identify Real-World Counterparts To Model
  Elements (Xs)
• Process Entities, Activities, Resources
• Develop The Mathematical or Logical Relationship
  Of Model Elements To Output Variables (i.e.
  Transfer Function)
• Engineering, Financial or Physical Relationship
• Flowchart/Process Map With Resources
• Review Model Against Real-World Process
• The Initial Sanity Check (The Beginning of Step
  5. Verification)

13
The Assembly Gap Model

• In This Case, the Model is Simple
• Assembly Gap L4 (L1 L2 L3)
• Other Models May Include Complex Engineering
  Relationships, Financial Calculations
• Some Models May Include a Time-Dimension e.g.
  Forecasting Sales, Revenue, Population, Pollution
  or Radiation Dispersion

14
Seven Principles of Effective Modelers

• Stays Oriented Toward Project Goals/Purpose
  Model Construction Is Not The End
• Includes Only Necessary Detail Practices KISS
• Evolves Model Over Time Starts Simple And Adds
  Complexity Until The Model Suits The
  Goals/Purpose
• Describes All Critical Activities/Events With
  Appropriate Detail
• Flexible Makes Model Design Easy To Modify
  Because It Will Change
• Robust Doesnt Make Model Applicability Narrow
  Through Structure Or Assumptions
• Clearly Displays Results Makes Sure All
  Measured Responses Are Available And Understood

15
Installing Crystal Ball Demo Package

• Well Lead You Through Installation of the Demo
  (Good for Seven Days)

16
Crystal Ball Toolbar in Excel
Commands to Define Input and Output Variables,
Probability Distributions
Assumptions include the probability distribution
for the variable, plus the necessary parameters
to specify the distribution (e.g. mean and
standard deviation for the Normal Distribution)
Defining a cell as a Forecast tells Crystal Ball
that this variable is an output or Y for the
simulation. Multiple Forecast cells may be
defined.
17
Crystal Ball Toolbar in Excel (continued)
Commands to Set Up and Run the Simulation, Reset
for Another Run and Debug the Simulation (Single
Step)
18
Crystal Ball Toolbar in Excel (continued)
Commands to View and Analyze the Results of the
Simulation Runs Obtain Crystal Ball Help
19
Crystal Ball Commands
Alternate Access to Crystal Ball Commands
20
3. Quantify the Model
Variable Variables Which Factors/Variables
Should be Assigned Probability Distributions?
Distributions Which Distributions Best Fit the
Variation Associated With the Variable
Variables?
X1
?
Y
Y f(X1, X2, X3)
X2
X3
Constants Which Variables Can Be Left as
Constants? (Not Everything Needs to be Variable)
21
Available Distributions in Crystal Ball
22
Typical Distributions Applications
Normal Machining Processes, Biological Population Characteristics (Height, Weight, etc.)
Log-Normal Time-to-Complete Process Steps, Particle Size Following Crushing Application
Weibull Time to Failure of Many Devices, Strength of Materials, Height of Males in British Isles
Exponential Inter-Arrival Time of Events (e.g. Time between Failure for Complex Systems)
Uniform Machining Process With Automatic Adjustment Close to Spec Limits
Binomial Go/No-Go Processes, Counts of Defective Items, Coin Toss
Poisson Counts of Defects/Item, Arrival Rate of Events (e.g. Number of Failures/Time Period)
One of Waloddi Weibulls Original Applications
of this Important Distribution
23
Obtaining Model Data

• Determine Data Needs From Objectives And Model
  Elements
• Desired Output Variables And Questions About
  Inputs
• Start Data Collection Early
• Data Collection Is Time Consuming
• Check Existing Databases For Availability
• Perform Experiments/Measurements On Real System

24
Fitting Data to a Distribution Crystal Ball
2. Use Historical Data as Input to Crystal Ball
Best Fit Distribution Will Be Recommended . .
1. Choose Fit... Command in Define Assumption
Dialog Box . . .
25
4. Implement the Model

• Build The Model In Crystal Ball or Other
  Simulation Software (ProcessModel, ARENA, etc.)
• Define Assumptions (e.g. Probability
  Distributions) for Input Variables
• Define the Output Variables As Forecast Cells
• Setup the Simulation Preferences
• Test and Debug the Model

26
Implementing the Model
The Functional Relationship between the Effect (Y
Assembly Gap) and Factors (X Part Dimensions)
is Entered In Excel
27
Quantifying the Model
Probability Distributions are Assigned to the
Factors
The Effect is Then Assigned to be a Forecast Cell
28
5. Verify and Validate Model

• Verification Ensuring That The Conceptual Model
  Is Faithfully Represented By The Implementation
  (In Software)
• Validation Ensuring That The Model Behaves The
  Same As The Real System
• Apply To Implemented Model And System Outputs
• Focuses On Accuracy Of Measured Properties

Develop Quantify Model
Implement Model
Real Process
Verify
Validate
29
Running the Model
The Distribution of the Effect (Assembly Gap) is
Developed Through Repeated Calculations, Drawing
Each Calc From the Factors Probability
Distributions
30
Crystal Ball Output Information/Statistics
Info Description/Interpretation
Frequency Chart Histogram of results of running the model for the set number of trials. Examine shape (probability distribution), can determine probability of less than, greater than
Cumulative Chart ( Reverse) Show the cumulative probability of output variable. Examine shape (similar to cumulative probability distribution), can determine probability of less than, greater than
Statistics Provides Descriptive Statistics for output variable (mean, median, variance, skewness, kurtosis, etc.).
Percentiles Use this view to quickly determine what percent of the data falls below a certain output variable value.
Preferences Menu From Forecast Display, the Output Chart/Statistics can be modified Experiment with Chart Type commands to see how the datas appearance can be changed.
Overlay Chart Allows Comparisons of Output Variables Distributions on a Single Graph.
31
Interpreting the Model Output Variation
The Likelihood of the Assembly Gap Being Less
Than 0 (Interference) Can Be Easily Determined
from the Forecast Graph
32
Interpreting the Model Output Distribution
1. Use Run Extract Data Command to Obtain Raw
Output Data
2. Select a Cell, Enter a Bogus Value and Click
on Define Assumption Button
3. Click on Fit Button and Define Range of Data
Corresponding to Output
4. Examine Possible Distributions and Select Best
Fit
33
Interpreting the Model - Sensitivity
Sensitivity Chart Displays Factors Contribution
to the Y Variation Identifies Best
Opportunities for Improvement
34
6. Plan Experiments
Term Definition
Scenarios Trials In Which The Value Of One Or More Model Factors Are Varied From Run To Run
Single Factor Experiments A Series Of Scenarios In Which Only One Factor Is Varied (Analyzed Via Hypothesis Test, Contingency Table Or One-Way ANOVA)
Multiple FactorExperiments A Series Of Scenarios In Which Multiple Factors Are Varied (Design Of Experiments Used To Plan Scenarios, ANOVA or Signal-To-Noise Ratio Used to Analyze Results)
35
7. Conduct Experiments
What Changes Could Be Tested in the Crystal
Ball Model to Improve the Gap Performance?
1. 2. 3. 4. 5.
36
8. Analyze Results

• Goals
• Use Simulation Data Model To Make Inferences
  About The Performance Of A Real Or Proposed
  System
• Use Simulation Data Model To Compare The
  Performance Of Alternative Systems
• Method
• Determine Point And Interval Estimates For One Or
  More System Parameters Using Simulation Output
• For Alternatives, Perform Hypothesis Testing (Or
  Other Appropriate Procedure) To Detect Differences

37
The Simulation Analysis Process
Real Process
Model
Simulation Runs
Output Analysis
SpecLimit

• Inferences
• Mean
• Variation
• Proportions
• Sigma, DPMO

38
9. Make Recommendations

• Report Findings And Recommendations To Study
  Customer(s), Stakeholders
• Document Analysis, Assumptions, Models, etc.
• Avoid Statistical/Simulation Jargon With
  Non-Technical People

39
Post Study Actions

• When Changes Are Made To Real-World Product or
  Process, Collect Variable (Xs) And Performance
  Data (Ys)
• Questions
• Does Change/New Process Perform As Predicted?
• Are Gaps Within Reason, Or As Expected?
• In Either Case, Ask Why?
• Record Lessons Learned For Future Modeling/
  Simulation Projects

40
When Not to Use Simulation

• When Analytical Models Are Available
• Think First, It Can Save Time And Effort
• When Behavior Is Deterministic And The System Is
  Simple
• When There Is No Expertise In Output Analysis
• When The System Has Intelligent Agents Deciding
  Actions
• In Difficult-To-Model Negotiation or Adversarial
  Situations

41
Simulation Limitations

• Cannot Give Accurate Results If The Data Are
  Inaccurate (Garbage In, Garbage Out)
• Cannot Describe Process Characteristics That Have
  Not Been Explicitly Modeled
• Simulation Does Not Optimize However It Can
  Provide The Function To Be Optimized

System Response
Optimization Search Strategy
Simulation Model
Control Variables
42
Time for One More???
Shampoo Bottle Design

• Filling Operation
• Average Volume 1250 cm3
• Std. Deviation 5 cm3
• Distribution Normal
• Container Molding Process
• Average Radius 3.99 cm.
• Std. Deviation 0.01 cm.
• Distribution Normal
• Key Relationship h V/(? x r2)
• Hint Excels ? function is PI()

Fill Height 25 cm /- 0.5 cm
43
References

• Tolerance Design A Handbook for Developing
  Optimal Specifications, C. M. Creveling,
  Addison-Wesely, ISBN 0-201-63473-2, 1997.
• Simulation Modeling and Analysis, Averill M. Law
  W. David Kelton, McGraw-Hill, ISBN
  0-07-036698-5, 1991.
• Crystal Ball Software, Decisioneering, Inc.
  (www.decisioneering.com)