Modeling Uncertainty

1
Modeling Uncertainty

• Fun with decisions and uncertainty
• Sensitivity analysis
• Preview Monte-Carlo simulation
• Brief review of probability distributions and
  statistics using Excel

2
Warning! Decision and risk analysis is a radical
concept

• People, in general, are not comfortable with
  probabilistic reasoning
• Most people commonly use point estimates for
  uncertain quantities and then may carry out a
  limited 1 or 2 variable sensitivity analysis
• Everyone will say, too much thinking and
  planning required, dont have time in the real
  world
• but somehow, people have time to revisit the
  messes they make with seat of the pants
  decision making

3
Importance and Difficulty of Uncertainty Modeling

• The world is uncertain
• Replacing random quantities with averages or
  single guesstimates can be dangerous
• The Flaw of Averages
• Allows prediction of distribution of results
• Not just one predicted number or outcome
• Sensitivity analysis of outputs to inputs
• Which inputs really affect the outputs?
• Fun with Uncertainty

4
Sensitivity Analysis

• Sensitivity analysis (SA) a big part of modeling
  and analysis
• SA What matters in this decision?
• which variables might I want to explicit model as
  uncertain and which ones might I just as well fix
  to my best guess of their value?
• On which variables should we focus our attention
  on either changing their value or predicting
  their value?
• No optimal SA procedure exists
• SA can help identify Type III errors - solving
  the wrong problem

5
Some SA Techniques

• Scenarios base, pessimistic, optimistic
• How did we do with scenario planning?
• 1-way and 2-way data tables and associated graphs
• as in the Break Even spreadsheet
• Tornado diagrams
• a one variable at a time technique
• Top Rank Excel add-in for simple What if?
• Risk Analysis or Spreadsheet Simulation
• direct modeling of uncertainty through
  probability distributions
• _at_Risk , CrystalBall sophisticated Excel add-ins

6
Tornado Diagrams

• Graphical sensitivity analysis technique
• Create base, low and high value scenarios for
  each input variable
• Set all variables at base value
• Wiggle each variable to its low and high
  values, one at a time.
• A one-way sensitivity analysis technique
• Calculate total profit for each scenario
• Create tornado diagram - Excel

GreatThreads-Tornado.xls
From Making Hard Decisions by Clemen
7
Using Top Rank (see p714-721 in PMS)

• Quit Excel (Top Rank acting flaky)
• Start Programs Palisade Decision Tools Top
  Rank 1.5
• it will launch Excel and start
• Open up your file
• GreatThreadsTornado.xls
• Select output cell and click Add output cells
  on TopRank toolbar
• Click Step through input cells on TopRank
  toolbar and specify which inputs are to be varied
  and by how much
• Click Run what-if analysis on toolbar
• When results come up, click the tornado graph
  toolbar button and select tornado

8
Sensitivity Analysis with TopRank
Big bars means high impact
9
Preview of Monte-Carlo Simulation

• Simple Excel Simulation example
• Revenue for 3 products with fixed prices and
  random demands
• What is simulation
• When do you use simulation
• Real applications of simulation
• Prob distributions the building blocks of
  simulation

10
Monte-Carlo Simulation

• A modeling approach that allows explicit
  incorporation of uncertainty in spreadsheet
  models
• Got its start during the Manhattan project in
  WWII for modeling nuclear devices
• One or more random elements modeled with
  probability distributions
• Sample from the input distributions many, many
  times
• Keep track of the values of the outputs for each
  sampling of the inputs
• Analyze the outputs
• Often called Risk Analysis
• Uncertainty is about values of unknown variables
• Risk is about consequences of uncertainty
• _at_RISK - Palisade Software
• Spreadsheets provide good environment for
  simulation
• Goes beyond expected values and point estimates
• Doing simulation involves more than just building
  models with software
• must be probability/stats literate to do proper
  input and output analysis

11
Our First Simple Simulation Model

• 3ProductSimulation-template.xls
• What is expected revenue?
• Deterministic prices of products A,B,C
• Stochastic demand for products A,B,C
• What is the variability in revenue?

12
A few simulation applications

• T Rowe Price 529 Simulator
• NFL play calling
• http//www.sciencedaily.com/releases/2006/04/06042
  0232621.htm
• http//www.pigskinrevolution.com/index.html
• _at_Risk case studies

13
Building a Spreadsheet Based Simulation Model
(1) Build deterministic model
Formulas
Inputs
Outputs
Deterministic Inputs
(2) Choose inputs to model as random
Inputs
Stochastic or Uncertain or Random Inputs
(3) Model uncertain inputs with probability
distributions
Uniform
Normal
Poisson
Exponential
Empirical
Many more
14
Building a Spreadsheet Based Simulation Model
Running the model
(4.1) Manually, through formulas and either
many rows or VBA (Ex 11.1)
(4) Recalculate spreadsheet many times
2 options
(4.2) Use spreadsheet simulation add-in such as
_at_Risk or Crystal Ball (Ex 11.2)

• _at_Risk
• www.palisade.com

• Crystal Ball
• www.decisioneering.com

15
Probability The Language of UncertaintyDistribut
ions Building Blocks of Simulation

• Random variables
• Discrete probability distributions
• Expected value of a discrete random variable
• Continuous probability distributions
• Using Excels probability and statistics
  functions
• Using the RiskView add-in

You learned about most of the above and more in
your Statistics course. Ill just do a quick
refresher as needed on some concepts well need
for this course.
16
Random variables (RV) and probability
distributions

• RV is a variable whose value depends on the
  outcome of an uncertain event(s)
• Low bid by competing firms, project completion
  date
• Demand for some product or service next year
• Number of patients requiring open heart surgery
  next month at Hospital H
• Cost of Drug X in December, 2004
• Probability of various outcomes determined by
  probability distribution associated with the RV
• Probability distributions are the shapes of
  RVs
• As modelers, we select appropriate distributions
• Probability distributions
• mathematical functions
• Assign numeric probabilities to uncertain events
  modeled by the distribution

See Distributions, Simulation and Excel
Functions handout that Prof. Doane created and
that Ive posted on Web.
17
Using Distributions for Simulation

• We will model uncertain inputs with probability
  distributions
• Need to be able to generate random numbers from
  various probability distributions
• We may fit probability distributions to raw
  data to serve as a convenient model of the data
• Simulation model outputs will be distributions
• Need to know how to compute various measures from
  distributions
• Simulating different scenarios - Need to know how
  to compare features of distributions with each
  other

18
Two Types of Distributions

• Discrete Distributions
• Integer, countable X
• EX of warranty claims in a day
• P(X) is the probability at each point
• P(X) may be summed over X values
• Continuous Distributions
• X defined over an interval
• EX Length of stay for open heart surgery
  patients
• Points have no area
• Calculus gives area under curve

19
P.D.F. vs. C.D.F

• Probability Density Function
• X axis shows values of X
• Y axis shows probability
• S P(X) 1 if discrete
• ? f(x) 1 if continuous
• Histogram is pdf for data
• Cumulative Distribution Function
• X axis shows values of X
• Y axis shows cumulative probability
• 0 ? F(X) ? 1 and is non-decreasing

20
Discrete RVs and Probability Distributions
DistributionReview.xls

• Countable of outcome values
• Each possible outcome has an associated
  probability

Expected Value of Discrete RV

• A few discrete distributions
• Empirical
• Binomial BINOMDIST()
• Poisson POISSON()

Expected Demand
Total Probability
21
Cumulative Distribution Function (CDF) for a
Discrete Random Variable
The probability a random variable X takes on a
value less than or equal to x.
Properties of the CDF
F(x) is nondecreasing in x
22
Distribution Review

• Download DistributionReview.xls
• Lets answer questions on sheet Discrete
• Well do Continuous sheet momentarily
• Excel has many probability and statistical
  related functions
• Remember, probability distributions are a type of
  model for some uncertain quantity
• Think of histograms as empirical probability
  distribution functions

23
Continuous RVs and Probability Distributions

• Infinite of outcome values
• Has a probability distribution (density) function
  (pdf), f(x),
• We calculate probabilities over intervals using
  the cumulative distribution function (cdf), F(x),
  which is PXltb

Area under the f(x) curve from infinity to b
Uniform f(x)
Exponential f(x)
Normal f(x)
24
RiskView

• Excel Add-In, Part of Palisade Decision Tools
  Suite
• Live distribution viewing, Huge number of
  distributions
• Online Help has background info on distributions
• Start Palisade Decision Tools RiskView 4.5
• Can also launch from within Excel from the
  Palisade Decision Tools toolbar (which is visible
  if any of the Palisade tools are running, e.g.
  _at_Risk)

25
A few useful distributions
26
The Normal Distribution

• Two parameters Mean, standard deviation
• Symmetric
• Standard normal distribution has mean0, std
  dev1
• Normally distributed data with any mean and
  standard deviation can be converted to a N(0,1)
  by standardizing

Excel has a number of functions related to the
normal distribution NORMDIST(),
NORMINV() NORMSDIST(), NORMSINV() Lets review
handout Excel Functions for Working with Normal
Distributions and do the Continuous tab in
DistributionReview.xls
27
Descriptive Statistics in Excel

• Data Analysis Tool-Pak

• AVERAGE(), STDEV(), MEDIAN()
• FREQUENCY()
• PERCENTILE()
• RANK(), PERCENTRANK()
• MIN(), MAX()

• 2 ways to create histograms
• Data Analysis Tool-Pak
• Default bins
• User specified bins
• FREQUENCY() array function

StatReview.xls
28
The very special Uniform Random Variable (r.v.)
r.v.
has a distribution of type Uniform with a min0
and max1

• If XUniform(0,1) Then EX1/2 (expected value)
• X is equally likely to take any value between 0
  and 1
• ProbabilityXltx x
• Excels RAND() function

29
Uniform Random Numbers for Simulation

• Building blocks of simulation
• Modeling randomness
• Basis for generating random variables
• Normal, exponential, Poisson, triangular, etc.
• Need reliable stream of Uniform(0,1) RVs
• Excels RAND() function
• How do computers generate random numbers?
• All examples in RandNum_Isken.xls. Lets open it.

30
Uniform Distribution Function
Implication of shape of distribution?
Question
How could you use a U(0,1) number to create a
random number between a and b? Lets do it in
RandNum_Isken.xls
31
Using U(0,1)s to generate other random variables
Walton example
Find U(0,1) random number in cumulative
distribution of random variable you want to
generate.
Return value of random variable.
32
Using U(0,1)s to generate Normal random variables
NORMINV(.1747,160,40)122.57
NORMDIST(122.57,160,40,TRUE).1747
CDF for N(160,40)
Random 122.57
33
Generating Random Numbers

• Excels Data Analysis Tool-Pak
• Excel RAND() along with transformations
• Not possible for all distributions
• _at_Risk functions
• _at_Risk has myriad of functions for generating
  random numbers from a wide variety of
  distributions
• The file ProbabilityDistributions.xls (Downloads
  section of course web) illustrates generating
  various random variables
• www.random.org

34
Some of the broadly applicable insights...

• Explicit incorporation and quantification of
  risks and uncertainties is often important
• Be wary of clairvoyant analysts!
• Several methods for trying to incorporate
  uncertainty in analysis
• Quantification of risk is difficult and subject
  to common human decision biases
• Humans have hard time with uncertainty
• Its important to guard against decision biases
• Awareness is half the battle
• Its OK to say I DONT KNOW
• Not all information is worth the cost or equally
  valid
• Obtaining data for some of these modeling
  approaches can be difficult
• probability estimation can be tough
• historical data may or may not exist

35
What is Simulation?

• A simulation is a computer model that attempts to
  imitate the behavior of a real system or
  activity.
• Simulations helps to quantify relationships among
  variables that are to complex to analyze
  mathematically.
• If the simulations predictions differ from what
  really happens, refine the model in a systematic
  way until its predictions are in close enough
  agreement with reality.

36
What is Simulation?

• When Do We Simulate?

• In general, consider simulation when
• - The system is complex
• - Uncertainty exists in the variables
• - Real experiments are impossible or costly
• - The processes are repetitive
• - Stakeholders cant agree on policy

37
What is Simulation?

• When Do We Simulate?

• Conversely, we are less inclined to simulate when
• - The system is simple
• - Variables are stable or nonstochastic
• - Real experiments are cheap and nondisruptive
• - The event will only happen once
• - Stakeholders agree on policy

38
What is Simulation?

• Advantages of Simulation

• In a deterministic model, variables cant vary.
• Simulation lets key variables change in random
  but specified ways.
• Simulation helps us understand the range of
  possible outcomes and their probabilities.
• Simulation allows sensitivity analysis.

39
Advantages of Simulation

• Simulation is useful because it
• - Is less disruptive than real experiments
• - Forces us to state our assumptions clearly
• - Helps us visualize the implications of our
  assumptions
• - Reveals system interdependencies
• - Quantifies risk by showing probabilities of
  events
• - Helps us see a range of possible outcomes
• - Promotes constructive dialogue among
  stakeholders

40
Risk Assessment

• Risk assessment means thinking about a range of
  outcomes and their probabilities.
• Variation is inevitable.
• Knowing the 95 range of possible values for the
  decision variable as well as the most likely
  value m, is the point of risk assessment.
• Risk assessment is useful when the model is
  complex.

41
What is Simulation?

• Components of a Simulation Model

42
What is Simulation?

• Components of a Simulation Model