Uncertainty

1
Uncertainty Decision Making

• James K. Hammitt
• Harvard Center for Risk Analysis

2
Outline

• Uncertainty aversion value of information
• Representing uncertainty as probability
• Policy evaluation
• Components of uncertainty
• Examples
• Diesel-vehicle emissions
• Mercury from power plants
• Expert judgment

3
Aversion to Risk, Uncertainty, Ambiguity,
Ignorance

• Humans dislike absence of certainty
• Risk "objective" probabilities
• Uncertainty subjective probabilities
• Ambiguity unknown probabilities
• Ignorance unknown possible outcomes
• Should we take greater precaution when risks are
  more uncertain?
• How should we describe uncertainty?

4
Perils of Prudence(Nichols Zeckhauser 1986)

• Conservative assumptions, worst-case analysis,
  uncertainty aversion can increase harm
• Technology Deaths Probability Expected deaths
• Uncertain 1 0.99
• 1,000 0.01 11
• Certain 101 1.0 101
• Using upper-bound risk estimates, Certain would
  be preferred to Uncertain

5
Perils of Prudence

• If decision is repeated for 10 pairs of
  technologies (and risks are independent)
• Technology Deaths Probability
• Uncertain 10 0.904
• lt 1,010 0.996
• Certain 1, 010 1.0
• Policy of choosing Certain (with smaller
  upper-bound risk) is almost sure to kill more
  people

6
Value of Information

• For each of 10 technologies, learn true number of
  deaths for ambiguous type
• Choose Uncertain if it causes 1 death
• Choose Certain otherwise
• Choice Expected deaths
• Uncertain (always) 110
• Certain (always) 1,010
• Perfect information 20
• Expected value of information 90 lives saved

7
Value(s) of Information

• Increase chance of choosing decision that is best
  for actual conditions
• "Expected value of information" in decision
  theory
• Overcome burden of proof needed to depart from
  status quo policy or default assumption
• Compensate for decision rule that does not
  maximize expected value of outcome
• Reassure decision makers and affected public that
  decision is appropriate
• Enhance compliance, minimize opposition legal
  challenges
• Incorporate compliance and challenges as factors
  in analysis?

8
Quantifying Uncertainty with Probability

• Probabilities of health risks are subjective
• Often extrapolated from animal experiments or
  observational human data
• Quantitative measure of degree of belief
• Individuals can have different probabilities for
  same event
• There is no "true" or "objective" probability
• All probabilities are subjective
• "Objective randomness" is not random but chaos
  (e.g., coin toss, roulette wheel)
• Deterministic process
• Sensitively dependent on initial conditions
  (butterfly flapping wings in China may cause
  hurricane in Atlantic)
• Insufficient information about initial conditions

9
Disagreement Among Experts

• Individuals can hold different probabilities
• When evidence to choose among them is inadequate
• As evidence accumulates
• Experts should update their probabilities
• "When somebody persuades me that I am wrong, I
  change my mind. What do you do?" - John Maynard
  Keynes
• Ultimately, probabilities should converge
• Coin toss, roulette wheel
• "In the long run we are all dead."- John Maynard
  Keynes

10
Quantifying Uncertainty About Policy Outcomes

• Use simulation model to combine multiple inputs
• Inputs releases to environment, fate
  transport, human exposure, dose-response function
• Outputs adverse health events, benefits and
  costs
• Represent uncertainty about each component of
  model as probability distribution
• Calculate probability distribution of output
  using Monte Carlo analysis (or alternatives)

11
Components of Uncertainty

• "Model uncertainty"
• Functional form
• Causality
• "Parameter uncertainty"
• Sampling variation in data (estimation error)
• Relevance of data to application
• May be helpful to distinguish, but can combine
  using "super-model"
• Weighted sum of alternative models, weights are
  uncertain parameters
• Note statistical confidence intervals are not
  sufficient exclude many important sources of
  uncertainty

12
Example Low-Dose Extrapolation

• Estimate risk at high dose, where risk is
  measurable (e.g., 1/10, 1/100)
• Extrapolate to risk at low dose
• Extrapolation can be sensitive to choice among
  models that fit observed data equally well

13
10-2
0.5
Dose Response
Low Dose Extrapolation
10-5
Probability of Response
Excess Risk
0.25
10-8
X
M
WL, G
P
0
10-6
10-2
102
0
75
150
Dose d (ppm)
X Linear Extrapolation L Logit Model M
Multi-Stage Model G Gamma Multi-Hit Model W
Weibull Model P Probit Model
Low-dose extrapolation for 2-acetylaminofluorene
under several mathematical models.
14
Policy-Evaluation Examples

• Retrofit diesel trucks buses in Mexico City
• Benefits of reducing mercury emissions from
  electric power plants

15
Diesel Retrofit Benefit-Cost Model
16
Annual Deaths Averted (per 1000 vehicles) (Error
bars show interquartile range)
17
Net Benefits of Catalyzed Filter v. Alternatives
(US millions per 1000 vehicles, model year
1994)
18
Relative importance of uncertainty about
variables (Interquartile range, annual net
benefits of retrofitting all vehicles with active
regeneration filters holding other variables
fixed at medians)
19
Mercury from Power Plants
Emissions
Deposition
Fate Bioaccumulation
IQ loss? Heart attack?
Exposure
20
Summary of Benefits
21
Benefits of Reducing MeHg Intake 10 (US)
22
Relative importance of uncertainty about
variables (Correlation of input and output)
23
Benefits Sensitivity to Key Parameter
24
Expert Judgment

• Risk assessment models incorporate many
  assumptions
• Choices usually made by modelers, informed by
  scientific literature
• Meta-analysis can be used when literature is rich
• Alternative (or complement) expert elicitation
• Experts provide probability distributions for key
  parameters
• Rigorous, replicable process
• Selection of experts
• Preparation
• Interview

25
Key Elicitation Question (Mortality Effect of
PM2.5)

• "What is your estimate of the true percent change
  in annual, all-cause mortality in the adult U.S.
  population resulting from a permanent 1µg/m3
  reduction in annual average PM2.5"
• 5th, 25th, 50th, 75th, and 95th percentiles of
  cumulative density function

26
Source EPA PM NAAQS RIA 2006
27
Source EPA PM NAAQS RIA 2006
28
Performance Expert Predictions of Ambient
Benzene Concentrations
Means
90th Percentiles
Source Walker et al. 2003
29
Conclusions

• Outcomes of any policy alternative are uncertain
  ex ante
• Characterize uncertainty as probability
  distributions
• Propagate uncertainties about model inputs using
  Monte Carlo analysis
• Agreement on probabilities may not exist
• Pattern of precautionary policies may be costly