AMBIGUITY

1
UNCERTAINTY
2 day short course on Expert Judgment
AMBIGUITY
Roger Cooke Resources for the Future Dept. Math,
Delft Univ. of Technology April 15,16 2008
INDECISION
2
schedule

• DAY 1
• Start up 900
• Ambiguity, Uncertainty, Indecision
  930
• Break 1030
• Expert Judgment for Quantifying Uncertainty
  1100
• Expert Judgment Exercise 1130
• Lunch 1200
• Bumper stickers for Expert Judgment 100
• Break 200
• How to do an Expert Judgment Study 230
• Round Table 400
• DAY 2
• Hands on EXCALIBUR 900
• Break 1030
• Stakeholder elicitation exercise
  1100
• UNIBALANCE demo 1130
• Lunch 1200
• Utilities in Stakeholders Population
  100
• BREAK 230

3
Materials

• On CD

4
Materials

• In Booklet
• EJshortcourse1.ppt
• EJshortcourse2.ppt
• EJshortcourse3.ppt
• EJshortcourse4.ppt
• EJshortcourse5.ppt
• EJcoursenotes-Classical-Model-Boilerplate
• EJcoursenotes-Probability-Intro
• EJcoursenotes-Theory-Rational-Decision
• EJcoursenotes-Subj-ProbRelFreq
• EJcoursenotes-Proper-Scoring-Rules
• EJcoursenotes-Review-Mathematics-Literature
• EJcoursenotes-PI-definitionstheorems
• EJcoursenotes-references

5
Websites Links

• Radiation Protection Dosimetry 90 (2000)
• http//rpd.oxfordjournals.org/cgi/content/short/90
  /3/295
• NUREG EU Probabilistic accident consequence
  uncertainty analysis
• http//www.osti.gov/bridge/basicsearch.jsp
• http//www.osti.gov/energycitations/basicsearch.js
  p
• EU Probabilistic accident consequence uncertainty
  assessment using COSYMA
• http//cordis.europa.eu/fp5-euratom/src/lib_docs.h
  tm
• RFF workshop expert judgment
• http//www.rff.org/rff/Events/Expert-Judgment-Work
  shop.cfm
• TU Delft Website
• http//dutiosc.twi.tudelft.nl/risk/

6
UNCERTAINTY How harmful is 100Gy gamma
radiation In 1 hr?
AMBIGUITY Is John (511) tall?
INDECISION Is LOR better than EOR?
7
(No Transcript)
8
UNCERTAINTY Do measurements, Quantify uncertainty
AMBIGUITY Define concepts, Domain of application
INDECISION Quantify utilities, preferences
9
UNCERTAINTY Experts job
AMBIGUITY Analyst job
INDECISION Stakeholder/problem owners job
10
cant remove uncertainty? Uncertainty Analysis
NUREGCR-6545-Earlyhealth-VOL1.pdf
11
Using Uncertainty to Manage Vulcano risk
responseAspinall et al Geol Soc _.pdf
12
What is Uncertainty?Probability?Fuzzy
sets?Degree of possibility?Certainty
factors?Dempster-Shafer Belief Functions?

• Mathematical representation Axioms
  Interpretation
• Interpretation aka
• operational definitions
• epistemic rules
• rules of correspondence
• etc etc
• squizzel.pdf

13
Operational Definitions

• The philosophy of science semantic analysis
  Mach, Hertz, Einstein, Bohr
• A Modern rendering
• IF BOB says
• The Loch Ness monster exists with degree of
  possibility 0.0731
• to which sentences in the natural language not
  containing degree of possibility is BOB
  committed?

14
Objective and Subjective ProbabilityEJCourseNotes
-Probability-Intro.doc

• Probability formalism is Kolmogorovs axioms, for
  all events A,B
• 0 ? P(A) ? 1
• P(A) 1 P(A)
• If A ? B ? ? P(A?B) P(A)P(B)
• These can be interpreted either
• OBJECTIVELY Limit Relative Frequency, OR
• SUBJECTIVELY Partial Belief

15
Objective Limit relative frequency

• Naïve
• Let A1, A2be independent trials of A, then
  P(A) lim (occurrences in N trials / N)
• Need probability to define independent trials
• Von Mises (1919)
• P(outcome i) lim relative freq of i in a
  kollectif of outcomes, i.e. random sequence
• Need definition of random sequence
• Kolmogorov, Martin-Lof, Schnorr, etc. (60s-70s)
• Random sequence is one which passes all recursive
  statistical tests ? is not predictable by any
  decidable rule.

16
Examples
Heads with bent coin
PROBABILITY Heads 3/10 THIS IS RANDOM
SEQUENCE
Thruster Failure on previous tests
PROBABILITY OF FAILURE ? 3/10 NOT RANDOM
SEQUENCE
USA wins of previous World Cup Soccer
championships
PROBABILITY OF USA WIN ? 0 Not a RANDOM
SEQUENCE
17
Subjective Degree of Partial Belief(Ramsey
1926, Borel, DeFinetti 1937, von Neumann
Morgenstern 1944, Savage, 1954)

• Measure partial belief
• See if it satisfies axioms of probability

18
Operational definition Subjective probability

• Consider two events
• F France wins next World Cup Soccer
  tournament
• US USA wins next World Cup Soccer
  tournament.
• Two lottery tickets
• L(F) worth 10,000 if F, worth 100
  otherwise
• L(US) worth 10,000 if US, worth 100
  otherwise.
• John may choose ONE .
• John's degree belief (F) ? Johns degree belief
  (US)
• is operationalized as
• John chooses L(F) in the above choice situation

19

• If your preferences satisfy principals of
  rationality
• B Belgium wins next World Cup Soccer tournament.
• L(F) gt L(US) L(US) gt L(B) ? L(F) gt L(B)
  ??
• L(F) gt L(US) ? L(F or B) gt L(US or B)
  ??
• (plus some technical axioms)
• Then (Fundamental Theorem of Decision Theory)
• There is a UNIQUE probability P which represents
  degree of belief
• DegBel(F) gt DegBel(US) ? P(F) gt P(US)
• AND a Utility function, unique op to 0 and 1,
  that represents values
• L(F) gt L(US) ? Expd Utility (L(F)) gt Expd
  Utility (L(US))
• PROOF (4 hrs) EJCoursenotes-Theory-Rational-Decisi
  on.doc

20
Can subjective probabilities be relative
frequencies???

• A1, A2.An.. yes-no experiments
• Spose partial belief independent of order
• PA1Y, A2 N, A3N PA1N, A2 N, A3Y
• THEN (barring pathological case)
• P(An1 Y A1Y,A2 N, A3NAnN) ?n? ? Y / n
• PROOF combinatorics (20 min),
• EJCoursenotes-SubjectiveProbRelFreq.doc

21
To clarify

• Subjective probabilities can relative
  frequencies
• You can be uncertain about a limit rel. frequency
• You can learn about a rel. freq. thereby reducing
  your uncertainty
• You can quantify your uncertainty conditional on,
  say, X, and be uncertain about X
• You CANNOT be uncertain about your uncertainty in
  any other useful sense.
• my uncertainty in success is 0.7, but my
  uncertainty in my uncertainty is 0.5, and my
  uncertainty in my uncertainty of my uncertainty
  is 0.3.... DONT GO THERE

22
Other interpretations of Probability axioms

• Classical interpretation (Laplace) ratio of
  favorable cases to all equi-possible cases
• Logical Interpretation (Keynes, Carnap) partial
  logical entailment
• Neither were able to provide successful
  operational definitions.

23
Alternative representations of uncertainty

• Fuzzy sets many axiomatizations, no operational
  definitions
• Degree of Possibility no operational definitions
• (see however Eur. J. of Oper. Res. 128, 459-478.p
  477).

24
CAN fuzziness represent uncertainty?

• µman(Quincy) µwoman(Quincy) ½
• ?
• µman AND woman(Quincy)
• Min µman(Quincy), µwoman(Quincy) ½

25
Lets have a break