A Probabilistic Treatment of Conflicting Expert Opinion

1
A Probabilistic Treatment of Conflicting Expert
Opinion
Luc Huyse and Ben H. Thacker Reliability and
Materials Integrity Luc.Huyse_at_swri.org,
Ben.Thacker_at_swri.org 45th Structures,
Structural Dynamics and Materials (SDM)
Conference 19-22 April 2004 Palm Springs, CA
2
Motivation

• Avoid arbitrary choice of PDF
• Account for vague data
• Efficient computational tools
• Account for model uncertainty

3
Probabilistic Assessment

• Choice of PDF
• Companion paper
• Dealing with (conflicting) expert opinion data
• Use Bayesian estimation
• Efficient Computation
• Method must be amenable to MPP-based methods
• Epistemic Uncertainty in the decision making
  process
• Minimum-penalty reliability level

4
Estimation with Interval Data

• Use Bayesian updating
• Bayesian updating equation for intervals is

5
Non-informative Priors and the Uniform
distribution

• Temptation is to assume uniform distribution when
  nothing is known about a parameter
• Non-Informative does NOT necessarily mean Uniform
• Illustration
• Choose uniform for X because nothing is known
• Choose uniform for X2 because nothing is known
• Rules of probability can be used to show that PDF
  for X2 is NOT uniform
• Selecting a uniform because nothing is known is
  not justified

6
Transformation to Uniform

• Transformation t exists such that random variable
  X can be transformed t X ? Y where Y has a
  uniform PDF.
• Question is no longer whether a uniform PDF is an
  appropriate selection for a non-informative prior
  but under which transformation t X ? Y the
  uniform is a reasonable choice for the
  non-informative distribution for Y.

7
Data-translated Likelihood

• Uniform PDF is non-informative if the shape of
  the likelihood does not depend on the data
• Jeffreys principle uniform PDF is appropriate
  in space where likelihood is data-translated.

8
Updating with Interval Info

• Variable y has a Poisson PDF estimate mean value
  of Y
• Non-informative prior used
• Consider six different updates for mean
• Posterior variance decreases as interval narrows
• Weight of expert depends on length of their
  interval estimate.

9
Combining Interval Point Data

• Variable y has a Poisson PDF estimate mean value
  of Y
• Non-informative prior used
• Consider five updates for mean
• Posterior variance reduces with successive
  addition of precise observations
• Narrow interval contains almost as much
  information as point estimate
• Wide interval estimate still adds some information

10
Conflicting Expert Opinion

• Source of conflicting expert opinion
• Elicitation questions not properly asked or
  understood
• Correct through iterative expert elicitation
  process
• Each person susceptible to differences in
  judgment
• Weighting of expert opinion data has been
  proposed
• Difficult to determine who is more right.
• Adding weights to experts is therefore a matter
  of the analysts judgment, and should be avoided.
• Proposed approach
• Each expert opinion treated as a random sample
  from a parent PDF describing all possible expert
  opinions.
• Weight is related to width of interval
• Conflict accounted for automatically in the
  updating process

11
Treatment of Conflicting Data

• Variable y has a Poisson PDF estimate mean value
  of Y
• Assume discrepancy in expert data is not due to
  misinterpretation and all data equally valid
• Uncertainty in y continues to narrow as data are
  added
• Effect of conflict is to shift the posterior
  distribution

12
Treatment of Model Uncertainty

• Separate inherent (X) and epistemic (Q) variables

13
Efficient Computation

• Because of model uncertainty Q, b (safety index)
  is a random variable
• Interval estimates with confidence level
• Compute CDF of b
• Exact confidence bounds determined from CDF
• Usually requires numerical tool ? NESSUS
• First-Order Second-Moment Approximation
• Requires only a single reliability computation
  using the mean value of epistemic variables Q

14
Analytical Example

• Limit State Function
• g X Q/100
• pf Prglt0
• Assume X is exponential PDF with uncertain mean
  value l
• Q represents model uncertainty assume
  Normal(1,s), with s 0.3
• Estimate the l using 5 interval data (shown)
• Reliability b (related to pf) is a function of
  epistemic parameters l and q

15
Uncertain Reliability Index
Confidence bounds shrink when more information is
available
16
Decision Making with Epistemic Uncertainty

• In a decision making context, a penalty p(b) is
  associated with using the wrong reliability
  index the expected value of the total penalty
  is
• Minimum penalty reliability index minimizes the
  expected value of the total loss (Der Kiureghian,
  1989)

17
Cost function and bmp

• Linear penalty function
• k is a measure for the asymmetry of (usually gt 1)
• Minimum penalty reliability index (Der
  Kiureghian, 1989)
• Normal Approximation

18
Minimum-Penalty Reliability Index

• bmp is a safe reliability level
• This level strongly depends on the severity of
  the consequence (value k)
• bmp increases with the number of experts

k 1
k 5
k 20
19
Summary

• Proposed method handles both precise and interval
  (expert opinion) data within probabilistic
  framework
• Conflicting information automatically accounted
  for
• Minimum-penalty reliability index can be
  estimated from a single reliability computation ?
  Highly efficient
• Allows effect of epistemic uncertainties to be
  determined
• Companion paper (tomorrow) will discuss use of a
  distribution system, whereby the data can
  determine the shape of the distribution as well
  as any parameter

20
Future Work

• Amenable to MPP-based solution (future work)
• Link to pre-posterior analysis, compute
  sensitivity of design decision to epistemic
  uncertainty.

21
Thank You!
Luc Huyse Ben Thacker Southwest Research
Institute San Antonio, TX