Probabilistic Approach to Design under Uncertainty

1
Probabilistic Approach to Design under Uncertainty

• Dr. Wei Chen
• Associate Professor
• Integrated DEsign Automation Laboratory (IDEAL)
• Department of Mechanical Engineering
• Northwestern University

2
Outline

• Uncertainty in model-based design
• What is probability theory?
• How does one represent uncertainty?
• What is the inference mechanism?
• Connection between probability theory and utility
  theory
• Dealing with various sources of uncertainty in
  model-based design
• Summary

3
Types of Uncertainty in Model-Based Design

• Model (lack of knowledge)
• Parametric (lack of knowledge, variability)
• Numerical
• Testing data

• Problem faced in design under uncertainty
• To choose from among one set of possible design
  options X, where each involves a range of
  uncertain outcomes Y
• To avoid making an illogical choice

4
Basic Concepts of Probability Theory

• Probability theory is the mathematical study of
  probability.
• Probability derives from fundamental concepts of
  set theory and measurement theory.

Example Flip two coins Sample space ? set
of all possible outcomes of a random experiment
under uncertainty Outcomes e1HH, e2HT,
e3TH, e4TT
Event subset of a sample space

e.g., A e2 and e3 experiments result in two
different faces Probability P(e1)P(e2)P(e3)P(
e4)0.25 P(A) P(e2)P(e3)0.5
P(null) 0 P(?)1
5
Mathematics in Probability Theory

• Three axioms of probability measure
• 0 ? P(A)? 1 P(?)1 P(?Ai)?P(Ai) Ai are
  disjoint events
• Arithmetic of probabilities
• Union, Intersection, and Conditional
  probabilities
• Random variable is a function that assigns a real
  number to each outcome in the sample space
• Probability density function arithmetic of
  moments of a random variable, e.g.,
• EXYEXEY if X and Y are independent
• Convergence (law of large numbers) and central
  limit theorem

Example define x total number of heads among
the two tosses Possible values X0TT
X1HT, TH, X2HH PX10.5
6
Probabilistic Design Metrics in Quality
Engineering
Robustness
Reliability
Probability Density (pdf)
Target M
RArea Probg(x)?c
pdf
Bias
0
Performance y
Performance g
my
C
sy
sy
Minimizing the effect of variations without
eliminating the causes
To assure proper levels of safety for the
system designed
7
Philosophies of Estimating Probability

• Frequentist
• Assign probabilities only to events that are
  random based on outcomes of actual or theoretical
  experiments
• Suitable for problems with well-defined random
  experiments
• Bayesian
• Assign probabilities to propositions that are
  uncertain according to subjective or logically
  justifiable degrees of belief in their truth
• Example of proposition there was life on Mars
  a billion years ago
• More suitable for design problems events in the
  future, not in the past all design models are
  predictive.
• More popular among decision theorists

8
Bayesian Inference

• In the absence of data (experiments), we have to
  guess
• A probability guess relies on our experience with
  related events
• Once data is collected, inference relies on Bayes
  theorem
• Probabilities are always personal degrees of
  belief
• Probabilities are always conditional on the
  information currently available
• Probabilities are always subjective
• Uncertainty of probability is not meaningful.

Bernardo, J.M. and Smith, A. F., Bayesian Theory,
John Wiley, New York, 2000.
9
Bayes Theorem
H - Hypothesis D - Data
Updated by data

• Bayes theorem provides
• A solution to the problem of how to learn from
  data
• A form of uncertainty accounting
• A subjective view of probability

10
Formalism of Bayesian Statistics

• Offers a rationalist theory of personalistic
  beliefs in contexts of uncertainty with axioms
  clearly stated
• Establishes that expected utility maximization
  provides the basis for rational decision making
• Not descriptive, i.e., not to model actual
  behavior.
• Prescriptive, i.e., how one should act to avoid
  undesirable behavioural inconsistency

11
Connection of Probability Theory and Utility
Theory

• Three basic elements of decision
• the alternatives (options) X
• the predicted outcomes (performance) Y
• decision makers preference over the outcomes,
  expressed as an objective function f in
  optimization
• Utility theory
• Utility is a preference function built on the
  axiomatic basis originally developed by von
  Neumann and Morgenstern (1947)
• Six axioms (Luce and Raiffa, 1957 Thurston,
  2006)
• Completeness of complete order
• Transitivity
• Monotonicity
• Probabilities exist and can be quantified
• Monotonicity of Probability
• Substitution-independence

In agreement to employing probability to model
uncertainty
12
Decision Making Ranking Design Alternatives

• Without uncertainty
• objective function f V(Y) V(Y(X))
  V - value
  function, e.g. profit
• With uncertainty
• objective function f
• E(U) - expected utility. The preferred choice
  is the alternative (lottery) that has the higher
  expected utility.

13
Issues in Model-Based Design

• How should we provide probabilistic
  quantification of uncertainty associated with a
  model?
• How should we deal with model uncertainty
  (reducible) and parameter uncertainty
  (irreducible) simultaneously?
• How should we make a design decision with good
  confidence?

• Chen, W., Xiong, Y., Tsui, K-L., and Wang, S.,
  Some Metrics and a Bayesian Procedure for
  Validating Predictive Models in Engineering
  Design, DETC2006-99599, ASME Design Technical
  Conference.

14
Bayesian Approach for Quantifying the Uncertainty
of Predictive Model
- Physical observation
- True but unknown real performance
- Computer model output
- Bias function (between and )
- Random error in physical experiment
Metamodel of
Computer experiments
Physical experiments
Observations (data) of
15
More about the Bayesian Approach
Known parameters (deterministic)
Estimated from data, by MLE or Cross validation
16
Integrated Framework for Handling Model and
Parameter Uncertainties
Given computer model
Sequential experiment design
Computer experiments
Physical experiments
Predictive model and uncertainty
quantification
Parameter uncertainty
Design objective function and
uncertainty quantification
Design validation metrics
Specified confidence level Pth
Design validity requirements satisfied (MD lt Pth)?
No
Yes
Design decision Expected Utility Optimization
17
Uncertainty Quantification of Design Objective
Function with Parameter Uncertainty
A robust design objective
(smaller-is-better) is used to determine the
optimal solutions.
x is a design variable and a noise variable
w1, w2 weighting factors
Uncertainty of
Uncertainty of
Apley et al. (2005) developed analytical
formulations to approximately quantify the mean
and variance of . In this example,
Monte Carlo Simulation is employed.
Mean of
Realizations of
95 PI
18
Validation Metrics
Probabilistic measure of whether a candidate
optimal design is better than other design
choices with respect to a particular design
objective
Larger confidence
Smaller confidence
Three types of design validation metrics (MD)
f is small-the-better

• Type 1 Multiplicative Metric

averaging

• Type 2 Additive Metric

• Type 3 Worst-Case Metric

MD is intended to quantify the confidence of
choosing x as the optimal design among all
design candidates or within design region .
19
Summary

• Prediction is the basis for all decision making,
  including engineering design.
• Probability is a belief (subjective), while
  observed frequencies are used as evidence to
  update the belief.
• Probability theory and the Bayes theorem provide
  a rigorous and philosophically sound framework
  for decision making.
• Predictive models in design should be described
  as stochastic models.
• The impact of model uncertainty and parameter
  uncertainty can be treated separately in the
  process of improving the predictive capability.
• Probabilistic approach offers computational
  advantages and mathematical flexibility.