Overview of Bayesian decision theory

1
Overview of Bayesian decision theory

• Chapter 2 in Duda et. al.

2
Problem formulation

• We have c classes (categories)
• Each class has a prior probability of
  occurenceand a class conditional pdf
• The classifier is given by a decision function

What classifier minimizes the probability of
error?
3
Problem solution for two category case
Choose
4
Generalizations

• Each decision is associated with an action
• Given an action and the true state of nature, we
  have a loss
• Associated with an action we have an expected
  loss given the observation
• The average risk is

What classifier minimizes the average risk?
5
What more in Chapter 2?

• Cost functions
• Zero-one loss (minimize error probability),
• minimax (minimize maximum possible risk max over
  the priors),
• Neyman-Pearson (minimize risk subject to some
  constraint)
• The Gaussian case
• Decision boundaries that are quadratic functions
• Error probabilities
• For the Gaussian case Chernoff bound -gt
  Battacharayya bound
• Missing/noisy features
• Classification of sequences