Bayes Theorem

1
Bayes Theorem

• Mutually exclusive events
• A collection of events (B1, B2, , Bk) is said to
  be mutually exclusive if no two of them overlap.
• If mutually exclusive events together add up to
  the entire sample space, i.e. cover sample space,
  they are called collectively exhaustive events.
  They partition the sample space, like walls
  partition a house.
• Example. B1, , B6

Events B1 B6 partition sample space.
2
Total Probability Formula

• Take any event A. A is the sum of its
  intersections with Bs.
• P(A) P(A and B1) P(A and B2) P(A and B3)
  P(A and Bk)
• Write P(A and Bi) using conditional
  probabilities
• Total Probability Formula
• P(A) P(AB1)P(B1) P(AB2)P(B2)
  P(ABk)P(Bk).

B5
B4
B2
A
B3
B6
B1
3
Example

• A chip manufacturing plant has 3 machines
  producing chips. Machine 1 produces 30 of the
  output and of these, 2 are defective machine 2
  produces 45 of the output and of these, 1 are
  defective machine 3 produces the remaining 25
  chips and of these 3 are defective. Find the
  probability that a randomly selected chip
  produced by this plant is defective.
• Solution. Define events.
• A randomly selected chip is defective
• B1chip was produced by machine 1, P(B1)0.3
• B2chip was produced by machine 2, P(B2)0.45
• B3chip was produced by machine 3, P(B3)0.25.
• P(AB1)0.02, P(AB2)0.01, P(AB3)0.03.
• By the Total Probability Formula
• P(A) P(AB1)xP(B1) P(AB2)xP(B2)
  P(AB3)xP(B3)
• 0.02 x 0.3 0.01 x 0.45
  0.03 x 0.25 0.018.

4
Bayes Formula

• Bayes formula goes the other way from the Total
  Probability Formula.
• We look for conditional probabilities of the sets
  from the partition.
• Bayes Formula
• P(Bi A) P(A and Bi)/P(A)
• P(ABi)xP(Bi)
• -----------------------------------
  ------------------------------------ .
• P(AB1)P(B1) P(AB2)P(B2)
  P(ABk)P(Bk)
• Bayes formula provides conditional probability of
  Bi given A in terms of the other conditional
  probabilities, i.e. P(A given Bi).

5
EXAMPLE

• Chips manufacturing example continued.
• New question. If you bought a defective chip
  produced by that factory, what is the probability
  that it was produced by machine 3?
• Solution. Need P(B3A).
• By Bayes formula,
• 
  P(AB3)xP(B3)
• P(B3A) ---------------------------------------
  --------------------------
• P(AB1)P(B1) P(AB2)P(B2)
  P(AB3)P(B3)
• 0.03 x 0.25
• -----------------------------------
  ------------------------------ 0.42
• 0.02 x 0.3 0.01 x
  0.45 0.03 x 0.25