Template for MCMA Poster Slides

1
Review

• Probability Axioms
• Non-negativity P(A)0
• Additivity P(A U B) P(A) P(B), if A and B are
  disjoint.
• Normalization P(O)1
• Independence of two events A and B
• P(AnB)P(A)P(B)
• Two coin tosses
• Afirst toss is a head, Bsecond toss is a
  head
• Disjoint vs independent
• P(AnB)0, if P(A)gt0 and P(B)gt0, P(AnB) lt
  P(A)P(B). They are never independent.
• Afirst toss is a head, Bfirst toss is a
  tail
• P(A)0.5, P(B)0.5, P(AnB)0

2
1.3 Conditional Probability 1.4 Total
Probability Theorem and Bayes Rule

3
Conditional Probability

• A way to reason about the outcome given partial
  information
• Example1
• To toss a fair coin 100 times, whats the
  probability that the first toss was a head?
• Fair coin 1/2
• To toss a fair coin 100 times, if 99 tails come
  up, whats the probability that the first toss
  was a head?
• Very small?
• Example2
• A fair coin and an unfair coin (1/4 tail, 3/4
  head)
• The first toss is fair, if the outcome is a head,
  use the fair coin for the 2nd toss, if the
  outcome is a tail, use the unfair coin for the
  2nd toss.
• Whats the probability that the 2nd toss was a
  tail?
• ½x½ ½x¼ 0.375
• Whats the probability that the 2nd toss was a
  tail if we know that the first toss was a tail?
• 1/4

4
Conditional Probability

• Definition of a conditional probability
• The probability of event A given event B (P(B)gt0
  )
• P(AB)P(A) if A and B are
• independent
• A new probability law (recall the definition of
  probability laws)

5
Conditional Probability

• Examples
• Two rolls of a die, whats the probability that
  the first roll was a 1?
• Fair dice 1/6
• Two rolls of a die, the sum of the two rolls is
  6, whats the probability that the first roll was
  a 1?
• B (1,5) (2,4) (3,3) (4,2) (5,1) , A and B (1,5)
• P(AB) (1/36)/(5/36)1/5
• Two rolls of a die, the sum of the two rolls is
  6, whats the probability that the first roll was
  EVEN?
• B (1,5) (2,4) (3,3) (4,2) (5,1) , A and B (2,4)
  (4,2)
• P(AB) (2/36)/(5/36)2/5

6
Conditional Probability

• The new universe is B
• P(A1)gt P(A2), does it mean that P(A1B)gt P(A2B)?
• No!
• An Example Two rolls of a die
• B the sum of the two rolls is 4, (1,3) (2,2)
  (3,1)
• A1 the first roll was 1 or 2
• A2 the first roll was 3, 4, 5 or 6
• P(A1 )1/3 P(A2)2/3
  P(B) 3/36 1/12
• P(A1 n B) 2/36 1/18 P(A2 n B) 1/36
• P(A1 B) (1/18)/(1/12) 2/3 P(A2
  B) (1/36)/(1/12) 1/3

7
Conditional Probability

• The Chain Rule

8
Conditional Independence

• Conditional independence, A and C are independent
  conditional on B, P(B)gt0
• P(AnCB)P(AB) P(CB)
• Example (conditional independence ?
  independence) unfair coins, coin 1- (0.9 head,
  0.1 tail) coin 2- (0.1 head, 0.9 tail), coin 3 is
  fair.
• Toss coin 3 first. If its head, toss coin 1
  twice. If its tail, toss coin 2 twice.
• A X H X, the event that the 2nd toss is a head
  
• C X X H, the event that the 3rd toss is a head
• B H X X, the event that the first toss is a head

9
Total Probability Theorem

• A1 , A2, An be a partition of O
• Recall the definition of a partition
• Total Probability Theorem

10
Total Probability Theorem

• An example
• A fair coin and an unfair coin (1/4 tail, 3/4
  head)
• The first toss is fair, if the outcome is a head,
  use the fair coin for the 2nd and 3rd toss, if
  the outcome is a tail, use the unfair coin.
• B the 2nd and 3rd tosses are both tails
• A1 the first toss is an head, A2 the first
  toss is a tail. A1 and A2 is a partition of the
  universe.. P(A1)P(A2) 1/2
• P(BA1 ) 1/4, P(BA2 ) 1/16

11
Bayes Rule

• A1 , A2, An be a partition of O
• Bayes Rule

12
Bayes Rule

• An Example
• Question
• How likely is there a tumor given that a shade is
  observed?
• P(A2 B)

13
Bayes Rule

• Bayes Rule from scratch

14
Sending a bit through a noisy channel

• Sender has a bit b- either 0 or 1 with equal
  probability to send to the receiver
• p0.1
• Question1 if the sender sends b once, and the
  receiver receives 1, what can the receiver say
  about b?
• Question2 if the sender sends b 3 times, and the
  receiver receives 1,1,1 what can the receiver say
  about b?
• Question3 if the sender sends b 3 times, and the
  receiver receives 1,0,1 what can the receiver say
  about b?