Lecture 5. Reminder Warming-up Conditional Probability. Independence

1
Lecture 5. Reminder Warming-upConditional
Probability.Independence
2
Reminder
3

• A few rules
• Complement Rule The probability that A does not
  occur is equal to the probability that the
  complement of event A occurs. P(Ac) 1 - P(A).
• Difference Rule If A is a subset of B, then the
  probability of B occurring but not A is P(B) -
  P(A) P(B Ac).
• Inclusion-Exclusion Rule The probability of
  either A or B (or both) occurring is P(A U B)
  P(A) P(B) - P(AB).
• Solve it
• If the probability of having blew eyes is 10,
  the probability of having dark hair is 75, and
  the probability of being a blue-eyed dark haired
  person is 9, what is the probability of
• not having blue eyes? find P(Ac)
• having blue eyes but not dark hair? find P(A) -
  P(AB)
• having blue eyes and/or dark hair? find P(A U
  B)

4
Warming up
5

• Consider three problems
• You make a new friend and you ask if he has any
  children. Yes, he says, two. Any girls? Yes, he
  says. What is the probability that both are girls?

(b) You make a new friend and you ask if she has
any children. Yes, she says, two ages 6 and 9.
Is the oldest a girl? You ask. Yes, she says.
What is the probability that both of her children
are girls?
(c) You make a new friend and you ask if he has
any children. Yes, he says, two. Any girls? Yes.
Next day you meet her with a young girl. Is this
your daughter? Yes, she says. Whats the
probability that both of her children are girls?
6
Seemingly, the problem (c) is not different from
( a). Why are the answers different? At
Home Please, try to figure out the nature of
this apparent paradox.
7
Solution of the extra credit question
(HW4)   Here is a solution to the test problem
2.1 offered by one of the students. Problem 2.1 A
committee of 5 is chosen from a group of 8 men
and 4 women. What is the probability the group
contains a majority of women? Suggested
Solution For a group to contain a majority of
women there are two possibilities wwwwm and
wwwmm. The probabilities are P(wwwwm)
(4/12)(3/11)(2/10)(1/9)(8/8) 0.002. P(wwwmm)
(4/12)(3/11)(2/10)(8/9)(7/8) 0.014. P(majority)
0.002 0.014 0.016. Question You can see
that this answer differs from the solution
discussed in the class. At the same time, the
calculation of P(wwwmm) and P(wwwmm) is
apparently correct. Please, fix the solution
using the same approach.  
8
Solution
The solution correctly described two of many
contributions to the event A (majority of
women), namely wwwwm and wwwmm. The total number
the contributions are C(5,1) 5 for the first
group (4 women, 1 man) and C(5,2) 10 for the
second group (3 women, 2 men). Its easy to
verify that, regardless the order the
probabilities within each group are the same. For
instance, P(wwwwm) P(wwmww) and
P(wwwmm)P(wmwwmw). As a result, P(A) C(5,1)
P(wwwwm) C(5,2) P(wwwmm) 5/33.
9
Some problems involving conditioning
Here is the classical hit and run problem
A cab was involved in a hit and run accident at
night. Two cab companies, Green Cab and Blue Cab,
operate in the city. You are given the following
data(a) 85 of cabs in the city are Green and
15 are Blue.(b) A witness identified the cab as
Blue. (c) The court tested the reliability of
the witness and found that he correctly
identified one of the two colors 80 of the time
and failed 20.What is the probability that the
cab involved in the accident was Blue rather than
Green? Make your best guess.
10
Consider a simpler problem
Bill's sister Debby lives in Canada. Learning
that Debby was pregnant, Bill drew a sample
space diagram based on (his interpretation of )
their family history . He was eager to have a
niece because he already had two nephews
Using this map, he estimated a probability of the
event F "at least one of newborns will be a
girl" (). What was P(F)?
11
He found that P(F)7/12. Later on it turned out
that Debby was expecting twins. Based on this
new information, Bill redrew the sample diagram,
First, he noticed that the probabilities of
singletons became 0. This fact turned the area
marked by a blue rectangle into the new sample
space. The probabilities of the twins should now
be rescaled, thought Bill, so that their ratios
will stay the same while they should now sum up
to 1. This resulted in the following
distribution of probabilities
Using this diagram Bill found P(F)P(BG)P(GG)3/
4. Its increase by 1/6 relative to the initial
estimate made Bill even happier.
12
The probability of F under the condition (the
hypothesis) that B has occurred is called a
conditional probability P(FB).
In our example, Bill was considering a
probability P(FT) that at least one of the
newborns will be a girl under the condition that
twins will be born. His rescaling approach is
correct but too clumsy. Instead of rescaling the
probabilities, Bill couldve recognized that
T
BB
B
1/3
1/12
BG
1/6
1/3
G
1/12
GG
F
F?T
P(FT) P(F?T) / P(T) (1)
To understand this formula, note (a) that only
the part of F that lies in T can possibly occur,
and (b) since the sample space is now T we have
to divide by P(T) to make P(TT) 1
13
Appling this formula to the Bills problem, we
find P(FT)(1/61/12)/(1/61/121/12)
(1/4)/(1/3)3/4. This is exactly the result that
Bill derived using his rescaling trick.
The formula (1) can also be rewritten as
P(F?T) P(FT)P(T) P(TF)P(F)) (2)
In this representation it is called "the
multiplication rule for the conditioning.
14
Examples

• A fair coin is flipped twice. What is the
  conditional probability that both flips result in
  heads, given that the first flip results in
  heads? 
• Solution If E HH and F HH, HT, then 
•    P(EF) P(EF)/P(F) PHH/PHH, HT 
•                                     (1/4)/(2/4)
  1/2. 
• Notice that the probability of two heads ( 1 / 4
  ) is less than the probability of two heads given
  that the first is head.

15
2. Suppose we draw two cards out of deck of 52
without replacement. Let A The first card is a
spade, and B The second card is a spade. What
is the probability that both A and B
occur. P(A?B) P(BA)P(A)(12/51)( 1/4) Note
that we computed P(BA) by thinking about the
situation that exists after A has occurred rather
than using the definition (1). Its more common
to use P(A) and P(BA) to compute P(A?B) rather
than using P(A?B) and P(A) to compute P(BA).
You probably noticed that we already used the
conditional probability when we solved similar
problems before, without calling it by name.
"Good Heavens! For more than forty years I have
been speaking prose without knowing it." Jean
Baptiste Moliere, The Middle Class Gentleman
16
3 Two fair coins are tossed and you are told that
at least one coin came up heads.What is the
probability that both coins came up
heads? Warning the two instinctive answers, ½
and ¼ are both incorrect. Use the conditional
probability formula (1) Solution (1) introduce
two eventsB At least one coin came up heads
A Both coins came up heads(2) chose the
relevant formulaP(AB) P(AB)/P(B).(3) to
find P(AB) and P(B), notice that B HT,TH,HH
AB HH (intersection) (4) As a
result, P(B) ¾ P(AB) ¼ P(AB) 1/3.
17
4. And here is one of many examples showing that
the results of conditioning are often not too
obvious. A person picks 13 cards out of a deck
of 52. Let A2He receives at least two Aces.
A1He receives at least one Ace, and HHe
has the Ace of Hearts. Question Which
probability is larger, P(A2A1) or P(A2 H)? In
other words, which is more likely having two
aces given that he has at least one Ace, or
having two aces given that he has an Ace of
hearts. Make your best guess. Solve this extra
credit problem at home
18
This and many other problems demonstrates
importance of very careful analysis of
conditioning in many situations where many
inter-related events are involved. Various court
cases, criminal evidence and so called
reasonable doubts provide a lot of examples.
19
5. DNA Fingerprinting in the Courts by  Bruce
Weir,  North Carolina State University As a
part of your home assignment, please watch this
video. You can find it here . Use CtrF to
search for the authors name, Weir, or the title
to locate the video. It runs about an hour. You
can browse the site to find a lot of other
interesting lectures.
20
6. Its time now to discuss the hit and run
problem. A cab was involved in a hit and run
accident at night. Two cab companies, Green Cab
and Blue Cab, operate in the city. You are given
the following data(a) 85 of cabs in the city
are Green and 15 are Blue.(b) A witness
identified the cab as Blue. (c) The court tested
the reliability of the witness and found that he
correctly identified one of the two colors 80 of
the time and failed 20.What is the probability
that the cab involved in the accident was Blue
rather than Green? Lets firs express the
question in terms of a conditional
probability. What event plays a role of the
condition? Can it be (a) ? (b) ? ( c) ?
21
A"Witness says it's blue", B"The car was
Blue. What is P(BA)? Lets make now a diagram
of everything that we know. First, I drew these
two rectangular areas representing the shares of
the Green and Blue cabs. But then I was confused
trying to figure put how to depict the A event in
the same picture. Lets try it together. First,
notice that due to a possible witness error, the
A event contains not only the blue area (which
would be the case if the witness was absolutely
reliable) but also the green area. What are their
relative portions? A?Green 0.20.850.17. A ?
Blue 0.80.150.12.
P(BA)P(B?A) / P(A)0.12/(0.170.12)0.12/0.290
.41
22
A question for the self-test What would be the
probability that the car was green What if the
witness testified that it was green?
23
7. A laboratory blood test is 95 effective in
detecting a certain disease when it is, in fact,
present (sensitivity). However, the test also
yields a "false positive" result for 1 of the
healthy persons tested. (That is, if a healthy
person is tested, then, with probability .01 the
test result will be positive.) If .5 of the
population actually has the disease (prevalence),
what is the probability a person has the disease
given that the test result is positive?    Solu
tion. Let D the tested person has the disease
and Pos the test result is positive. The
desired probability P(DPos) is given
by  P(DPos)P(D ? Pos)/P(Pos)   P(PosD)
P(D)/P(PosD) P(D)P(PosDC)P(DC)    (.95)
(.005)/(.95) (.005) (.01) (.995) .323. 
24
8. The Monty Hall problem (MHP)
This problem is named for the host of the TV
show lets make a deal. Three curtains are
numbered 1,2 and 3. Behind one curtain is a car,
behind two others are donkeys. You pick a
curtain, say 1. Then host opens one of the two
remaining curtains, say 3, to reveal a donkey.
What is the probability that you will win given
that there is a donkey behind 3? Should you
switch curtains and pick 2 if you are given a
chance?
25
A few historical notes ( http//www.mathpuzzle.co
m/MontyHall.txt, The man who loved only numbers ,
etc). Btw, does anyone want to make a special
presentation, and to discuss the solution using
the Mathematica modeling? I will supply a
volunteer with required materials ?
An earlier version of MHP, the Three Prisoner
Problem, was analyzed in 1959 by Martin Gardner
in the journal Scientific American. He called it
"a wonderfully confusing little problem" and
presciently noted that "in no other branch of
mathematics is it so easy for experts to blunder
as in probability theory."
26
The fierce debates around the MHP erupted after
Marylin vos Savant , the columnist for the
Parade magazine, answered this brain teaser
advising her correspondent to switch the door.
Sticking with the first choice gives a 1/3
chance of winning, but switching doubles it to
2/3.
Marylin von Savant was pronounced by the
Guinness Book of World Records as the person
with Highest IQ ever recorded (228).? She
sported a wedding ring of pyrolytic carbon, a
special material that was invented by her
husband, Robert Jarvik, creator of the First
Artificial (Jarvik) Heart . ?
27
The experts responded in force to Ms. 27 Savant's
column. Of the critical letters she received,
close to 1,000 carried signatures with Ph.D.'s,
and many were on letterheads of mathematics and
science departments. "Our math department had a
good, self-righteous laugh at your expense,"
wrote Mary Jane Still, a professor at Palm Beach
Junior College. ". . . I am sure you will
receive many letters from high school and college
students. Perhaps you should keep a few addresses
for help with future columns." -W. Robert Smith,
Ph.D., Georgia State University  "You are
utterly incorrect. . . How many irate
mathematicians are needed to get you to change
your mind? -E. Ray Bobo, Ph.D., Georgetown
University  ". . . If all those Ph.D.s were
wrong, the country would be in very serious
trouble." -Everett Harman, Ph.D., U.S. Army
Research Institute 
28
Robert Sachs, a professor of mathematics at
George Mason University "You blew it! Let me
explain If one door is shown to be a loser, that
information changes the probability of either
remaining choice -- neither of which has any
reason to be more likely -- to 1/2. As a
professional mathematician, I'm very concerned
with the general public's lack of mathematical
skills. Please help by confessing your error and,
in the future, being more careful." ? Later, Dr.
Sachs, was one of the few with the grace to
concede his mistake. "I wrote her another
letter," Dr. Sachs said last week, "telling her
that after removing my foot from my mouth I'm now
eating humble pie. I vowed as penance to answer
all the people who wrote to castigate me. It's
been an intense professional embarrassment." ?
29
Many people, including some professional
mathematicians, argue that two unopened curtains
are the same, so they each will contain car with
probability ½ , and hence there is no point in
switching. This answer is incorrect. To compute
an answer, we should first make an assumption
about how the host behaves. Suppose that he
always chooses to show you a donkey, and picks at
random if there are two unchosen curtains with
donkeys. Assuming you pick curtain 1, there are
three possibilities
30
1 2 3 Hosts action
Case 1 donkey donkey car Opens 2
Case 2 donkey car donkey Opens 3
Case 3 car donkey donkey Opens 2 or 3
What is the probability you will win given that
there is donkey behind 3 (or the probability of
winning given that MH opens 3)
Now, P(open door 3) P(case 2, open door3)
P(case 3, open door 3)(note that , also means
and or ? ) P(case 2, open door3) 11/3
1/3 P(case 3, open door3) 1/31/21/6. It
gives P(open door 3) 1/31/61/2.
31
And it follows that P(case 3open door 3)
P(case 3,Open door 3)/P(open door 3) 1/6/1/2
1/3.
This answer is obvious. Initially we have 1/3
probability of picking a winning door. Opening
the door with a donkey does not change this
probability. It means that switching the door
increases the probability from 1/3 to 2/3.
32
Independent events
Definition Two events E and F are independent if
P(EF)P(E) and P(FE) P(F) (3)
The equivalent definition The events A and B are
independent if P(AB)P(A)P(B) (4)
Assignment prove that these definitions are
equivalent
33
Two examples of the independent events

• Flip two coins. A The first coin shows H,
  BThe second coin shows H. Prove that A and B
  are independent.
• Hint OmegaHH,HT,TH,TT
• 2. Roll two dice. A"The first die shows 5,
  B"The second die shows 2". P(A)1/6, P(B)
  1/6, P(AB) P(52)1/36.

Dependent events
Roll two dice. A"The sum of the two dice is 9".
B"The first die is 2". P(B)1/6. A
6,3,5,4,4,5,3,6. P(A)4/361/9.
P(AB)0.
34
Suppose an unfair die has the property that the
probability of number i turning up is i/21. Are
the events A an even number turns up and B a
number 4 or less turns up independent. What is
the corresponding result for a fair die? Solution
1 (1) describe the events A 2,4,6
B1,2,3,4. AB 2,4, (2) find the
probabilities of these events using the
probability function P(A) 2/214/21 6/21
12/21 P(B) 1/21 2/21 3/21
4/2110/21 P(AB) 2/21 4/21 6/21. (3)
find P(A)P(B) 120/212140/147 lt P(AB)6/21
42/147 Answer the events are not
independent Solution 2 (1) the events are the
same as above (2) find the probabilities of these
events using the probability function P(A)
1/61/6 1/6 1/2 P(B) 41/6 2/3. P(AB)
21/6 1/3. (3) find P(A)P(B) 1/3
P(AB)Answer the events are independent
35
Some more examples.
It is not always self-evident if the events are
independentTry to analyze the following example
(group work) Roll two dice. Let S7"The sum of
the two dice is 7", F4"The first die is 4",
DE"The difference of two dice is even".Are
S7and F4 independent? S7 and DE? F4 and DE?
36
Solution(a) Describe the events and their
probabilities S71,6,2,53,4,4,3,5,2,6,
1 P(S7)1/6 P(F4)1/6. DE
1,3,1,5,2,4,2,6,3,5,4,6,6,4,5,3,
6,2,4,2,5,1,3,1 P(DE)1/3 (b) Find the
intersections and their probabilities. S7F4
4,3, P(S7F4) 1/36 P(S7)P(F4) S7 and F4
are independent. S7DE?, P(S7DE)0 ?
(non-equal) P(S7)P(DE) S7 and DE are
not independent.They are disjoined. They can
never occur together. Occurrence of S7 makes
simultaneous occurrence of DE impossible. F4DE4
,6, P(F4DE)1/36 ? (non-equal) P(F4)P(DE) F4
and DE are not independent.
37
Definition A set of events A1, A2, An is said
to be mutually independent if for any subset Ai,
Aj, Am of these events we have
P(Ai ? Aj ? ? Am)P(Ai)P( Aj) P(Am)
(5)
It is natural to ask if all pairs of a set of
events are independent, is the whole set mutually
independent? The answer is not necessary.
Building various events with two dice
experiments. The sample space can be presented
through the table 66.Different groups of cells
correspond to different events. Two events are
shown in the picture Green "Sum of two throws
lt5" and Grey"Sum of two throws 9".
38
Are they independent?
Now, try to name the events shown in Figures a
through d and evaluate their (in)dependence.
Colored black are the intersection points between
different events. In the case(d), be careful
evaluating the mutual (in)dependence of three
events.
39
(No Transcript)
40
11 12 13 14 15 16
21 22 23 24 25 26
31 32 33 34 35 36
41 42 42 44 45 46
51 52 53 54 55 56
61 62 63 64 65 66
Here is another example of two dependent
eventsS4 Sum 4, X2 Second is 2 P(S4
X2) 1/36 P(S4)P(X2) 1/61/121/72.
41
Suppose we roll a green and a red die. Let A"The
red die shows 2 or 5. B"The some of the two
dice is at least 7". Are A and B independent?
P(A)P(GreyBlack)1/3
P(B)P(yellowBlack)21/36. P(AB)P(black)7/36.
The events are independent
42

• Home assignment.
• Watch the video (p.20).
• Read the lecture
• Prove the equivalence of two definitions of the
  independence (p. 31).
• Solve the self-test problems 1-5 below and
  compare your results with the solutions.
• The assignment with Mathematica will be posted
  soon.

43
HW5 problems

• Lets modify Problem 7 (page 23) to a more
  general test. Assume a test has a probability p
  of indicating the disease among patients actually
  having a disease. Assume also that the test
  indicates the presence of the diseases with
  probability 1-p among the patients not having
  the disease (false positive). Finally, suppose
  the incident rate of the disease is r.
• (a) Find the probability that the person tested
  positive actually has the disease.Hint Consider
  the events T tested positive and A Has
  a disease. You have to find P(AT).
• (b) If p 0.95 and r 0.005, what is the
  probability of having a disease being tested
  positive?
• ( c) Leaving p 0.95, find how the result
  depends on r (use Mathematica to make a plot).
  At what r the test reliably indicates the disease?

44
HW problems

• Problem 1 Lets modify Problem 7 (page 23) to a
  more general test. Assume a test has a
  probability p of indicating the disease among
  patients actually having a disease. Assume also
  that the test indicates the presence of the
  diseases with probability 1-p among the
  patients not having the disease (false
  positive). Finally, suppose the incident rate of
  the disease is r.
• (a) Find the probability that the person tested
  positive actually has the disease.Hint Consider
  the events T tested positive and A Has
  a disease. You have to find P(AT).
• If p 0.95 and r 0.005, what is the probability
  of having a disease being tested positive?
• Leaving p 0.95, find how the result depends on
  r (use Mathematica to make a plot). At what r
  the test reliably indicates the disease?

45
Problem 2Formula P(AUB) P(A) P(B) P(AB)
can be generalized for any number of events. Thus
for the union of three events the result
isP(AUBUCU) P(A) P(B) P( C) P(AB) P(AC)
P(BC) P (ABC)Suppose that A and B are
independent. B and C are mutually exclusive and ,
and A and C are independent. If P(AUBUCU)
0.9P(B) - 0.5 and P(C ) 0.3, find P(A).
46
Problem 3 A loaded coin with p(H) ¾ is
tossed twice. Let the events A, B and C be
respectively first toss is heads, second toss
is heads and tosses show the same face.(a)
Are A and B independent?(b) Are A and BUC
independent?( c) are A,B and C independent?
47
Problem 4 Your friend flips three coins and
tells you that there is at least two heads. What
is the probability that the first coins is heads?
48
Problem 5 Suppose that the probability that
married man votes is 0.45, the probability a
married woman votes is 0.4 and the probability a
woman votes given her husband votes is 0.6. What
is the probability (a) that both vote (b) a man
votes given that his wife does?