Chapter 3: Probability

1
Chapter 3 Probability

• One day there was a fire in the wastebasket in
  the Deans office. In rushed a physicist, a
  chemist, and a statistician. The physicist
  immediately started calculations to determine how
  much energy would have to be removed from the
  fire to stop combustion.
• The chemist tried to figure out what chemical
  reagent would have to be added to the fire to
  prevent oxidation. While they were doing this,
  the statistician set fires in all the other
  wastebaskets in the office.
• What are you doing? they demanded. Well, to
  solve the problem, you obviously need a larger
  sample size, the statistician replied.

2
Dice Simulation

• Do Excel Demo
• Points to be made
• Randomness means unpredictable results
• Probability means long run is predictable
• We imagine a mechanism, rule, or law that
  produces results with definite probabilities
• If we know the probability law, we can make
  meaningful predictions about the likelihood of
  various outcomes

3
Why we play silly games

• In the study of probability, we need simple
  examples to learn from. Some of these may seem
  silly or unrealistic, but they are actually
  models of real problems. If we understand the
  examples, we can tackle real problems by
  formulating them in terms of simple examples like
  dice games, coin tosses, spinners, etc.
  Recognizing a familiar set-up is often the key to
  success.

4
Counting 3s

• Another dice game Roll two dice and record the
  number of 3s.
• The possible outcomes are 0, 1, or 2.
• We will count the frequency of each outcome as we
  repeat the process.
• (Excel Simulation)

5
Properties of this Experiment

• If we continue this experiment indefinitely
• The frequencies will have approximately a 25101
  ratio (we need to find out why)
• The relative frequencies will settle down.
• A computer simulation of experimental outcomes is
  a helpful tool that may lead to important
  insights regarding a probability problem. But,
  it is not a substitute for the theoretical
  development that we will begin now.

6
Definitions

• Probability Experiment A process that yields an
  observation that cant be predicted with
  certainty
• Trial One instance of an experiment in which
  the same process is repeated
• Outcome One possible result of an experiment.
• The language of set theory is used
• The set of all possible outcomes is the Sample
  Space, often denoted by S.
• Events are subsets of the Sample Space they
  contain one or more outcomes, and are often
  denoted by A, B, or E.

7
Some Examples

• An Experiment Select two students at random and
  ask them if they have cars on campus. Record Y
  if the answer is yes and N if the answer is
  no.
• There are two trials, because each student yields
  one observation. Each trial has an outcome of Y
  or N
• But the outcomes of the experiment are ordered
  pairs
• The Sample Space S(N,N), (N,Y), (Y,N), (Y,Y)
• One example event both students have
  cars.A(Y,Y)
• Another event only one student has a
  car.B(Y,N),(N,Y)
• Yet another event at least one has a
  car.C(Y,N),(N,Y),(Y,Y)

8
More Examples

• Toss one coin, then toss one die.
• SH1 H2 H3 H4 H5 H6 T1 T2 T3 T4 T5 T6
• (The notation has been simplified.)
• AThe coin was a head
• BThe die toss was an even number
• Randomly select three voters and ask if they
  favor an increase in property taxes for road
  construction in the county.
• S NNN, NNY, NYN, NYY, YNN, YNY, YYN, YYY
• CAt least one voter said yes
• Exercise List the elements of A, B, and C.

9
More Terms

• Outcomes are also called sample points.
• n(S) the number of outcomes in the sample space.
• Events containing only one outcome are called
  Simple Events.
• Events containing two or more outcomes are called
  Compound Events.

10
Notes

• The outcomes in a sample space can never overlap
  (they are mutually exclusive).
• The sample space must contain all possible
  outcomes (relate to exhaustive, below).
• Two events may or may not be mutually exclusive.
• If two or more events together include all
  outcomes, they are called exhaustive.
• In some cases a collection of events may be both
  mutually exclusive and exhaustive.

11
When Events Occur

• Remember, events can contain multiple outcomes.
• An event occurs if it contains the actual outcome
  of the experiment.
• More than one event can occur for a single trial
  (if not mutually exclusive).
• Example On the way to work, some employees at a
  certain company stop for a bagel and/or a cup of
  coffee. Possible outcomes for (bagel,coffee)
  are
• (n,n) Dont stop
• (b,n) Get only a bagel
• (n,c) Get only coffee
• (b,c) Get bagel and coffee

12
Example Not Mutually Exclusive/Exhaustive

• Define event B as gets bagel
• Define event C as gets coffee
• Then B(b,n),(b,c) and C(n,c),(b,c)
• BnC(b,c) so B and C are not mutually
  exclusive.
• If the outcome is (b,c), then both B and C have
  occurred.
• It is also true that B and C are not exhaustive,
  since BUC?S.

13

• The accompanying Venn diagram illustrates the
  choices of the employees for a randomly selected
  work day.

Coffee
Coffee
Bagel
14
Determining Probability

• Probability of an Event The expected relative
  frequency of the event
• Three ways to determine the probability of an
  event
• Empirically
• Theoretically
• Subjectively

15
Empirical Probability

• Based on counts of data. It is the observed
  relative frequency.
• Use prime notation
• n(A) number of times the event A has occurred
• n number of trials or observations, or sample
  size.
• The Law of Large Numbers says the larger the
  number of experimental trials n, the closer the
  empirical probability P(A) is expected to be to
  the true probability P(A).
• In symbolsAs

16
Theoretical and Subjective

• Theoretical Probability, P(A), is the expected
  relative frequency (long run)
• P(A) is based on knowledge (or assumptions) of
  the fundamental properties of the experiment.
• Subjective Probability is based on someones
  opinion and/or experience. It is usually just a
  guess and subject to bias.

17
Theoretical Probability

• Toss a fair coin. Let event H be the occurrence
  of a head. What is P(H)?
• In a single toss of the coin, there are two
  possible outcomes.
• Since the coin is fair, each outcome is equally
  likely.
• Therefore it follows that P(H) 1/2.
• This doesnt mean one head occurs in every two
  tosses.
• After many trials, the proportion of heads is
  expected to be close to half, not based on data,
  but by reasoning from the fundamental properties
  of the experiment.

18
Equally Likely Outcomes

• The previous example of a coin toss is an example
  of an experiment in which all outcomes are
  equally likely.
• Many common problems (coins, dice, cards, SRS)
  have this property.
• If this property holds, the probability of an
  event A is the ratio of the number of outcomes in
  A to the number of outcomes in S.

19
Examples

• A die toss has six equally likely outcomes.
• S1,2,3,4,5,6, thus n(S)6.
• Define event E as E2,4,6. Then n(E)3.
• P(E)3/61/2.
• Toss two coins there are 4 outcomes.
• STT,TH,HT,HH.
• Define event E as Eat least one head.
• ETH,HT,HH
• P(E)3/4

20

• Example A fair coin is tossed 5 times, and a
  head (H) or a tail (T) is recorded each time.
  What is the probability of
• A exactly one head in 5 tosses, and
• B exactly 5 heads?
• The outcomes consist of a sequence of 5 Hs and
  Ts
• A typical outcome HHTTH
• There are 32 possible outcomes, all equally
  likely.
• A HTTTT, THTTT, TTHTT, TTTHT, TTTTH
• B HHHHH

21
Hmmm

• How many statisticians does it take to screw in a
  light bulb?
• We dont know yetthe entire sample was skewed to
  the left.
• Hope that didnt go right by you

22
Revisit a Previous Example

• An Experiment Select two students at random and
  ask them if they have cars on campus. Record Y
  if the answer is yes and N if the answer is
  no.
• We can use a tree diagram to enumerate the
  elements of the sample space.

23

• Tree Diagram of Sample Space
• Student 1 Student 2 Outcomes
• Y Y, Y
• Y
• N Y, N
• Y N, Y
• N
• N N, N
• -Tree diagrams start from a common point, or
  root
• -This tree has four branches (from root to ends)
• -There are 2 first- and 4 second-generation
  branches.
• -The path along each branch shows a possible
  outcome.

24

• Example An experiment consists of selecting
  electronic parts from an assembly line and
  testing each to see if it passes inspection (P)
  or fails (F). The experiment terminates as soon
  as one acceptable part is found or after three
  parts are tested. Construct the sample space.
• Outcome
• F FFF
• F
• F P FFP
• P FP
• P P
• S FFF, FFP, FP, P

25
Laws or Axioms of Probability

• The probability of any event A is between 0 and
  1.
• The sum of the probabilities of all outcomes is
  1.
• A probability of 0 means the event cannot occur.
• A probability of 1 means the event is certain, it
  must occur every time.

26
Introducing Odds

• Example On the way to work Bobs personal
  judgment is that he is four times more likely to
  get caught in a traffic jam (J) than have an easy
  commute (E). What values should be assigned to
  P(J) and P(E)?

27
Definition of Odds

• The complement of A is denoted by .
• contains all outcomes in S not in A.
• Two events are complementary if they are mutually
  exclusive and exhaustive.
• Odds are a way of expressing probabilities for
  complementary events as a ratio of expected
  frequencies.
• If the odds in favor of an event A are a to b
  then the odds against A are b to a.
• Then the probability that A occurs is
• The probability A does not occur is

28

• Example
• 1. The complement of the event success is
  failure.
• 2. The complement of the event rain is no
  rain.
• 3. The complement of the event at least 3
  patients recover out of 5 patients is 2 or
  fewer recover.
• Notes
• 1.
• 2.
• 3. Every event A has a complementary event
• 4. Useful in calculations such as when the
  question asks for the probability of at least
  one.
• 5. The complement of S is Ø, the empty set.
• 6. Obviously, P(Ø)1-P(S)1-10.

29
Addition Rules

• If A and B occur, the outcome is in both, i.e.,
  AnB has occurred. So P(A and B)P(AnB).
• If A and B are mutually exclusive, AnBØ so
  P(A and B)P(Ø)0.
• If A or B occurs, the outcome is in at least one
  of them, i.e., AUB has occurred. So P(A or
  B)P(AUB) P(A)P(B)P(AnB).
• Note If A and B are NOT mutually exclusive,
  just adding P(A)P(B) would count the outcomes in
  the intersection twice, so we have to correct for
  this double-count.
• But, if A and B ARE mutually exclusive, P(AnB)0
  so P(A or B)P(AUB) P(A)P(B).

30
Example
This diagram shows the probability that a
randomly selected consumer has tried a snack food
(F) is .5, tried a new soft drink (D) is .6, and
tried both the snack food and the soft drink is
.2.
.3
.4
.2
D
F
.1
S
31
Examples

• Suppose A and B are mutually exclusive, and
  P(A).12 and P(B).34. Find P(AUB).
• Suppose P(A).6, P(AUB).9, and P(B).5. Find
  P(AnB).
• Suppose A, B, and C are mutually exclusive and
  exhaustive. If P(A).2, P(B).4, find P(C).

32
Conditional Probability

• Sometimes two events are related in such a way
  that the probability of one depends upon whether
  the other occurs.
• Partial information about the outcome may alter
  our assessment of the probabilities.
• The symbol P(A B) represents the probability
  that A will occur given B is known (assured).
  This is called conditional probability.
• Suppose I toss a die and show you that there is a
  3 on the front face. What can you say about the
  probabilities for the top face?
• What is P(1 on top3 on front)?
• What is P(4 on top3 on front)?

33
Attention!!

• It is crucial to realize we are not talking about
  two sequential events. This is for one outcome
  of one experiment, for which we have partial
  information, allowing us to remove some of the
  uncertainty.
• When I show you the three on the front face, the
  toss has already occurred, but you dont know the
  result. The chance involved is in your ability
  to guess the correct value, rather than in a
  particular value coming up.

34
Die Example

• Normally, There are six possibilities with P1/6
  for each.
• With the three showing on front, we eliminate two
  outcomes, restricting the sample space.
• The four remaining numbers are equally likely,
  with P1/4.

1 2 3 6 5 4
1 2 3 6 5 4
1 2 6 5
35
Calculating Conditional Probability

• Recall our definition of probability in terms of
  frequencies of equally likely outcomes
• Given B has occurred, the numerator becomes the
  number of outcomes of A that are still in the
  sample space. Any outcomes in A that were not in
  B are eliminated now. The denominator is the
  number of outcomes in B, the new sample space.
• To relate this back to the original
  probabilities, divide the numerator and
  denominator by n(S).
• Though this formula was derived using the idea of
  equal probabilities for all outcomes, the final
  form works in general.

36
Independent Events

• Two events, defined for one trial of an
  experiment, are independent iff P(A B) P(A)
  or P(B A) P(B).
• This should be understood to mean that if A and B
  are independent, the occurrence of B does not
  affect the probability of A, and visa versa.
• If A and B are independent, then so are

37
Example of Independent Events

• Consider the experiment in which a single fair
  die is rolled S 1, 2, 3, 4, 5, 6 . Define
  the following events
• A 1, 2
• B an odd number occurs

38
Example of non-Independent Events

• Consider the experiment in which a single fair
  die is rolled S 1, 2, 3, 4, 5, 6 . Define
  the following events
• A 1
• B an odd number occurs

39
General Multiplication Rule

• A little algebra gives this variation
• Which might be more usefully thought of as
• Note How to recognize phrasing that indicates
  intersections
• Both A and B
• A but not B
• Neither A nor B Not A and Not B Not (A or
  B)
• Not (A and B)Not A or Not B

40
Special Multiplication Rule

• If A and B are independent events in S, then
• , so
  .
• Example Suppose the event A is Allen gets a
  cold this winter, B is Bob gets a cold this
  winter, and C is Chris gets a cold this
  winter. P(A) .15, P(B) .25, P(C) .3, and
  all three events are independent. Find the
  probability that
• 1. All three get colds this winter.
• 2. Allen and Bob get a cold but Chris does not.
• 3. None of the three gets a cold this winter.

41

• Solution

42
Summary Notes

• Independent and mutually exclusive are two very
  different concepts.
• Mutually exclusive says the two events cannot
  occur together, that is, they have no
  intersection.
• Independence says each event does not affect the
  other events probability.
• P(A and B) P(A) P(B) when A and B are
  independent.
• Since P(A) and P(B) are not zero, P(A and B) is
  nonzero.
• Thus, independent events have an intersection.
• Events cannot be both mutually exclusive and
  independent.
• If two events are independent, then they are not
  mutually exclusive.
• If two events are mutually exclusive, then they
  are not independent.

43
Tree Diagrams

• Tree Diagrams can be used to calculate
  probabilities that involve the multiplication and
  addition rules.
• A set of branches that initiate from a single
  point has a total probability 1.
• Each outcome for the experiment is represented by
  a branch that begins at the common starting point
  and ends at the terminal points at the right.

44

• Example A certain company uses three overnight
  delivery services A, B, and C. The probability
  of selecting service A is 1/2, of selecting B is
  3/10, and of selecting C is 1/5. Suppose the
  event T is on time delivery. P(TA) 9/10,
  P(TB) 7/10, and P(TC) 4/5. A service is
  randomly selected to deliver a package overnight.
  Construct a tree diagram representing this
  experiment.

45

• The resulting tree diagram
• Service Delivery
• T
• A
• T
• B
• T
• C

46

• Using the tree diagram
• 1. The probability of selecting service A and
  having the package delivered on time.
• 2. The probability of having the package
  delivered on time.

47
Outcomes with Unequal Probabilities

• A scenario in which outcomes have different
  probabilities may be illustrated by the urn
  problems.
• Suppose we have an urn (an opaque container from
  which we may randomly select items) containing
  marbles of different colors, such as
• Two red
• Three blue
• Five white
• Represent the outcome of one draw as R, B, or W.
  Clearly,
• P(R).2
• P(B).3
• P(W).5

48
Clarification

• There are only three outcomes, R, B, and W. This
  is because the information obtained from a draw
  is the color, not the particular marble.
• However, we realize that there are several
  marbles associated with each outcome.
• If we choose to use the notation n(A) in this
  case, we will have to define it as the number of
  marbles associated with the event A, and n(S)
  would be the total number of marbles. Doing this
  will enable us to correctly use the definition of
  probability P(A)n(A)/n(S)

49
Two Draws with Replacement

• Suppose we draw a marble, return it to the urn,
  and draw again. Since the first marble is
  replaced, the first draw has no effect on the
  probability of the second draw thus the draws
  are independent. P(R,R)(.2)(.2).04 P(W
  ,R)(.5)(.2).10 P(1 red)(.2)(.3)(.2)(.5)
  (.3)(.2)(.5)(.2)
  .32 P(at least one red) .32(.2)(.2)
  .36 P(no red)1.36.64 P(1 red and 1
  blue) (.2)(.3)(.3)(.2) .12

50
Two Draws, without Replacement

• Suppose we draw two marbles, sequentially. When
  the first marble is taken out, the proportions of
  the remaining marbles change thus the draws are
  not independent. P(R,R)1/45.022 P(W,R
  )1/9.111 P(1 red)1/151/91/15
  1/916/45.356 P(at least one
  red) 16/451/4517/45 .378
  P(no red)117/4528/45 .622
  P(1 red and 1 blue) 1/151/152/15
  .133

51
Hmmm

• Why did the statistician cross the interstate?
• To get data from the other side of the median.