Algorithms CSCI 235, Fall 2005 Lecture 9 Probability

1
Algorithms CSCI 235, Fall 2005Lecture
9Probability
2
Permutations
How many ways can you take k items from a set of
n items? Order matters (ab is counted separately
from ba) No duplicates allowed (don't count aa)
Example S a, b, c, d, how many ways can we
take 2 items?
Total pairs 12 4 ways to pick first 3 ways to
pick second
ab ac ad
ba bc bd
ca cb cd
da db dc
In general The number of permutations of k
items from a set of n is
3
Combinations
Combinations are like permutations, except order
doesn't matter. ab and ba are considered the
same.
Combination of 4 things taken 2 at a time
ab ac ad
bc bd
cd
Each set of k items has k! possible
permutations. number of combinations number of
permutations divided by k!
4
k-tuples and k-selections
k-tuple (or k-string) Like a permutation, but
can have duplicates.
aa ab ac ad
ba bb bc bd
n choices for 1st item n choices for 2nd item,
etc.
ca cb cc cd
da db dc dd
Total possible k-tuples for k items taken from a
group of n items is nk
k-selection Like combination, but can have
duplicates.
aa ab ac ad
bb bc bd
cc cd
dd
Formula
5
Sample Spaces
A sample space is the set of all possible
outcomes for an experiment.
Experiment A flip 3 coins
Sample space HHH, HHT, HTH, HTT, THH, THT,
TTH, TTT
of possibilities ?
23
Experiment B Flip a coin until you get heads.
Sample space H, TH, TTH, TTTH, . . .
6
Diagramming sample spaces
Experiment A (flip 3 coins)
7
Diagramming sample spaces
Experiment A (flip 3 coins)
H3 H1H2H3 T3 H1H2T3 H3 H1T2H3 T3 H1T2T3 H3 T1H2H3
T3 T1H2T3 H3 T1T2H3 T3 T1T2T3
H2
Sample space (the set of all possible outcomes)
H1
T2
H2
T1
T2
Experiment B (flip until heads)
8
Diagramming sample spaces
Experiment A (flip 3 coins)
H3 H1H2H3 T3 H1H2T3 H3 H1T2H3 T3 H1T2T3 H3 T1H2H3
T3 T1H2T3 H3 T1T2H3 T3 T1T2T3
H2
Sample space (the set of all possible outcomes)
H1
T2
H2
T1
T2
Experiment B (flip until heads)
...
H1 T1H2 T1T2H3 T1T2T3H4 H1 H2 H3 H4 T1 T2 T3 T4
. . .
9
Events
An event is a subset of the sample
space Experiment A (flip 3 coins) Event A1
First flip is head HHH, HHT, HTH, HTT Event
A2 Second flip is tail HTH, HTT, TTH,
TTT Event A3 Exactly 2 tails HTT, THT,
TTH Event A4 Two consecutive flips are the
same HHH, HHT, HTT, THH, TTH,
TTT Experiment B (flip until get heads) Event
B1 First flip is head H Event B2 First
flip is tail TH, TTH, TTTH ... Event B3
Even number of flips TH, TTTH, TTTTTH
10
Definitions
If A and B are events in sample space S, then
"A and B" is translated
"A or B" is translated
"not A" is translated
Two events are mutually exclusive if
11
Examples
1. Of the four events in Experiment A, which
pairs are mutually exclusive?
2. Of the three events in Experiment B, which
pairs are mutually exclusive?
12
Recall events
An event is a subset of the sample
space Experiment A (flip 3 coins) Event A1
First flip is head HHH, HHT, HTH, HTT Event
A2 Second flip is tail HTH, HTT, TTH,
TTT Event A3 Exactly 2 tails HTT, THT,
TTH Event A4 Two consecutive flips are the
same HHH, HHT, HTT, THH, TTH,
TTT Experiment B (flip until get heads) Event
B1 First flip is head H Event B2 First
flip is tail TH, TTH, TTTH ... Event B3
Even number of flips TH, TTTH, TTTTTH
13
Probability Distribution
A probability distribution Pr on sample space
S is any mapping from events of S to 0...1 such
that the following axioms hold
1) PrA gt 0 for any event A
2) PrS 1
Example Flip two coins S HH, HT, TH,
TT Event A HT Event B TH What is
14
Some useful theorems
Example Flip two coins. Event A HH, HT,
TH Event B HT, TH, TT What is
15
Working with probability trees
If events at distinct stages of a probability
tree are independent, then the probability of a
leaf is the product of the probabilities on the
path to the leaf.
Experiment A (flip 3 weighted coins) H11/3, T1
2/3 H21/4, T23/4 H31/5, T3 4/5
16
Working with probability trees
If events at distinct stages of a probability
tree are independent, then the probability of a
leaf is the product of the probabilities on the
path to the leaf.
Experiment A (flip 3 weighted coins) H11/3, T1
2/3 H21/4, T23/4 H31/5, T3 4/5
H3 H1H2H3 T3 H1H2T3 H3 H1T2H3 T3 H1T2T3 H3 T1H2H3
T3 T1H2T3 H3 T1T2H3 T3 T1T2T3
(1/3)(1/4)(1/5) 1/60
H2
(1/3)(1/4)(4/5) 4/60 1/15
H1
(1/3)(3/4)(1/5) 3/60 1/20
T2
(1/3)(3/4)(4/5) 12/60 1/5
(2/3)(1/4)(1/5) 2/60 1/30
H2
(2/3)(1/4)(4/5) 8/60 2/15
T1
(2/3)(3/4)(1/5) 6/60 1/10
T2
(2/3)(3/4)(4/5) 24/60 2/5
17
Another example
Experiment B (flip a weighted coin until heads)
Hi 1/3, Ti 2/3
(2/3)(2/3)(2/3)(1/3) 8/81
(2/3)(2/3)(1/3) 4/27
(2/3)(1/3) 2/9
1/3
H1 T1H2 T1T2H3 T1T2T3H4 H1 H2 H3 H4 T1 T2 T3 T4
. . .
Probability of S
1/3 2/9 4/27 8/81 ...