45733: lecture 2 chapter 3

1
45-733 lecture 2 (chapter 3)

• Probability

2
What do we do with statistics?

• Describe a sample well
• Analyze the sample to estimate properties of the
  population
• Analyze the analysis to describe how sure we are
  of it

3
Why do we need probability?

• Utility outside statistics
• Gambling, physics, chemistry, asset pricing,
  insurance, etc

4
Why do we need probability?

• Utility within statistics
• When we are describing how sure we are that our
  analysis of the population is right, probability
  gives us a precise language in which to speak.
• We will want to say things like
• I am more than 95 sure that US household income
  is greater than 30,000
• I am 99 certain that the mean time to failure of
  our light bulbs is between 100 and 120 hours
• I am 80 sure that GDP growth will be between
  1.2 and 3.5 next year

5
Experiment, sample space, event

• Consider an experiment
• A process which can have one of several possible
  outcomes
• Which outcome will occur is unknown to the
  experimenter or observer
• Examples
• Coin toss, die throw
• Light bulb tested to failure
• Economy evolves for one year

6
Experiment, sample space, event

• The sample space
• A list of all the possible outcomes of an
  experiment
• Examples
• Coin toss sample space heads,tails
• Die throw sample space1,2,3,4,5,6
• Light bulb failure time Sall positive real
  numbers
• Economy growth Sall real numbers gt -100

7
Experiment, sample space, event

• A basic event
• One point in the sample space
• Examples
• Coin toss heads
• Die throw 3
• Light bulb 400 hours
• Economy growth 1.7

8
Experiment, sample space, event

• An event
• A collection of one or more basic events
• A collection of one or more points in sample
  space
• Examples
• Coin toss heads tails heads,tails
• Die throw 3 3,6 1,2,3,4,5,6
• Light bulb 400 hours between 5 and 18 hours
• Economy growth 1.7 2.1,between 1 and 1

9
Experiment, sample space, event

• Notation
• We often write the sample space as S
• We often denote basic events as s
• We often write events as A, B, C, etc

10
Experiment, sample space, event

• Venn diagram
• A Venn diagram is a way of representing sample
  space, events, and operations
• Elements of Venn diagram
• Large rectangle representing the sample space
• Circles or other shapes representing events
• (optional) points representing basic events

11
Experiment, sample space, event

• Venn diagram
• Example the sample space of the die throw

1
4
6
3
5
2
12
Experiment, sample space, event
B
A

• Venn diagram
• Example
• A4,5,3
• B3,2,6

1
4
6
3
5
2
13
Experiment, sample space, event

• Notice
• All basic events are events
• The sample space is an event
• There is a special event called the null set or
  the null event or the empty event. It is ?

14
Experiment, sample space, event

• Membership
• A basic event may either belong to an event or
  not
• We will write s?A when the basic event s is in
  the event A
• We will write s?A when the basic event s is not
  in the event A

15
Experiment, sample space, event

• Membership
• Examples
• heads? heads,tails
• 1 ? 1,4,5
• 1 ? 3,6
• 1 ? between 0 and 3
• 140 hours ? between 50 and 100 hours

16
Experiment, sample space, event

• Membership Venn diagram
• Example
• A4,5,3
• 3 ? A
• 1 ? A

1
4
6
3
5
2
A
17
Experiment, sample space, event

• Sub-event (subset)
• We say that an event B is a sub-event of A if
  every member of B is also in A, and we write B?A
• Examples
• heads ? heads,tails
• 3,4,5 ? 1,2,3,4,5
• between 1 and 1.3 ? between 0.5 and 4
• 3,4,5 ? 1,2,3,4

18
Experiment, sample space, event
B

• Sub-event Venn diagram
• Example
• A4,5,3
• B4,5
• B ? A

1
4
6
3
5
2
A
19
Experiment, sample space, event

• Intersection
• A way of making a new event from two events
• The intersection of events A and B is the event
  consisting of all the basic events A and B have
  in common.
• CA?B means C is the intersection of A and B

20
Experiment, sample space, event

• Intersection
• Examples
• 1 1,2,3 ? 1,4
• between 1 and 1.5 btw 1 and 2 ? btw
  0.8 and 1.5
• ?1 ? 3,4,5

21
Experiment, sample space, event

• Intersection Venn diagram
• Example Intersection
• A4,5,3
• B3,2,6
• CA ? B3

C
1
4
6
5
3
2
22
Experiment, sample space, event

• Union
• A way of making a new event from two events
• The union of two events is the event which
  contains all the basic events which are in
  either.
• CA?B says C is the union of A and B --- C
  contains all the basic events in either A or B

23
Experiment, sample space, event

• Union
• Examples
• 1,2,3 1,2 ? 2,3
• btw 1 and 3 btw 1 and 1.5 ? btw 1.1
  and 3
• heads heads ? ?

24
Experiment, sample space, event

• Union Venn diagram
• Example Union
• A4,5,3
• B3,2,6
• DA ? B 2,3,4,5,6

D
1
4
6
3
5
2
25
Experiment, sample space, event

• Mutual Exclusivity
• A and B share no basic events in common
• A ?B?
• Example A1,4 B3,2

1
4
6
B
3
5
2
A
26
Experiment, sample space, event

• Collective exhaustivity
• A bunch of events are collectively exhaustive if
  their union is the sample space
• Example E11,4 E23,2,6 E33,4,5
• E1 ? E2 ? E31,2,3,4,5,6

27
Experiment, sample space, event

• Collective exhaustivity
• A bunch of events are collectively exhaustive if
  their union is the sample space

1
4
6
3
5
2
28
Experiment, sample space, event

• Partitioning
• A bunch of events partition the sample space if
  they are mutually exclusive and collectively
  exhaustive

1
4
6
3
5
2
29
Experiment, sample space, event

• Complement
• A complement is all the basic events in the
  sample space which are not in A
• Complements are partitioning

6
1
4
3
5
2
30
Experiment, sample space, event

• Some useful rules

6
1
4
3
5
2
31
Experiment, sample space, event

• Some useful rules

32
Experiment, sample space, event

• Some useful rules
• If E1,E2,E3,,Ek are partition the sample space,
  then
• E1 ?A, E2 ?A, E3 ?A,,Ek ?A, are mutually
  exclusive
• (E1 ?A) ?(E2 ?A) ?(E3 ?A) ? ?(Ek ?A)A

33
Experiment, sample space, event

• Some useful rules

34
What is probability?

• Probability is a language within which to
  describe uncertainty
• Uncertainty about which event will occur
• Some events are more likely than others
• When one event occurs, that may make other events
  more/less likely to occur
• Since it is a language it has rules
• Since it is a mathematical language, the rules
  are precise
• Since the rules are precise, the statements it
  can make are correspondingly precise

35
What is probability?

• Probability is a number between 0 and 1
• When we say the probability of an event is 0,
  that means it is impossible for the event to
  occur
• When we say the probability of an event is 1,
  that means it is certain that the event will
  occur
• If the probability of A occurring is greater than
  the probability of B occurring, that means that A
  is more likely than B

36
What is probability?

• There are differing interpretations of what this
  number between 0 and 1 means (in terms of the
  external world)
• Frequentist
• Imagine doing an experiment many independent
  times
• Each time, we record whether or not the event A
  occurred
• As N (number of experiments) goes to infinity
• P(A) NA/N

37
What is probability?

• There are differing interpretations of what this
  number between 0 and 1 means (in terms of the
  external world)
• Subjectivist
• The probability of an event A occurring exists
  only in our minds, reflecting our
  uncertainty/ignorance
• When I say P(A)0.5 that means I think a 11 bet
  on whether A occurs is a fair bet
• When I say P(A)0.33 that means I think a 21 bet
  on whether A occurs is a fair bet

38
What is probability?

• Venn diagram
• It is often useful to think of probability as
  area in a Venn diagram

A
B
39
Postulates of probability

• For any event A, 0?P(A) ?1
• For any event A, P(A)?s?AP(s)
• The probability of an event is just the sum of
  the probabilities of the basic events which make
  it up
• P(1,2,3)P(1)P(2)P(3)
• P(S)1 and P(?)0

40
Consequences

• If S has n equally likely basic events, each one
  has probability 1/n
• If S has n equally likely basic events and nA of
  them are in A, then A has probability nA/n

41
Consequences

• If A and B are mutually exclusive events, then
  P(A ?B)P(A)P(B)

1
4
6
3
A
B
5
2
42
Consequences

• In general P(A ?B)?P(A)P(B)

1
4
6
3
A
B
5
2
43
Consequences

• If B ? A, then P(B) ?P(A)

B
A
44
Rules of probability

45
Rules of probability

A
B
46
Conditional probability

• Conditional probability is used to deal with
  partial information
• Suppose there are two events, A and B and we wish
  to know the probability of A occurring
• Estimate this probability somehow
• Now we learn that B has occurred
• How should we change our assessment of the
  probability of A occurring given that we know for
  sure B has occurred?

47
Conditional probability

• Example, die throw
• A1,2,3,5
• B3,4
• P(A)1/61/61/61/62/3
• The probability of A given that we know B has
  occurred is ½
• Two equally likely outcomes in B
• Only one of them is also in A

48
Conditional probability

• Notation
• We write P(AB)
• This is said The probability of A given Bor
  The probability of A conditional on B
• So, we could write P(AB)1/2 in our previous
  example

49
Conditional probability

• Rule for calculating

A
B
50
Conditional probability

• Rule for calculating
• In die throw, A1,2,3,5 B3,4

51
Conditional probability

• Multiplication rule

52
Conditional probability

• Independence
• Two events A,B are independent if

53
Conditional probability

• Independence, consequences
• If two events A,B are independent

54
Conditional probability

• Bayes rule

55
Conditional probability

• Bayes rule, example
• Bob is a stock analyst
• A Stock price increases
• B Bob recommends the stock
• We know
• 80 of stock prices rise
• When a stock price rises, bob picked it 15 of
  the time
• When a stock price falls, bob picked it 10 of
  the time
• We want to know, what is the probability that the
  stock price rises, given that Bob picked it?

56
Conditional probability

• Bayes rule, example
• We want
• We know

57
Conditional probability

• Bayes rule, example

58
Conditional probability

• Bayes rule, example