Title: Chapter 1: Discrete and Continuous Probability Distributions
1Chapter 1 Discrete and Continuous Probability
Distributions
- 1.1 Probability Theory
- 1.2 Discrete Probability Distribution
- 1.3 Continuous Probability Distribution
-
21.1 Probability Theory
- 1.1.1 Basic Probability Theory
- 1.1.2 Set Theory Operations
- 1.1.3 Counting Techniques
- 1.1.4 Addition, Multiplication
- 1.1.5 Conditional Probability
- 1.1.6 Independence
- 1.1.7 Total Probability Rule Bayes Theorem
3Basics of Probability Theory
- Probability refers to the study of randomness and
uncertainty. - In other words, probability is a numerical
measure of chance for the occurrence of an event.
4- Example 1
-
- An experiment consists of tossing three coins.
Find the sample space if - We are interested in the observed face of each
coin, - We are interested in the total number of heads
obtained. - Solution
- S HHH, HHT, HTH, THH, HTT, THT, TTH, TTT
- S 0, 1, 2, 3
5Frequentist Interpretation
- Intuitively, the probability of an event is the
proportion of times the event should occur when
the experiment is run a large number of times. - Suppose S is a sample space in which all
outcomes are assumed to be equally likely, and E
is an event. Then the probability of E,
denoted by P(E), is -
- Probability is quantified as a percentage or a
proportion of the total possible outcomes.
6Subjective Interpretation
- There is yet another definition of probability,
called as the subjective probability, which is
the foundation of Bayesian Econometrics. - Under the subjective or degrees of belief"
definition of probability, you can ask question
such as - What is the probability that there will be a
stock market boom in 2005? - What is the probability that Brazil will win the
World Cup?
7Set Operations on Events
8(No Transcript)
9- Example 2
- For the experiment in which the number of pumps
in use at a single six-pump gas station is
observed. Let A 0, 1, 2, 3, 4, B 3, 4, 5,
6, and C 1, 3, 5. Then find - (a) A?B (b) A?B (c) A?C (d) (A?C)
-
- Solutions
- (a) A?B 0, 1, 2, 3, 4 ? 3, 4, 5, 6 3, 4
- (b) A?B 0,1,2,3,4 ? 3, 4, 5, 6 0, 1, 2,
3, 4, 5, 6 S - (c) A?C 0, 1, 2, 3, 4? 1, 3, 5 0, 1, 2,
3, 4, 5 - (d) (A?C) S (A?C) 6
10Counting Techniques
- Product Rule
- If an operation can be described as a sequence of
k steps, and - if the number of ways of completing step 1 is
n1, and - if the number of completing step two is n2 for
each way of completing step 1, and so forth, - the total number of ways of completing the
operation is n1 x n2 x n3 x .. X nk
11- Permutation
- Any ordered sequence of k objects taken from a
set of n distinct objects is called a permutation
of size k of the objects. - The number of permutation of size k that can be
constructed from n objects is denoted by - Combination
- Given a set of n distinct objects, any unordered
subset of size k of the objects is called a
combination. - The number of combination of size k that can be
constructed from n objects is denoted by
12- Example 3
- A restaurant has four kinds of soups, eight
kinds of main course, five kinds of dessert, and
six kinds of drinks. If a customer selects one
item from each category, how many different
outcomes are possible? - Solution
- 4 x 8 x 5 x 6 960
13- Example 4
- There are ten teaching assistants available for
grading papers in a particular course. The exam
consists of four questions, and the professor
wishes to select different assistant to grade
each question (one assistant per question). In
how may ways can assistant be chosen to grade the
exam? - Solution
- Recall that , where n
10 and k 4 -
- thus number of permutations is
14- Example 5
- A printed circuit board has 8 different
locations in which a component can be placed. If
5 identical component are to be placed on the
board, how many different designs are possible? - Solution
- Each design is a subset of the 8 locations that
are to contain the components. The number of
possible designs is - 56
15Addition and Multiplication Rule
- Addition Rules
- Multiplication Rule
If A and B are mutually exclusive
16- Example 6
- Assume that the engine component of a spacecraft
consists of two engines in parallel. If the main
engine is 95 reliable, the backup is 80
reliable, and the engine component as a whole is
99 reliable, what is the probability that - Both engines will be operable i.e P(M ? B)?
- The main engine will fail but the backup will be
operable? i.e P(M ?B) - The engine component will fail i.e P(M ? B) ?
17- Solution
- Engine reliability
- main P(M) 0.95
- backup P(B) 0.80
- whole P(M ? B) 0.99
- Both engines operable P(M ? B)
- Recall P(M ? B) P(M)P(B) ?- P(M ? B)
- 0.95 0.80 0.99 0.76
- Main engine fail but backup operable P(M ?B)
- P(M?B) P(B) P(M?B)
- 0.80 0.76 0.04
- Engine component fails
- P(M ? B) 1 P(M ? B)
- P(M ? B) 0.01
18Conditional Probability
19- Example 7
- In studying the causes of power failures, these
data have been gathered. 5 are due to
transformer damage, 80 are due to line damage
and 1 involve both problems. Based on these
percentages, what is the probability that a given
power failure involves - line damage given that there is transformer
damage, P(LT) - transformer damage given that there is line
damage, P(TL) - transformer damage but not line damage, P(T?L)
- transformer damage given that there is no line
damage, P(TL)
20- Transformer damage P(T) 0.05,
- Line damage P(L) 0.80
- Transformer and Line damage P(T ? L) 0.01
- i)
- ii)
- iii)
- iv)
21- Example 8
-
- Table above show an example of 400 parts
classified by surface flaws and as defective.
Determine the following probabilities - Part is defective given that there is surface
flaw - Part is defective given that there is no surface
flaw - Part has surface flaw given that it is defective
- Part has no surface flaw and it is not defective
22(No Transcript)
23Independence
- Independent Events
- Two events are independent if the occurrence of
one does not affect the probability of the other
occurring. - Dependent Events
- Two events are dependent if the first event
affects the outcome or occurrence of the second
event in a way the probability is changed. - Two events A and B are called independent events
if - 1) P(AB) P(A)
- 2) P(BA) P(B)
- 3)
- Otherwise, A and B are dependent.
24- Example 9
- It is known that 30 of a certain companys
washing machines require service while under
warranty, whereas only 10 of its dryers need
such service. If someone purchases both a washer
and a dryer made by this company, what is the
probability that both machines need warranty
service? - Solution
- P(A) 0.3, P(B) 0.1. Assuming the two
machines function independently, the probability
is
25Total Probability Rule
26Bayes Theorem
27- Example 10
- A certain disease occurs in mild or severe form
three-quarter of patients have the mild form. A
new drug is available. The probability that a
mild case of the disease responds to the drug is
0.9, and the probability that a severe case
responds is 0.5. - What is the probability that a randomly chosen
case will respond to the drug, P(R)? - You are told that a certain patient has responded
to the drug. What is the probability that the
patient has the mild form of disease, P(MR)?
28- Solution
- Mild cases P(M) 0.75 Severe cases P(S)
0.25 - Response P(RM) 0.9 P(RS) 0.5
- P(R) P(R ? M) P(R?S)
-
- P(R) P(R ? M) P(R?S) 0.8.
- ,
29- Example 11
- In a bolt factory, machine 1, 2 and 3
respectively produce 20, 30 and 50 of the
total output. Of their respective outputs, 5, 3
and 2 are defective. A bolt is selected at
random - What is the probability that it is defective,
P(D)? - Given that it is defective, what is the
probability that it was made by machine 1? - Solution
- Let machine 1, 2, 3 be A, B, C respectively.
- Denote the probabilities P(A)0.2, P(B)0.3,
P(C) 0.5. - Defective Output
- P(DA)0.05, P(DB) 0.03, P(DC)0.02
30- Defective bolt P(D) P(D?A) P(D?B) P(D?C)
- Recall P(D?A) P(DA)P(A) and likewise
- P(D) P(DA)P(A) P(DB)P(B) P(DC)P(C)
- 0.01 0.009 0.01 0.029 .
- P(AD)
-
- Recall
311.2 Discrete Random Variable and Probability
Distribution
- 1.2.1 Discrete Random Variables
- 1.2.2 Discrete Probability Distribution
- 1.2.3 Expected Values of Discrete Random
Variables - 1.2.4 Binomial Distribution
- 1.2.5 Negative Binomial Distribution
- 1.2.6 Poisson Distribution
32Discrete Random Variables
- X is discrete random variable if it takes
discrete values x1, x2, , xn, with respective
probabilities associated with these values are
p1, p2, , pn, where - P(X x1) p1
- P(X x2) p2
-
- P(X xn) pn
- Furthermore, p1 p2 ... pn 1
33- Example 12
- Observe the outcome of tossing a die
34Discrete Probability Distribution
- The probability distribution of a discrete random
variable lists all the possible values that the
random variable can assume and their
corresponding probabilities. - The probability distribution or probability mass
function (pmf) for the discrete random variable
is defined for every number x by f(x)P(X x)
P - The following conditions are required for any pmf
- (positivity)
- ? f(xi) 1 (normalization)
35- The cumulative distribution function (cdf) F(x)
of a discrete random variable X with probability
mass function f(x) is defined by - F(x)
- ,
36- Example 13
-
- From a box containing four 10 cents and two 5
cents, 3 coins are selected at random without
replacement. Find - the probability distribution for the sum X, of
the 3 coins. - the cumulative mass function for the sum X, of
the 3 coins - Solution
- Let F Five cent, T Ten cent, X sum of 3
coins - Possible outcome FFT, FTF, TFF, FTT, TFT, TTF,
TTT - Possible sum, X 20, 25, 30 random variable
37 38Expected Values of Discrete Random Variables
- Expected Value of X
- Let X be a discrete random variable with set of
possible values and probability mass function
p(x). - The expected vale or mean value of X denoted by
E(X) or ?X is - Expected Value of a Function h(X)
- If the random variable has a set of possible
values n and pmf p(x), then the expected value of
any function h(X), is given by
39- Laws of Expectation (Properties of Mean, E(X))
- E(a) a where a is any constant
- E(aX) aE(X)
- E(aX b) aE(X) b
- E(X ? Y) E(X) ? E(Y)
- Variance of a Random Variable
- Let X be a random variable with probability
distribution f(x) and mean ?. The variance of X,
denote by V(X) or , or just , is
40- Example 14
- There is a chance that a bit transmitted through
a digital transmission channel is received in
error. Let X equal the number of bits in error in
the next four bits transmitted. The possible
values for X are 0, 1, 2, 3, 4. Based on a
model for the errors that is presented in the
following section, probabilities for these values
will be determined. Suppose the probabilities
are - P(X 0) 0.6561 P(X 1) 0.2916 P(X 2)
0.0486 - P(X 3) 0.0036 P(X 4) 0.0001
- The mean value of X is
- E(X) ?X
- 0(0.6561) 1(0.2916) 2(0.0486)
3(0.0036) 4(0.0001) - 0.4
41 42- The function of Eh(X)
- What is the expected value of the square of the
number of bits in error? - h(X) X2
- Remember
- Eh(X) 02 x 0.6561 12 x 0.2916 22 x
0.0486 - 32 x 0.0036 42 x 0.0001
- 0.52
43Binomial Distribution
- Binomial distribution is a commonly used discrete
probability distribution since many statistical
problems are deal with the situations referred to
as repeated trial. - Those experiment that possess the following
properties are called binomial experiment - Properties of a Binomial Experiment
- The experiment consists of a sequence of n
identical trials. - Two outcomes, success and failure, are possible
on each trial. - The probability of a success, denoted by p, does
not change from trial to trial. - The trials are independent.
44- Our interest is in the number of successes
occurring in the n trials. Let x denote the
number of successes occurring in the n trials. - Number of experimental outcomes providing exactly
x successes in n trials -
-
- where n! n(n 1)(n 2) . . . (2)(1)
- 0! 1
- Probability of a particular sequence of trial
outcomes with x successes in n trials is
45- Probability mass function
- Probability mass function of binomial random
variable X depends on the two parameters n(number
of trials) and p (probability of success),
denoted as -
- where n number of trials
- x number of success among n trials
- p probability of success in any one trial
- q probability of failure in any one
trial.
46- P(Xx) f(x) probability of getting
exactly x success among the n trails. - Note A short notation to designate that X has
the binomial distribution with parameter n and p
is and - Mean ? np
- Variance ?2 n pq
47(No Transcript)
48- Example 15
- A software company Mysoftcom is concerned about
a low retention rate for employees. On the basis
of past experience, management has seen a
turnover of 10 of the hourly employees annually.
Thus, for any hourly employees chosen at random,
management estimates a probability of 0.1 that
the person will not be with the company next
year. - Choosing 3 hourly employees at random, what is
the probability that 1 of them will leave the
company this year?
49- Solution
- Recall the Binomial probability distribution
- Given n 3, x 1, p0.1, thus
- One can also use the Tables of Binomial
Probabilities
50- Example 16
- The probability that a patient recovers from
SARS is 0.4. If 15 people are known to have
contracted this disease, what is the probability
that - at least 13 survive
- at most 2 die
- Solution
- n15, survival probability, ps 0.4, dying
probability pd0.6 - X BIN(15, 0.4)
- P(X ?13) P(X13) P(X14) P(X15)
0.0002789 - Y BIN(15, 0.6)
- P(Y ? 2) P(Y0) P(Y1) P(Y2) 0.0002789
51Negative Binomial Distribution
- The negative binomial random variable and
distribution are based on an experiment
satisfying the following conditions - There are only two possible outcomes for each
trial - The trails must be independent
- The probability must remain constant for each
trial. - The experiment continues (trials are performed)
until a total of r successes have been observed,
where r is a specified positive integer.
52- Probability mass function
- The pmf of the negative binomial random variable
X with parameter r and p is - x 0, 1, 2,
- where r number of success (S)
- x number of failures that precede the rth
success - p probability of success in any one trail
- Mean
- Variance
53- For special case r 1, the pmf becomes
-
- x 0, 1 , 2,
- This pmf is called geometric distribution. In is
usually written as
54Poisson Distribution
- The Poisson distribution is a discrete
probability distribution that applies to
occurrences of some event over a specific
interval. The random variable X is the number of
occurrences of the event in an interval. - The interval can be time, distance, area, volume,
or some similar unit. - Those experiment that possess the following
properties are called Poisson experiment - The occurrences must be random.
- The occurrences must be independent from one
interval to another. - The occurrence must be uniformly distributed over
the interval being used.
55- The probability distribution of the Poisson
random variable X, representing the number of
outcomes occurring over an interval is given by
the formula - P(Xx) f(x ) x 0, 1, 2, and
- with
- Mean ? ?,
- Variance ?2 ?
- Note A short notation to designate that X has
the Poisson distribution with parameter ? is
56- Example of occurrences
- The number of cars that pass through a certain
point on a road (sufficiently distant from
traffic lights) during a given period of time. - The number of spelling mistakes one makes while
typing a single page. - The number of phone calls at a call center per
minute. - The number of times a web server is accessed per
minute. - The number of mutations in a given stretch of DNA
after a certain amount of radiation. - The number of pine trees per unit area of mixed
forest. - The number of stars in a given volume of space.
57- Example 17
- Patients arrive at the emergency room of
Putrajaya Hospital at the average rate of 6 per
hour on weekend evenings. What is the
probability of 4 arrivals in 30 minutes on a
weekend evening? - Solution
- Given that ? 6/hour 3/half-hour, and x 4,
therefore probability of 4 arrivals in 30 minutes
is
58- Or one can also use the Tables of Poisson
Probabilities
59- Example 18
- Customers arrive randomly at a department store
at an average rate of 3.4 per minute. Find - No customer arrives in any particular minute.
- Two or more customers arrive in any particular
minute. - One or more customers arrive in any 30-second
period. - Solution
- ?3.4, X POI(3.4)
- P(X 0) 0.03337
- P(X ?2) 1? P(X?1) 0.8532
- Y POI(1.7), P(Y?1) 0.8173
601.3 Continuous Random Variable and Probability
Distribution
- 1.3.1 Continuous Random Variables
- 1.3.2 Continuous Probability Distribution
- 1.3.3 Expected Values of Continuous Random
Variables - 1.3.4 Uniform Distribution
- 1.3.5 Normal (Gauss) Distribution
- 1.3.6 Relationship between Gaussian and Binomial
Distribution
61Continuous Random Variables
- A continuous random variable has infinite many
values, and those values can be associated with
measurements on a continuous scale. - For instance, if X is the result of rolling a die
(and observing the uppermost face), then X is a
discrete random variable with possible values 1,
2, 3, 4, 5 and 6. On the other hand, if X is a
random choice of a real number in the interval
1,6, then it is a continuous random variable. - If X is a random variable, we are usually
interested in the probability that X takes on a
value in a certain range.
62Continuous Probability Distribution
- The function f(x) is a probability distribution
or probability density function (pdf) for the
continuous random variable X, defined over the
set of real number R, if - f(x) ? 0
- ,
- ,
63- The cumulative distribution function, F(x) of a
continuous random variable X with density
function f(x) is defined by - Note Let X be a continuous random variable with
pdf f(x) and cdf F(x), then for any number a, - and for any two numbers a and b with a lt b,
64- Example 19
- Suppose that a battery failure time, measured in
hours, has a probability density function of - What is the probability that the battery fails
within the first 5 hours?
65(No Transcript)
66- Example 20
- Let X denote the amount of time for which a book
on two hour reserve at a university library is
checked out by a randomly selected student and
suppose that X has probability density function -
-
- Calculate the following probabilities
- P(X ? 1)
- P(0.5 ? X ? 1.5)
- P(1.5 lt X)
67- Solution
- P(x ? 1)
- P(0.5 ? X ? 1.5)
- P(x gt 1.5)
68Expected Values of Continuous Random Variables
- Expected Value of X
- Let X be a continuous random variable with
probability distribution f(x). The mean or
expected value of X is given by - ?x E(X)
- Expected Value of a Function h(X)
- The mean or expected value of a function of
random variable h(X) is - Eh(X) ?h(x)
69- Variance of a Random Variable
- The variance of X, denote by V(X) or , or
just , is - V(X) ?2
- or
- ?2 EX2 - ?2
- The positive square root of the variance, ?x, is
called the standard deviation of X. - Law of Variance
- Var (a)0 where a is any constant
- Var (aX) a2Var (X)
- Var (aXb) a2Var (X)
- Var (Xb) Var (X)
70- Example 21
- The weekly demand for Pepsi, in thousands of
liters, from a local chain of efficiency stores,
is a continuous random variable X having the
probability density -
- Find the variance of X.
71- Solution
- Recall that
-
- where the mean,
- And where the mean of a function is,
- Therefore the variance,
-
72- Example 22
- Let X be a random variable with density function
- Find the variance of the random variable g(X)
4X 3 - Solution
73Uniform Distribution
- The uniform distribution is a probability
distribution in which the probability of a value
occurring between two points, a and b, is the
same as the probability between any other two
points, c and d, given that the distribution
between a and b is equal to the distance between
c and d. - where
- f(x) Value of the density function
at any x value - a lower limit of the interval from a to b
- b upper limit of the interval from a to b
74- The probability density function and cumulative
distribution function for a continuous uniform
distribution on the interval are -
-
- Mean
- Variance
75 76Normal / Gaussian Distribution
- The bell shaped curve also known as normal curve
is widely used to approximately many phenomena. - The random variable with normal distribution is
characterized by with its mean ? and variance ?2.
- The pdf is given by
77- Standard Normal Distribution
- The standard normal distribution is a normal
probability distribution that has a mean of 0 and
a standard deviation of 1. - All the observations of any normal variable X can
be transformed to a new set of observations of
standard normal variable z with mean 0 and
variance 1. - We can find the areas under the standard normal
curve by referring to Standard Normal Tables
which give cumulative probabilities ?(z).
78- Standard Normal Curve Areas, ?(z) P(Z ? z)
79- Converting Nonstandard Normal Distribution
- If X has a normal distribution with mean u and
standard deviation s, then - has a standard normal distribution. Thus
-
- and
- and
80Normal Distribution Table
81(No Transcript)
82 83- Example 25
- The masses of articles produced in a particular
workshop are normally distributed with mean ? and
standard deviation ?. 5 of the articles have a
mass greater than 85g and 10 have a mass less
than 25g. Find - ? and ?.
- P(X gt 60)
- a, given that P( a lt X lt 74.85) 0.75
84 85- P(X gt 60)
- a, given that P( a lt X lt 74.85) 0.75
86Relationship between Gaussian and Binomial
Distribution
- The Gaussian distribution can be derived from the
binomial (or Poisson) assuming - p is finite
- N is very large
- we have a continuous variable rather than a
discrete variable - Limit distribution
- ? np and
Normal approximation to the binomial
87- Normal Approximation to the Binomial Distribution
- Normal Approximation to the Poisson Distribution
88END CHAPTER 1