Chapter 1: Discrete and Continuous Probability Distributions - PowerPoint PPT Presentation

1 / 88
About This Presentation
Title:

Chapter 1: Discrete and Continuous Probability Distributions

Description:

Probability refers to the study of randomness and uncertainty. ... A restaurant has four kinds of soups, eight kinds of main course, five kinds of ... – PowerPoint PPT presentation

Number of Views:1679
Avg rating:3.0/5.0
Slides: 89
Provided by: nmy5
Category:

less

Transcript and Presenter's Notes

Title: Chapter 1: Discrete and Continuous Probability Distributions


1
Chapter 1 Discrete and Continuous Probability
Distributions
  • 1.1 Probability Theory
  • 1.2 Discrete Probability Distribution
  • 1.3 Continuous Probability Distribution

2
1.1 Probability Theory
  • 1.1.1 Basic Probability Theory
  • 1.1.2 Set Theory Operations
  • 1.1.3 Counting Techniques
  • 1.1.4 Addition, Multiplication
  • 1.1.5 Conditional Probability
  • 1.1.6 Independence
  • 1.1.7 Total Probability Rule Bayes Theorem

3
Basics of Probability Theory
  • Probability refers to the study of randomness and
    uncertainty.
  • In other words, probability is a numerical
    measure of chance for the occurrence of an event.

4
  • Example 1
  • An experiment consists of tossing three coins.
    Find the sample space if
  • We are interested in the observed face of each
    coin,
  • We are interested in the total number of heads
    obtained.
  • Solution
  • S HHH, HHT, HTH, THH, HTT, THT, TTH, TTT
  • S 0, 1, 2, 3

5
Frequentist Interpretation
  • Intuitively, the probability of an event is the
    proportion of times the event should occur when
    the experiment is run a large number of times.
  • Suppose S is a sample space in which all
    outcomes are assumed to be equally likely, and E
    is an event. Then the probability of E,
    denoted by P(E), is
  • Probability is quantified as a percentage or a
    proportion of the total possible outcomes.

6
Subjective Interpretation
  • There is yet another definition of probability,
    called as the subjective probability, which is
    the foundation of Bayesian Econometrics.
  • Under the subjective or degrees of belief"
    definition of probability, you can ask question
    such as
  • What is the probability that there will be a
    stock market boom in 2005?
  • What is the probability that Brazil will win the
    World Cup?

7
Set Operations on Events
8
(No Transcript)
9
  • Example 2
  • For the experiment in which the number of pumps
    in use at a single six-pump gas station is
    observed. Let A 0, 1, 2, 3, 4, B 3, 4, 5,
    6, and C 1, 3, 5. Then find
  • (a) A?B (b) A?B (c) A?C (d) (A?C)
  • Solutions
  • (a) A?B 0, 1, 2, 3, 4 ? 3, 4, 5, 6 3, 4
  • (b) A?B 0,1,2,3,4 ? 3, 4, 5, 6 0, 1, 2,
    3, 4, 5, 6 S
  • (c) A?C 0, 1, 2, 3, 4? 1, 3, 5 0, 1, 2,
    3, 4, 5
  • (d) (A?C) S (A?C) 6

10
Counting Techniques
  • Product Rule
  • If an operation can be described as a sequence of
    k steps, and
  • if the number of ways of completing step 1 is
    n1, and
  • if the number of completing step two is n2 for
    each way of completing step 1, and so forth,
  • the total number of ways of completing the
    operation is n1 x n2 x n3 x .. X nk

11
  • Permutation
  • Any ordered sequence of k objects taken from a
    set of n distinct objects is called a permutation
    of size k of the objects.
  • The number of permutation of size k that can be
    constructed from n objects is denoted by
  • Combination
  • Given a set of n distinct objects, any unordered
    subset of size k of the objects is called a
    combination.
  • The number of combination of size k that can be
    constructed from n objects is denoted by

12
  • Example 3
  • A restaurant has four kinds of soups, eight
    kinds of main course, five kinds of dessert, and
    six kinds of drinks. If a customer selects one
    item from each category, how many different
    outcomes are possible?
  • Solution
  • 4 x 8 x 5 x 6 960

13
  • Example 4
  • There are ten teaching assistants available for
    grading papers in a particular course. The exam
    consists of four questions, and the professor
    wishes to select different assistant to grade
    each question (one assistant per question). In
    how may ways can assistant be chosen to grade the
    exam?
  • Solution
  • Recall that , where n
    10 and k 4
  • thus number of permutations is

14
  • Example 5
  • A printed circuit board has 8 different
    locations in which a component can be placed. If
    5 identical component are to be placed on the
    board, how many different designs are possible?
  • Solution
  • Each design is a subset of the 8 locations that
    are to contain the components. The number of
    possible designs is
  • 56

15
Addition and Multiplication Rule
  • Addition Rules
  • Multiplication Rule

If A and B are mutually exclusive
16
  • Example 6
  • Assume that the engine component of a spacecraft
    consists of two engines in parallel. If the main
    engine is 95 reliable, the backup is 80
    reliable, and the engine component as a whole is
    99 reliable, what is the probability that
  • Both engines will be operable i.e P(M ? B)?
  • The main engine will fail but the backup will be
    operable? i.e P(M ?B)
  • The engine component will fail i.e P(M ? B) ?

17
  • Solution
  • Engine reliability
  • main P(M) 0.95
  • backup P(B) 0.80
  • whole P(M ? B) 0.99
  • Both engines operable P(M ? B)
  • Recall P(M ? B) P(M)P(B) ?- P(M ? B)
  • 0.95 0.80 0.99 0.76
  • Main engine fail but backup operable P(M ?B)
  • P(M?B) P(B) P(M?B)
  • 0.80 0.76 0.04
  • Engine component fails
  • P(M ? B) 1 P(M ? B)
  • P(M ? B) 0.01

18
Conditional Probability
19
  • Example 7
  • In studying the causes of power failures, these
    data have been gathered. 5 are due to
    transformer damage, 80 are due to line damage
    and 1 involve both problems. Based on these
    percentages, what is the probability that a given
    power failure involves
  • line damage given that there is transformer
    damage, P(LT)
  • transformer damage given that there is line
    damage, P(TL)
  • transformer damage but not line damage, P(T?L)
  • transformer damage given that there is no line
    damage, P(TL)

20
  • Transformer damage P(T) 0.05,
  • Line damage P(L) 0.80
  • Transformer and Line damage P(T ? L) 0.01
  • i)
  • ii)
  • iii)
  • iv)

21
  • Example 8
  • Table above show an example of 400 parts
    classified by surface flaws and as defective.
    Determine the following probabilities
  • Part is defective given that there is surface
    flaw
  • Part is defective given that there is no surface
    flaw
  • Part has surface flaw given that it is defective
  • Part has no surface flaw and it is not defective

22
(No Transcript)
23
Independence
  • Independent Events
  • Two events are independent if the occurrence of
    one does not affect the probability of the other
    occurring.
  • Dependent Events
  • Two events are dependent if the first event
    affects the outcome or occurrence of the second
    event in a way the probability is changed.
  • Two events A and B are called independent events
    if
  • 1) P(AB) P(A)
  • 2) P(BA) P(B)
  • 3)
  • Otherwise, A and B are dependent.

24
  • Example 9
  • It is known that 30 of a certain companys
    washing machines require service while under
    warranty, whereas only 10 of its dryers need
    such service. If someone purchases both a washer
    and a dryer made by this company, what is the
    probability that both machines need warranty
    service?
  • Solution
  • P(A) 0.3, P(B) 0.1. Assuming the two
    machines function independently, the probability
    is

25
Total Probability Rule
26
Bayes Theorem
27
  • Example 10
  • A certain disease occurs in mild or severe form
    three-quarter of patients have the mild form. A
    new drug is available. The probability that a
    mild case of the disease responds to the drug is
    0.9, and the probability that a severe case
    responds is 0.5.
  • What is the probability that a randomly chosen
    case will respond to the drug, P(R)?
  • You are told that a certain patient has responded
    to the drug. What is the probability that the
    patient has the mild form of disease, P(MR)?

28
  • Solution
  • Mild cases P(M) 0.75 Severe cases P(S)
    0.25
  • Response P(RM) 0.9 P(RS) 0.5
  • P(R) P(R ? M) P(R?S)
  • P(R) P(R ? M) P(R?S) 0.8.
  • ,

29
  • Example 11
  • In a bolt factory, machine 1, 2 and 3
    respectively produce 20, 30 and 50 of the
    total output. Of their respective outputs, 5, 3
    and 2 are defective. A bolt is selected at
    random
  • What is the probability that it is defective,
    P(D)?
  • Given that it is defective, what is the
    probability that it was made by machine 1?
  • Solution
  • Let machine 1, 2, 3 be A, B, C respectively.
  • Denote the probabilities P(A)0.2, P(B)0.3,
    P(C) 0.5.
  • Defective Output
  • P(DA)0.05, P(DB) 0.03, P(DC)0.02

30
  • Defective bolt P(D) P(D?A) P(D?B) P(D?C)
  • Recall P(D?A) P(DA)P(A) and likewise
  • P(D) P(DA)P(A) P(DB)P(B) P(DC)P(C)
  • 0.01 0.009 0.01 0.029 .
  • P(AD)
  • Recall

31
1.2 Discrete Random Variable and Probability
Distribution
  • 1.2.1 Discrete Random Variables
  • 1.2.2 Discrete Probability Distribution
  • 1.2.3 Expected Values of Discrete Random
    Variables
  • 1.2.4 Binomial Distribution
  • 1.2.5 Negative Binomial Distribution
  • 1.2.6 Poisson Distribution

32
Discrete Random Variables
  • X is discrete random variable if it takes
    discrete values x1, x2, , xn, with respective
    probabilities associated with these values are
    p1, p2, , pn, where
  • P(X x1) p1
  • P(X x2) p2
  • P(X xn) pn
  • Furthermore, p1 p2 ... pn 1

33
  • Example 12
  • Observe the outcome of tossing a die

34
Discrete Probability Distribution
  • The probability distribution of a discrete random
    variable lists all the possible values that the
    random variable can assume and their
    corresponding probabilities.
  • The probability distribution or probability mass
    function (pmf) for the discrete random variable
    is defined for every number x by f(x)P(X x)
    P
  • The following conditions are required for any pmf
  • (positivity)
  • ? f(xi) 1 (normalization)

35
  • The cumulative distribution function (cdf) F(x)
    of a discrete random variable X with probability
    mass function f(x) is defined by
  • F(x)
  • ,

36
  • Example 13
  • From a box containing four 10 cents and two 5
    cents, 3 coins are selected at random without
    replacement. Find
  • the probability distribution for the sum X, of
    the 3 coins.
  • the cumulative mass function for the sum X, of
    the 3 coins
  • Solution
  • Let F Five cent, T Ten cent, X sum of 3
    coins
  • Possible outcome FFT, FTF, TFF, FTT, TFT, TTF,
    TTT
  • Possible sum, X 20, 25, 30 random variable

37
  • pmf
  • cdf

38
Expected Values of Discrete Random Variables
  • Expected Value of X
  • Let X be a discrete random variable with set of
    possible values and probability mass function
    p(x).
  • The expected vale or mean value of X denoted by
    E(X) or ?X is
  • Expected Value of a Function h(X)
  • If the random variable has a set of possible
    values n and pmf p(x), then the expected value of
    any function h(X), is given by

39
  • Laws of Expectation (Properties of Mean, E(X))
  • E(a) a where a is any constant
  • E(aX) aE(X)
  • E(aX b) aE(X) b
  • E(X ? Y) E(X) ? E(Y)
  • Variance of a Random Variable
  • Let X be a random variable with probability
    distribution f(x) and mean ?. The variance of X,
    denote by V(X) or , or just , is

40
  • Example 14
  • There is a chance that a bit transmitted through
    a digital transmission channel is received in
    error. Let X equal the number of bits in error in
    the next four bits transmitted. The possible
    values for X are 0, 1, 2, 3, 4. Based on a
    model for the errors that is presented in the
    following section, probabilities for these values
    will be determined. Suppose the probabilities
    are
  • P(X 0) 0.6561 P(X 1) 0.2916 P(X 2)
    0.0486
  • P(X 3) 0.0036 P(X 4) 0.0001
  • The mean value of X is
  • E(X) ?X
  • 0(0.6561) 1(0.2916) 2(0.0486)
    3(0.0036) 4(0.0001)
  • 0.4

41
  • The variance of X is

42
  • The function of Eh(X)
  • What is the expected value of the square of the
    number of bits in error?
  • h(X) X2
  • Remember
  • Eh(X) 02 x 0.6561 12 x 0.2916 22 x
    0.0486
  • 32 x 0.0036 42 x 0.0001
  • 0.52

43
Binomial Distribution
  • Binomial distribution is a commonly used discrete
    probability distribution since many statistical
    problems are deal with the situations referred to
    as repeated trial.
  • Those experiment that possess the following
    properties are called binomial experiment
  • Properties of a Binomial Experiment
  • The experiment consists of a sequence of n
    identical trials.
  • Two outcomes, success and failure, are possible
    on each trial.
  • The probability of a success, denoted by p, does
    not change from trial to trial.
  • The trials are independent.

44
  • Our interest is in the number of successes
    occurring in the n trials. Let x denote the
    number of successes occurring in the n trials.
  • Number of experimental outcomes providing exactly
    x successes in n trials
  • where n! n(n 1)(n 2) . . . (2)(1)
  • 0! 1
  • Probability of a particular sequence of trial
    outcomes with x successes in n trials is

45
  • Probability mass function
  • Probability mass function of binomial random
    variable X depends on the two parameters n(number
    of trials) and p (probability of success),
    denoted as
  • where n number of trials
  • x number of success among n trials
  • p probability of success in any one trial
  • q probability of failure in any one
    trial.

46
  • P(Xx) f(x) probability of getting
    exactly x success among the n trails.
  • Note A short notation to designate that X has
    the binomial distribution with parameter n and p
    is and
  • Mean ? np
  • Variance ?2 n pq

47
(No Transcript)
48
  • Example 15
  • A software company Mysoftcom is concerned about
    a low retention rate for employees. On the basis
    of past experience, management has seen a
    turnover of 10 of the hourly employees annually.
    Thus, for any hourly employees chosen at random,
    management estimates a probability of 0.1 that
    the person will not be with the company next
    year.
  • Choosing 3 hourly employees at random, what is
    the probability that 1 of them will leave the
    company this year?

49
  • Solution
  • Recall the Binomial probability distribution
  • Given n 3, x 1, p0.1, thus
  • One can also use the Tables of Binomial
    Probabilities

50
  • Example 16
  • The probability that a patient recovers from
    SARS is 0.4. If 15 people are known to have
    contracted this disease, what is the probability
    that
  • at least 13 survive
  • at most 2 die
  • Solution
  • n15, survival probability, ps 0.4, dying
    probability pd0.6
  • X BIN(15, 0.4)
  • P(X ?13) P(X13) P(X14) P(X15)
    0.0002789
  • Y BIN(15, 0.6)
  • P(Y ? 2) P(Y0) P(Y1) P(Y2) 0.0002789

51
Negative Binomial Distribution
  • The negative binomial random variable and
    distribution are based on an experiment
    satisfying the following conditions
  • There are only two possible outcomes for each
    trial
  • The trails must be independent
  • The probability must remain constant for each
    trial.
  • The experiment continues (trials are performed)
    until a total of r successes have been observed,
    where r is a specified positive integer.

52
  • Probability mass function
  • The pmf of the negative binomial random variable
    X with parameter r and p is
  • x 0, 1, 2,
  • where r number of success (S)
  • x number of failures that precede the rth
    success
  • p probability of success in any one trail
  • Mean
  • Variance

53
  • For special case r 1, the pmf becomes
  • x 0, 1 , 2,
  • This pmf is called geometric distribution. In is
    usually written as

54
Poisson Distribution
  • The Poisson distribution is a discrete
    probability distribution that applies to
    occurrences of some event over a specific
    interval. The random variable X is the number of
    occurrences of the event in an interval.
  • The interval can be time, distance, area, volume,
    or some similar unit.
  • Those experiment that possess the following
    properties are called Poisson experiment
  • The occurrences must be random.
  • The occurrences must be independent from one
    interval to another.
  • The occurrence must be uniformly distributed over
    the interval being used.

55
  • The probability distribution of the Poisson
    random variable X, representing the number of
    outcomes occurring over an interval is given by
    the formula
  • P(Xx) f(x ) x 0, 1, 2, and
  • with
  • Mean ? ?,
  • Variance ?2 ?
  • Note A short notation to designate that X has
    the Poisson distribution with parameter ? is

56
  • Example of occurrences
  • The number of cars that pass through a certain
    point on a road (sufficiently distant from
    traffic lights) during a given period of time.
  • The number of spelling mistakes one makes while
    typing a single page.
  • The number of phone calls at a call center per
    minute.
  • The number of times a web server is accessed per
    minute.
  • The number of mutations in a given stretch of DNA
    after a certain amount of radiation.
  • The number of pine trees per unit area of mixed
    forest.
  • The number of stars in a given volume of space.

57
  • Example 17
  • Patients arrive at the emergency room of
    Putrajaya Hospital at the average rate of 6 per
    hour on weekend evenings. What is the
    probability of 4 arrivals in 30 minutes on a
    weekend evening?
  • Solution
  • Given that ? 6/hour 3/half-hour, and x 4,
    therefore probability of 4 arrivals in 30 minutes
    is

58
  • Or one can also use the Tables of Poisson
    Probabilities

59
  • Example 18
  • Customers arrive randomly at a department store
    at an average rate of 3.4 per minute. Find
  • No customer arrives in any particular minute.
  • Two or more customers arrive in any particular
    minute.
  • One or more customers arrive in any 30-second
    period.
  • Solution
  • ?3.4, X POI(3.4)
  • P(X 0) 0.03337
  • P(X ?2) 1? P(X?1) 0.8532
  • Y POI(1.7), P(Y?1) 0.8173

60
1.3 Continuous Random Variable and Probability
Distribution
  • 1.3.1 Continuous Random Variables
  • 1.3.2 Continuous Probability Distribution
  • 1.3.3 Expected Values of Continuous Random
    Variables
  • 1.3.4 Uniform Distribution
  • 1.3.5 Normal (Gauss) Distribution
  • 1.3.6 Relationship between Gaussian and Binomial
    Distribution

61
Continuous Random Variables
  • A continuous random variable has infinite many
    values, and those values can be associated with
    measurements on a continuous scale.
  • For instance, if X is the result of rolling a die
    (and observing the uppermost face), then X is a
    discrete random variable with possible values 1,
    2, 3, 4, 5 and 6. On the other hand, if X is a
    random choice of a real number in the interval
    1,6, then it is a continuous random variable.
  • If X is a random variable, we are usually
    interested in the probability that X takes on a
    value in a certain range.

62
Continuous Probability Distribution
  • The function f(x) is a probability distribution
    or probability density function (pdf) for the
    continuous random variable X, defined over the
    set of real number R, if
  • f(x) ? 0
  • ,
  • ,

63
  • The cumulative distribution function, F(x) of a
    continuous random variable X with density
    function f(x) is defined by
  • Note Let X be a continuous random variable with
    pdf f(x) and cdf F(x), then for any number a,
  • and for any two numbers a and b with a lt b,

64
  • Example 19
  • Suppose that a battery failure time, measured in
    hours, has a probability density function of
  • What is the probability that the battery fails
    within the first 5 hours?

65
(No Transcript)
66
  • Example 20
  • Let X denote the amount of time for which a book
    on two hour reserve at a university library is
    checked out by a randomly selected student and
    suppose that X has probability density function
  • Calculate the following probabilities
  • P(X ? 1)
  • P(0.5 ? X ? 1.5)
  • P(1.5 lt X)

67
  • Solution
  • P(x ? 1)
  • P(0.5 ? X ? 1.5)
  • P(x gt 1.5)

68
Expected Values of Continuous Random Variables
  • Expected Value of X
  • Let X be a continuous random variable with
    probability distribution f(x). The mean or
    expected value of X is given by
  • ?x E(X)
  • Expected Value of a Function h(X)
  • The mean or expected value of a function of
    random variable h(X) is
  • Eh(X) ?h(x)

69
  • Variance of a Random Variable
  • The variance of X, denote by V(X) or , or
    just , is
  • V(X) ?2
  • or
  • ?2 EX2 - ?2
  • The positive square root of the variance, ?x, is
    called the standard deviation of X.
  • Law of Variance
  • Var (a)0 where a is any constant
  • Var (aX) a2Var (X)
  • Var (aXb) a2Var (X)
  • Var (Xb) Var (X)

70
  • Example 21
  • The weekly demand for Pepsi, in thousands of
    liters, from a local chain of efficiency stores,
    is a continuous random variable X having the
    probability density
  • Find the variance of X.

71
  • Solution
  • Recall that
  • where the mean,
  • And where the mean of a function is,
  • Therefore the variance,

72
  • Example 22
  • Let X be a random variable with density function
  • Find the variance of the random variable g(X)
    4X 3
  • Solution

73
Uniform Distribution
  • The uniform distribution is a probability
    distribution in which the probability of a value
    occurring between two points, a and b, is the
    same as the probability between any other two
    points, c and d, given that the distribution
    between a and b is equal to the distance between
    c and d.
  • where
  • f(x) Value of the density function
    at any x value
  • a lower limit of the interval from a to b
  • b upper limit of the interval from a to b

74
  • The probability density function and cumulative
    distribution function for a continuous uniform
    distribution on the interval are
  • Mean
  • Variance

75
  • Example 23

76
Normal / Gaussian Distribution
  • The bell shaped curve also known as normal curve
    is widely used to approximately many phenomena.
  • The random variable with normal distribution is
    characterized by with its mean ? and variance ?2.
  • The pdf is given by

77
  • Standard Normal Distribution
  • The standard normal distribution is a normal
    probability distribution that has a mean of 0 and
    a standard deviation of 1.
  • All the observations of any normal variable X can
    be transformed to a new set of observations of
    standard normal variable z with mean 0 and
    variance 1.
  • We can find the areas under the standard normal
    curve by referring to Standard Normal Tables
    which give cumulative probabilities ?(z).

78
  • Standard Normal Curve Areas, ?(z) P(Z ? z)

79
  • Converting Nonstandard Normal Distribution
  • If X has a normal distribution with mean u and
    standard deviation s, then
  • has a standard normal distribution. Thus
  • and
  • and

80
Normal Distribution Table
81
(No Transcript)
82
  • Example 24

83
  • Example 25
  • The masses of articles produced in a particular
    workshop are normally distributed with mean ? and
    standard deviation ?. 5 of the articles have a
    mass greater than 85g and 10 have a mass less
    than 25g. Find
  • ? and ?.
  • P(X gt 60)
  • a, given that P( a lt X lt 74.85) 0.75

84
  • ? and ?.



85
  • P(X gt 60)
  • a, given that P( a lt X lt 74.85) 0.75

86
Relationship between Gaussian and Binomial
Distribution
  • The Gaussian distribution can be derived from the
    binomial (or Poisson) assuming
  • p is finite
  • N is very large
  • we have a continuous variable rather than a
    discrete variable
  • Limit distribution
  • ? np and

Normal approximation to the binomial
87
  • Normal Approximation to the Binomial Distribution
  • Normal Approximation to the Poisson Distribution

88
END CHAPTER 1
Write a Comment
User Comments (0)
About PowerShow.com