3rd Edition: Chapter 1

1
Review of Probability

• 0.1 Axioms of Probability
• 0.2 Conditional Probability and Independence
• 0.3 Discrete Random Variables
• 0.4 Continuous Random Variables
• 0.5 Expectations and Variances
• 0.6 Joint Distributions of Two Random Variables
• 0.7 Some Famous Random Variables

2
Random Certain

• Overview
• Whats Random?
• Whats Certain?
• Whats Impossible?
• Examples
• Disintegration of a given atom of radium
• Finding no defect during inspection of a
  microwave oven
• Orbit satellite in space is at a certain position
• An object travels faster than light
• A thunderstorm flashes of lighting precede any
  thunder echoes

An event may or may not occur
Occurrence of an event is inevitable
An event can never occur
3
Sample Space and Events

• Sample Space

The set of all possible outcomes, denoted by S

• Sample points

These outcomes are called sample points, or
points.

• Events

Certain subsets of S are referred to as events.
4
0.1 Axioms of Probability

• Definition Probability Axioms

Let S be the sample space of a random phenomenon.
Suppose that to each event A of S, a number
denoted by P(A) is associated with A. If P
satisfies the following axioms, then it is called
a probability and the number P(A) is said to be
the probability of A. Axiom 1 P(A) ?
0 Axiom 2 P(S) 1 Axiom 3 if A1, A2,
A3, is a sequence of
mutually exclusive events, then
Equally likely P(A) P(B)
P(?1) P(?2)
5
Inclusion-Exclusion Principle

• For 3 events
• For n events
• Theorem 0.1

6
0.2 Conditional Probability

• Definition
• If P(B) gt 0, the conditional probability of A
  given B, defined by P(AB), is

7
Example 0.1
From the set of families with two children, a
family is selected at random and is found to have
a girl. What is the probability that the other
child of the family is a girl? Assume that in a
two-child family all sex distributions are
equally probable. Sol
Ans1/3
8
Example 0.2
An English class consists of 10 Koreans, 5
Italians, and 15 Hispanics. A paper is found
belonging to one of the students of this class.
If the name on the paper is not Korean, what is
the probability that it is Italian? Assume that
names completely identify ethnic groups. Sol
Ans1/4
9
Reduction of Sample Space
The conditional probability P(AB) over sample
space S is equal to the probability P(A) over
the reduced sample space S B.
10
Example 0.3
We draw eight cards at random from an ordinary
deck of 52 cards. Given that three of them are
spades, what is the probability that the
remaining five are also spades? Sol
11
Theorem 0.2
Let S be the sample space of an experiment, and
let B be an event of S with P(B) gt 0. Then (a)
P(AB) ? 0 for any event A of S. (b) P(SB)
1. (c) If A1, A2,, is a sequence of mutually
exclusive events, then Pf
12
Properties Summary
1. P(?B) 0 2. P(AcB) 1-P(AB) 3.
If C?A, then P(ACcB) P(A-CB)
P(AB)-P(CB) 4. If C?A, then P(CB) ? P(AB)
5. P(A?CB) P(AB) P(CB)-P(ACB) 6.
P(AB) P(ACB) P(ACcB) 7. P(A1?A2?
?AnB) ? 8.
13
Law of Multiplication
From the definition of conditional probability
Then we have
. Similarly,
.
14
Example 0.4

• Suppose that 5 good fuses and 2 defective
  ones have been mixed up. To find the defective
  fuses, we test them one-by-one, at random and
  without replacement. What is the probability that
  we are lucky and find both of the defective fuses
  in the first two tests?
• Sol

Ans1/21
15
Theorem 0.3
Pf
16
Example 0.5
Suppose that 5 good fuses and 2 defective ones
have been mixed up. To find the defective fuses,
we test them one-by-one, at random and without
replacement. What is the probability that we are
lucky and find both of the defective fuses in
exactly 3 tests? Sol
Ans2/21
17
Theorem 0.4 Law of Total Probability
Let B be an event with P(B) gt 0 and P(Bc) gt 0.
Then for any event A, Pf
18
Example 0.6
In a trial, the judge is 65 sure that Susan has
committed a crime. Julie and Robert are two
witness who know whether Susan is innocent or
guilty. However, Robert is Susans friend and
will lie with probability 0.25 if Susan is
guilty. He will tell the truth if she is
innocent. Julie is Susans enemy and will lie
with probability 0.30 if Susan is innocent. She
will tell the truth if Susan is guilty. What is
the probability that Robert and Julie will give
conflicting testimony? Sol
Ans0.2675
19
Definition of Partition
Let B1, B2, , Bn be a set of nonempty subsets
of the sample space S of an experiment. If the
events B1, B2, , Bn are mutually exclusive and
, the set B1, B2, , Bn is called
a partition of S.
20
Theorem 0.5Law of Total Prob.
If B1, B2, , Bn is a partition of the sample
space of an experiment and P(Bi) gt 0 for i
1,2,, n, then for any event A of S, Pf
21
Example 0.7
Suppose that 80 of the seniors, 70 of the
juniors, 50 of the sophomores, and 30 of the
freshmen of a college use the library of their
campus frequently. If 30 of all students are
freshmen, 25 are sophomores, 25 are juniors,
and 20 are seniors, what percent of all students
use the library frequently? Sol
Ans0.55
22
Example 0.8
An urn contains 10 white and 12 red chips. Two
chips are drawn at random and, without looking at
their colors, are discarded. What is the
probability that a third chip drawn is red? Sol
Ans6/11
23
Theorem 0.6 Bayes Formula
Let B1, B2, , Bn is a partition of the sample
space of an experiment. If for i 1,2,, n,
P(Bi) gt 0, then for any event A of S with P(A) gt
0 , If P(B) gt 0 and P(Bc) gt 0, then for any
event A of S with P(A) gt 0,
24
Example 0.9
During a double homicide murder trial, based on
circumstantial evidence alone, the jury becomes
15 certain that a suspect is guilty. DNA samples
recovered from the murder scene are then compared
with DNA samples extracted from the suspect.
Given the size and conditions of the recovered
samples, a forensic scientist estimates that the
probability of the sample having come from
someone other than the suspect is 10-9. With this
new information, how certain should the jury be
of the suspects guilt? Sol
Ans0.9999999943
25
Example 0.10
A box contains 7 red and 13 blue balls. 2 balls
are selected at random and are discarded without
their colors being seen. If a third ball is drawn
randomly and observed to be red, what is the
probability that both of the discarded balls were
blue? Sol
Ans26/57
26
Independence
Definition Two events A and B are called
independent if and only if
P(AB) P(A)P(B). If two events are not
independent, they are called dependent. If A and
B are independent, we say that A, B is an
independent set of events.
27
Independence of Three Events
Definition The events A, B and C are called
independent iff P(AB)
P(A)P(B), P(AC)
P(A)P(C), P(BC)
P(B)P(C), P(ABC)
P(A)P(B)P(C). If A, B and C are independent
events, we say that A, B, C is an independent
set of events.
28
Independence of n Events
Definition The set of events A1, A2, , An are
called independent if for every subset

29
Example 0.11
In the experiment of tossing a fair coin
twice, Let A and B be the events of getting heads
on the first and second tosses, respectively. Are
A and B independent? Sol
30
Example 0.12
An urn contains 5 red and 7 blue balls.
Suppose that 2 balls are selected at random and
with replacement. Let A and B be the events that
the first and the second balls are red,
respectively. Are A and B independent? Pf
31
Example 0.13
In the experiment of selecting a random
number from the set of natural numbers
1,2,3,,100, let A, B, and C denote the events
that they are divisible by 2, 3, and 5,
respectively. Are A, B, and C are
independent? Sol
32
Example 0.14
A spinner is mounted on a wheel. Arcs A, B,
and C, of equal length, are marked off on the
wheels perimeter. In a game of chance, the
spinner is flicked, and depending on whether it
stops on A, B, or C, the player wins 1, 2, or 3
points, respectively. A player plays this game
twice. Let Ehe wins 1 point in the first
game and any number of points in the
second. Fhe wins a total of 3 points in
both game, Ghe wins a total of 4 points in
both games. Are E, F, and G are independent?
Sol
33
Theorem 0.7
If A and B are independent, then A and Bc are
independent as well. Pf Corollary If A and B
are independent, then Ac and
Bc are independent as well. Pf
34
Example 0.15
Dennis arrives at his office every day at a
random time between 800 Am and 900 AM. Let
A he arrives 815 AM and 845 Am. B he
arrives between 830 Am and 900 AM, C he
arrives either between 815 AM and 830
AM or between 845 AM and 900 AM. Are A,
B, and C are independent? Sol
35
0.3 Random Variables
Definition Let S be the sample space of an
experiment. A real-valued function XS?R is
called a random variable of the experiment if,
for each interval I? R, sX(s) ? I is an
event. ExampleIf in rolling two fair dice, X is
the sum, then X can only assume the values 2,3,4,
, 12 with the following probabilities P(X
2) P((1,1)) 1/36, P(X 3) P((1,2),
(2,1)) 2/36 P(X 4) P((1,3), (2,2),
(3,1)) 3/36 and, similarly
36
Another Definition
Definition A random variable X is a process
of assigning a number X(s) to every outcome s of
an experiment. The resulting function must
satisfy the following two conditions but is
otherwise arbitrary 1. The set X? x is an
event for every x. 2. The probabilities of the
events X ? and X -? equal 0 PX
? 0, PX -? 0. P.S. X(s) is a
real-valued function XS?R
37
Example 0.16

• Suppose that 3 cards are drawn from an ordinary
  deck of 52 cards, one by one, at random and with
  replacement.
• Let X be the number of spades drawn then X
  is a random variable.
• If an outcome of spades is denoted by s, and
  other outcomes are represented by t, then X is a
  real-valued function defined on the sample space
• S(s,s,s), (t,s,s), (s,t,s), (s,s,t),
  (t,t,s), (t,s,t), (s,t,t), (t,t,t)
• ? X(s,s,s) 3, X(t,s,s) X(s,s,t)
  X(s,t,s) 2,
• X(t,t,s)
  X(s,t,t) X(t,s,t) 1,
• X(t,t,t) 0,

38
Example 0.16 (Contd)

• What are the probabilities of X 0, 1, 2, 3 ?
• Sol

39
Example 0.17
A bus stops at a station every day at some
random time between 1100 AM and 1130 AM. If X
is the actual arrival time of the bus, X is a
random variable. It is defined on the sample
space Then
40
Example 0.18
The diameter of the metal disk manufactured by a
factory is a random number between 4 and 4.5 .
What is the probability that the area of such a
flat disk chosen at random is at least 4.41? ?
Sol
Ans3/5
41
Example 0.19
A random number is selected from the interval
(0, ?/2). What is the probability that its sine
is greater than its cosine? Sol
Ans1/2
42
Distribution Functions
Definition If X is a random variable, then
the function F defined on (??, ?) by F(t)P(X ?
t) is called the distribution function or
cumulative distribution function (CDF) of
X. Properties 1. F is nondecreasing. 2.
lim t? ? F(t) 1. 3. lim t? ?? F(t) 0.
4. F is right continuous. F(t)F(t)
43
Properties of CDF

• P(X gt a) 1 ? F(a)
• P(a lt X ? b) F(b) ? F(a)
• P(X lt a) lim n?? F(a ? 1/n)
• P(X ? a) 1 ? F(a ?)
• P(X a) F(a) ? F(a ?)

44
Example 0.20
The distribution function of a random variable X
is given by Compute the following
quantities (a) P(X lt 2) (b)
P(X 2) (c) P(1? X lt 3) (d) P(X
lt 3/2) (e) P(X 5/2)
(f) P(2ltX ? 7) Sol
45
Example 0.21

• The sales of a convenience store on a
  randomly selected day are X thousand dollars,
  where X is a random variable with a distribution
  function of the following form
• Suppose that this convenience stores total
  sales on any given day are less than 2000.
• Find the value of k.
• Let A and B be the events that tomorrow the
  stores total sales are between 500 and 1500
  dollars, and over 1000 dollars, respectively.
  Find P(A) and P(B).
• Are A and B independent events?

46
Probability Mass Function (PMF)

• Definition

The probability mass function p of a random
variable X whose set of possible values is x1,
x2, x3, is a function from R to R that
satisfies the following properties. (a) p(x) 0
if x ? x1, x2, x3, (b) p(xi) P(X xi) and
hence p(xi) ? 0 (i 1, 2, 3, ) (c)
Also called probability density function.
47
Example 0.22
Can a function of the form be a
probability mass function ? Sol
48
Expectations of Discrete R.V.

• Definition

The expected value of a discrete random variable
X with the set of possible values A and
probability mass function p(x) is defined by We
say that E(X) exists if this sum converges
absolutely. The expected value of a random
variable X is also called the mathematical
expectation, or mean, or simply
expectation of X.
49
Example 0.23
We flip a fair coin twice and let X be the number
of heads obtained. What is the expected value of
X ? Sol
AnsEX1
50
Example 0.24
In the lottery of a certain state, players pick
six different integers between 1 and 49, the
order of selection being irrelevant. The lottery
commission then selects six of these numbers at
random as the winning numbers. A player wins the
grand prize of 1,200,000 if all six numbers that
he has selected match the winning numbers. He
wins the 2nd and 3rd prizes of 800 and 35,
respectively. What is the expected value of the
amount a player wins in one game?
51
Example 0.24 (Contd)
Sol
Ans0.13
52
Theorem 0.8
If X is a constant random variable, that is, if
P(X c) 1 for a constant c, then E(X) c.
Pf
53
Theorem 0.9
Let X be a discrete random variable with set of
possible values A and probability mass function
p(x), and let g be a real-valued function. Then
g(X) is a random variable with Pf
54
Corollary
Let X be a discrete random variable g1, g2, ,
gn be real-valued functions, and let ?1, ? 2, ,
? n be real numbers. Then Pf
55
Example 0.25
The probability mass function of a discrete
random variable X is given by What is the
expected value of X(6 ?X) ? Sol
Ans7
56
Variances and Moments of Discrete R.V.
Definition Let X be a discrete random variable
with a set of possible values A and probability
mass function p(x), and E(X) ?. Then Var(X) and
?X, called the variance and the standard
deviation of X, respectively, are defined by
57
Example 0.26
Two gamesBolita and Keno. To play
Bolita, you buy a ticket for 1, draws a ball at
random from a box of 100 balls numbered 1 to 100.
If the ball draw matches the number on your
ticket, you win 75 otherwise, you lose. To
play Keno, you bet 1 on a single number that has
a 25 chance to win. If you win, they will return
you dollar plus two dollars more other, they
keep the dollar. Let B and K be the amounts
that you gain in one play of Bolita and Keno,
respectively. Find the means and variances for B
and K.
58
Example 0.26 (Contd)
Sol
Ans E(B) ?0.25, E(K) ?0.25 Var(B)
55.69, Var(K) 1.6875
59
Theorem 0.10
Pf
60
Example 0.27
What is the variance of the random variable
X, the outcome of rolling a fair die? Sol
Ans 35/12
61
Theorem 0.11
Let X be a discrete random variable with the
set of possible values A and mean ?. Then Var(X)
0 if and only if X is a constant with
probability 1. Pf
62
Theorem 0.12
Let X be a discrete random variable then for
constants a and b we have that Pf
63
Example 0.28
Suppose that, for a discrete random variable
X, E(X) 2 and EX(X ? 4) 5. Find the
variance and the standard deviation of ?4X 12.
Sol
Ans Var(?4X12) 144
64
Concentration
Definition Let X and Y be two random variables
and ? be a given point. If for all t gt 0,
Then we say that X is more concentrated about ?
than is Y. Theorem 0.13 Suppose that X and Y
are two random variables with E(X) E(Y) ?. If
X is more concentrated about ? than is Y, then
Var(X) ? Var(Y) .
65
Moments
Definition Eg(X)
Definition E(Xn) The nth
moment of X E(Xr) The
rth absolute moment of X E(X ? c)
The first moment of X about c E(X ?
c)n The nth moment of X about c
E(X ? ?)n The nth central moment of
X about ? EX(X?1)...(X?k) The kth
factorial moment of X Remark 4.2The existence
of higher moments implies the existence of
lower moments.
66
0.4 Continuous Random Variables

• Definition
• Let X be a random variable. Suppose that there
  exists a nonnegative real-value function
  fR?0,?) such that for any subset of real
  numbers A that can be considered from intervals
  by a countable number of set operations.
• Then X is called absolutely continuous or,
  for simplicity, continuous. The function f is
  called a probability density function, or simply
  the density function of X.

67
Properties of Density Function
1. 2. 3. If f is continuous, then 4. For
real numbers a ? b, 5.
68
Example 0.29

• Experience has shown that while walking in a
  certain park,
• the time X, in minutes, between seeing two people
  smoking
• has a density function of the form
• Calculate the value of ?.
• Find the probability distribution function of X.
• What is the probability that Jeff, who has just
  seen a person smoking, will see another person
  smoking in 2 to 5 minutes? In at least 7 minutes?
  
• Sol

69
Example 0.30

• (a) Sketch the graph of the function
• and show that it is the probability
  density
• function of a random variable X.
• (b) Find F, the distribution function of X,
  and show that it
• is continuous.
• (c) Sketch the graph of F.
• Sol

70
Density Function of a Function of a R.V.
QIf the density function of a random variable
X is known as f, how to obtain the density
function of the random variable Yh(X)
? ATwo methods are frequently used. 1.
Method of distribution functions. 2.
Method of Transformations.
71
Example 0.31
Let X be a continuous random variable with the
probability density function Find the
distribution and the density functions of Y X
2. Sol
72
Example 0.32
Let X, be a continuous random variable with
distribution function F and probability density
function f. In terms of f, find the distribution
and the density functions of Y X 3. Sol
73
Example 0.33
The error of a measurement has the density
function Find the distribution and the density
functions of the magnitude of the error. Sol
74
Theorem 0.13Method of Transformations
Let X be a continuous random variable with
density function fX and the set of possible
values A. For the invertible function h A?R,
let Yh(X) be a random variable with the set of
possible values B h(A) h(a)a ?A. Suppose
that the inverse of yh(x) is the function
xh?1(y), which is differentiable for all values
of y ? B. Then fY, the density function of Y, is
given by Pf
75
Example 0.34
Let X be a random variable with the density
function Using the method of transformations,
find the probability density function of
. Sol
76
Example 0.35
Let X be a continuous random variable with the
probability density function Using the method
of transformations, find the probability density
function of Y 1?3X 2. Sol
77
Expectations and Variances

• Definition

If X is a continuous random variable with
probability density function f, the expected
value of X is defined by The expected value of
X is also called the mean, or mathematical
expectation, or simply the expectation of X, and
as in the discrete case, sometimes it is denoted
by EX, EX, ?, or ?X.
78
Example 0.36
In a group of adult males, the difference between
the uric acid value and 6, the standard value, is
a random variable X with the following
probability density function Calculate the
mean of these differences for the group. Sol
Ans 283/120
79
Example 0.37

• A random variable X with density function
• is called a Cauchy random variable.
• Find c.
• Show that E(X) does not exist.
• Sol

80
Theorem 0.14
For any continuous random variable X with
probability distribution function F and density
function f, Pf
81
Theorem 0.15
Let X be a continuous random variable with
probability density function f(x), then for any
function hR?R, Pf
82
Corollary
Let X be a continuous random variable with
probability density function f(x). Let h1, h2,
, hn be real-valued functions, and ?1, ?2, ,
?n be real numbers. Then Pf
83
Example 0.38
A point X is selected from the interval (0, ?/4)
randomly. Calculate E(cos2X) and E(cos2X). Sol
84
Variances of Continuous R.V.
Definition If X is a continuous random variable
with E(X)?, then Var(X) and ?X, called the
variance and standard deviation of X,
respectively, are defined by Properties
85
Example 0.39

• The time elapsed, in minutes, between the
  placement of an
• order of pizza ans its delivery is random with
  the density
• function
• Determine the mean and standard deviation of the
  time it takes for the pizza shop to deliver
  pizza.
• Sol

86
Example 0.39 (Contd)
(b) Suppose that it takes 12 minutes for the
pizza shop to bake pizza. Determine the mean and
standard deviation of the time it takes for the
delivery person to deliver pizza. Sol
87
0.6 Joint Distributions of Two R.V.s
Definition Let X and Y be two discrete R.V.s
defined on the sample space. Let the sets of
possible values of X and Y be A and B,
respectively. The function
pXY(x,y)P(Xx, Yy) is called the joint
probability mass function of X and Y. And
88
Marginal Probability Mass Functions
Definition Let X and Y have joint mass
function pXY(x,y) on the sets of possible values
of X and Y be A and B. Then the functions,
are called, respectively, the marginal
probability mass functions of X and Y.
89
Joint Probability Density Functions
Definition Two R.V.s X and Y, defined on the
same sample space, have a continuous joint
distribution if there exists a nonnegative
function of two variables, fXY(x,y) on R?R, such
that for any region R in the xy-plane that can be
formed from rectangles by a countable number of
set operations, fXY(x,y) is called the
joint probability density function of X and Y.
And
90
Marginal Probability Density Functions
Definition Let X and Y have joint density
function fXY(x,y) on R?R . Then the functions,
are called, respectively, the marginal
probability density functions of X and Y.
91
Joint Probability Distribution Functions
Definition Two R.V.s X and Y, continuous or
discrete, the joint probability distribution
function, or joint cumulative probability
distribution function, or simply the joint
distribution of X and Y, is defined by
92
Marginal Probability Distribution Functions
Definition Two R.V.s X and Y, continuous or
discrete, the marginal probability distribution
function of X and Y, is defined by
93
Mean of Joint PMF or PDF
Definition The mean of h(X,Y) defined on two
discrete R.V.s X and Y, is defined by The
mean of h(X,Y) defined on two discrete R.V.s X
and Y, is defined by
94
Independent of Two R.V.s
Definition Two R.V.s X and Y are independent if
and only if
95
O.7 Some Famous Random Variables
Discrete R.V. Bernoulli R.V. A random
variable is called Bernoulli with parameter p if
its probability mass function is given by
Mean and Variance
96
Discrete R.V.Binomial R.V.
Let X be a binomial random variable with
parameters n and p. Then p(x), the probability
mass function of X, is Definition The
function p(x) given by the above is called the
binomial probability mass function with
parameters (n, p).
97
Discrete R.V.Poisson R.V.
Definition A discrete random variable X with
possible values 0, 1, 2, 3, is called Poisson
with parameter ?, ? gt 0, if Properties
98
Discrete R.V.Geometric R.V.

• Definition

The probability mass function p(x), is called
geometric. Memoryless Property
99
Continuous R.V.Uniform R.V.
Definition A random variable X is said to be
uniformly distributed over an interval (a,b) if
its probability density function is given by
The probability distribution function is
given by
100
Continuous R.V.Normal R.V.
Definition A random variable X, is called normal,
with parameters ? and ?, if its density
distribution is given by We write X N(?,? 2),
it was given by Gauss in 1809. So normal
distribution is sometime called the
Gaussian distribution.
101
Continuous R.V.Exponential R.V.

• Definition

A continuous random variable X is called
exponential with parameter ? gt 0 if its density
function f, is given by The exponential
distribution function