Stochastic Process - Electronics & Telecommunication Engineering

1
Digital Communication Unit-III Stochastic Process
Ashok N Shinde ashok.shinde0349_at_gmail.com Interna
tional Institute of Information Technology
Hinjawadi Pune
July 26, 2017
1/28
2
Introduction to Stochastic Process
Signals Deterministic can be reproduced exactly
with repeated measurements.
2/28
3
Introduction to Stochastic Process
Signals Deterministic can be reproduced exactly
with repeated measurements. s(t) Acos(2pfct
?)
2/28
4
Introduction to Stochastic Process
Signals Deterministic can be reproduced exactly
with repeated measurements. s(t) Acos(2pfct
?) where A,fc and ? are constant.
2/28
5
Introduction to Stochastic Process
Signals Deterministic can be reproduced exactly
with repeated measurements. s(t) Acos(2pfct
?) where A,fc and ? are constant. Random signal
that is not repeatable in a predictable manner.
2/28
6
Introduction to Stochastic Process
Signals Deterministic can be reproduced exactly
with repeated measurements. s(t) Acos(2pfct
?) where A,fc and ? are constant. Random signal
that is not repeatable in a predictable
manner. s(t) Acos(2pfct ?)
2/28
7
Introduction to Stochastic Process
Signals Deterministic can be reproduced exactly
with repeated measurements. s(t) Acos(2pfct
?) where A,fc and ? are constant. Random signal
that is not repeatable in a predictable
manner. s(t) Acos(2pfct ?) where A,fc and ?
are variable.
2/28
8
Introduction to Stochastic Process
Signals Deterministic can be reproduced exactly
with repeated measurements. s(t) Acos(2pfct
?) where A,fc and ? are constant. Random signal
that is not repeatable in a predictable
manner. s(t) Acos(2pfct ?) where A,fc and ?
are variable. Unwanted signals Noise
2/28
9
Stochastic Process
Definition A stochastic process is a set of
random variables indexed in time. Mathematically
3/28
10
Stochastic Process
Definition A stochastic process is a set of
random variables indexed in time. Mathematically
Mathematically relationship between probability
theory and stochastic processes is as follows-
3/28
11
Stochastic Process
Definition A stochastic process is a set of
random variables indexed in time. Mathematically
Mathematically relationship between probability
theory and stochastic processes is as
follows- Sample point ?Sample Function
3/28
12
Stochastic Process
Definition A stochastic process is a set of
random variables indexed in time. Mathematically
Mathematically relationship between probability
theory and stochastic processes is as
follows- Sample point ?Sample Function Sample
space ?Ensemble
3/28
13
Stochastic Process
Definition A stochastic process is a set of
random variables indexed in time. Mathematically
Mathematically relationship between probability
theory and stochastic processes is as
follows- Sample point ?Sample Function Sample
space ?Ensemble Random Variable? Random Process
3/28
14
Stochastic Process
Definition A stochastic process is a set of
random variables indexed in time. Mathematically
Mathematically relationship between probability
theory and stochastic processes is as
follows- Sample point ?Sample Function Sample
space ?Ensemble Random Variable? Random Process
Sample point s is function of time
3/28
15
Stochastic Process
Definition A stochastic process is a set of
random variables indexed in time. Mathematically
Mathematically relationship between probability
theory and stochastic processes is as
follows- Sample point ?Sample Function Sample
space ?Ensemble
Random Variable? Random Process
Sample point s is function of time X(s, t), -T
t T
3/28
16
Stochastic Process
Definition A stochastic process is a set of
random variables indexed in time. Mathematically
Mathematically relationship between probability
theory and stochastic processes is as
follows- Sample point ?Sample Function Sample
space ?Ensemble
Random Variable? Random Process
Sample point s is function of time X(s, t), -T
t T Sample function denoted as
3/28
17
Stochastic Process
Definition A stochastic process is a set of
random variables indexed in time. Mathematically
Mathematically relationship between probability
theory and stochastic processes is as
follows- Sample point ?Sample Function Sample
space ?Ensemble
Random Variable? Random Process
Sample point s is function of time X(s, t), -T
t T Sample function denoted as xj (t) X(t,
sj ), -T t T
3/28
18
Stochastic Process
A random process is defined as the
ensemble(collection) of time functions together
with a probability rule
4/28
19
Stochastic Process
A random process is defined as the
ensemble(collection) of time functions together
with a probability rule xj (t), j 1, 2, . . .
, n
4/28
20
Stochastic Process
Stochastic Process Each sample point in S is
associated with a sample function x(t)
5/28
21
Stochastic Process
Stochastic Process Each sample point in S is
associated with a sample function x(t) X(t, s)
is a random process
5/28
22
Stochastic Process
Stochastic Process Each sample point in S is
associated with a sample function x(t) X(t, s) is
a random process is an ensemble of all time
functions together with a probability rule
5/28
23
Stochastic Process
Stochastic Process Each sample point in S is
associated with a sample function x(t) X(t, s) is
a random process is an ensemble of all time
functions together with a probability rule X(tk ,
sj ) is a realization or sample function of the
random process x1(tk ), x2(tk ), . . . , xn(tk )
X(tk , s1), X(tk , s2), . . . , X(tk , sn)
5/28
24
Stochastic Process
Stochastic Process Each sample point in S is
associated with a sample function x(t) X(t, s) is
a random process is an ensemble of all time
functions together with a probability rule X(tk ,
sj ) is a realization or sample function of the
random process x1(tk ), x2(tk ), . . . , xn(tk )
X(tk , s1), X(tk , s2), . . . , X(tk , sn)
Probability rules assign probability to any
meaningful event associated with an observation
An observation is a sample function of the
random process
5/28
25
Stochastic Process
Stochastic Process Each sample point in S is
associated with a sample function x(t) X(t, s) is
a random process is an ensemble of all time
functions together with a probability rule X(tk ,
sj ) is a realization or sample function of the
random process x1(tk ), x2(tk ), . . . , xn(tk )
X(tk , s1), X(tk , s2), . . . , X(tk , sn)
Probability rules assign probability to any
meaningful event associated with an observation
An observation is a sample function of the
random process A stochastic process X(t, s) is
represented by time indexed ensemble (family) of
random variables X(t, s)
5/28
26
Stochastic Process
Stochastic Process Each sample point in S is
associated with a sample function x(t) X(t, s) is
a random process is an ensemble of all time
functions together with a probability rule X(tk ,
sj ) is a realization or sample function of the
random process x1(tk ), x2(tk ), . . . , xn(tk )
X(tk , s1), X(tk , s2), . . . , X(tk , sn)
Probability rules assign probability to any
meaningful event associated with an observation
An observation is a sample function of the
random process A stochastic process X(t, s) is
represented by time indexed ensemble (family) of
random variables X(t, s) Represented compactly
by X(t)
5/28
27
Stochastic Process
Stochastic Process Each sample point in S is
associated with a sample function x(t) X(t, s) is
a random process is an ensemble of all time
functions together with a probability rule X(tk ,
sj ) is a realization or sample function of the
random process x1(tk ), x2(tk ), . . . , xn(tk )
X(tk , s1), X(tk , s2), . . . , X(tk , sn)
Probability rules assign probability to any
meaningful event associated with an observation
An observation is a sample function of the
random process A stochastic process X(t, s) is
represented by time indexed ensemble (family) of
random variables X(t, s) Represented compactly
by X(t) A stochastic process X(t) is an
ensemble of time functions, which, together with
a probability rule, assigns a probability to any
meaningful event associated with an observation
of one of the sample functions of the stochastic
process.
5/28
28
Stochastic Process Stationary Vs Non-Stationary
Process
Stationary Process
6/28
29
Stochastic Process Stationary Vs Non-Stationary
Process
Stationary Process If a process is divided into
a number of time intervals exhibiting same
statistical properties, is called as Stationary.
6/28
30
Stochastic Process
Stationary Vs Non-Stationary Process
Stationary Process If a process is divided into
a number of time intervals exhibiting same
statistical properties, is called as
Stationary. It is arises from a stable phenomenon
that has evolved into a steady-state mode of
behavior.
6/28
31
Stochastic Process
Stationary Vs Non-Stationary Process
Stationary Process If a process is divided into
a number of time intervals exhibiting same
statistical properties, is called as
Stationary. It is arises from a stable phenomenon
that has evolved into a steady-state mode of
behavior. Non-Stationary Process
6/28
32
Stochastic Process
Stationary Vs Non-Stationary Process
Stationary Process If a process is divided into
a number of time intervals exhibiting same
statistical properties, is called as
Stationary. It is arises from a stable phenomenon
that has evolved into a steady-state mode of
behavior. Non-Stationary Process If a process is
divided into a number of time intervals
exhibiting different statistical properties, is
called as Non-Stationary.
6/28
33
Stochastic Process
Stationary Vs Non-Stationary Process
Stationary Process If a process is divided into
a number of time intervals exhibiting same
statistical properties, is called as
Stationary. It is arises from a stable phenomenon
that has evolved into a steady-state mode of
behavior. Non-Stationary Process If a process is
divided into a number of time intervals
exhibiting different statistical properties, is
called as Non-Stationary. It is arises from an
unstable phenomenon.
6/28
34
Classes of Stochastic Process Strictly
Stationary and Weakly Stationary
The Stochastic process X(t) initiated at t -8
is said to be Stationary in the strict sense, or
strictly stationary if,
7/28
35
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
The Stochastic process X(t) initiated at t -8
is said to be Stationary in the strict sense, or
strictly stationary if, FX(t1 t ),X(t2 t
),...,X(tk t )(x1, x2, . . . , xk ) FX(t1
),X(t2 ),...,X(tk )(x1, x2, . . . , xk ) Where,
7/28
36
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
The Stochastic process X(t) initiated at t -8
is said to be Stationary in the strict sense, or
strictly stationary if, FX(t1 t ),X(t2 t
),...,X(tk t )(x1, x2, . . . , xk ) FX(t1
),X(t2 ),...,X(tk )(x1, x2, . . . , xk )
Where, X(t1), X(t2), . . . , X(tk ) denotes RVs
obtained by sampling process X(t) at t1, t2, . .
. , tk respectively.
7/28
37
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
The Stochastic process X(t) initiated at t -8
is said to be
Stationary in the strict sense, or strictly
stationary if, FX(t1 t ),X(t2 t ),...,X(tk t
)(x1, x2, . . . , xk ) FX(t1 ),X(t2 ),...,X(tk
)(x1, x2, . . . , xk ) Where, X(t1), X(t2), . . .
, X(tk ) denotes RVs obtained by sampling
process X(t) at t1, t2, . . . , tk
respectively. FX(t1 ),X(t2 ),...,X(tk )(x1, x2, .
. . , xk ) denotes Joint distribution function
of RVs.
7/28
38
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
The Stochastic process X(t) initiated at t -8
is said to be
Stationary in the strict sense, or strictly
stationary if, FX(t1 t ),X(t2 t ),...,X(tk t
)(x1, x2, . . . , xk ) FX(t1 ),X(t2 ),...,X(tk
)(x1, x2, . . . , xk ) Where, X(t1), X(t2), . . .
, X(tk ) denotes RVs obtained by sampling
process X(t) at t1, t2, . . . , tk
respectively. FX(t1 ),X(t2 ),...,X(tk )(x1, x2, .
. . , xk ) denotes Joint distribution function
of RVs. X(t1 t ), X(t2 t ), . . . , X(tk t
) denotes new RVs obtained by sampling process
X(t) at t1 t, t2 t, . . . , tk t
respectively. Here t is fixed time shift.
7/28
39
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
The Stochastic process X(t) initiated at t -8
is said to be
Stationary in the strict sense, or strictly
stationary if, FX(t1 t ),X(t2 t ),...,X(tk t
)(x1, x2, . . . , xk ) FX(t1 ),X(t2 ),...,X(tk
)(x1, x2, . . . , xk ) Where, X(t1), X(t2), . . .
, X(tk ) denotes RVs obtained by sampling
process X(t) at t1, t2, . . . , tk
respectively. FX(t1 ),X(t2 ),...,X(tk )(x1, x2, .
. . , xk ) denotes Joint distribution function
of RVs. X(t1 t ), X(t2 t ), . . . , X(tk t
) denotes new RVs obtained by sampling process
X(t) at t1 t, t2 t, . . . , tk t
respectively. Here t is fixed time shift. FX(t1
t ),X(t2 t ),...,X(tk t )(x1, x2, . . . , xk )
denotes Joint distribution function of new RVs.
7/28
40
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process
8/28
41
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t.
8/28
42
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t. For k 2, we have FX(t1
),X(t2 )(x1, x2) FX(0),X(t1 -t2 )(x1, x2)
for all t1 and t2. Second-order distribution
function of a strictly stationary stochastic
process depends only on the time difference
between the sampling instants and not on time
sampled.
8/28
43
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t. For k 2, we have FX(t1
),X(t2 )(x1, x2) FX(0),X(t1 -t2 )(x1, x2)
for all t1 and t2. Second-order distribution
function of a strictly stationary stochastic
process depends only on the time difference
between the sampling instants and not on time
sampled. Weakly Stationary Process
8/28
44
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t. For k 2, we have FX(t1
),X(t2 )(x1, x2) FX(0),X(t1 -t2 )(x1, x2)
for all t1 and t2. Second-order distribution
function of a strictly stationary stochastic
process depends only on the time difference
between the sampling instants and not on time
sampled. Weakly Stationary Process A stochastic
process X(t) is said to be weakly
stationary(Wide-Sense Stationary) if its
second-order moments satisfy
8/28
45
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t. For k 2, we have FX(t1
),X(t2 )(x1, x2) FX(0),X(t1 -t2 )(x1, x2)
for all t1 and t2. Second-order distribution
function of a strictly stationary stochastic
process depends only on the time difference
between the sampling instants and not on time
sampled. Weakly Stationary Process A stochastic
process X(t) is said to be weakly
stationary(Wide-Sense Stationary) if its
second-order moments satisfy The mean of the
process X(t) is constant for all time t.
8/28
46
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t. For k 2, we have FX(t1
),X(t2 )(x1, x2) FX(0),X(t1 -t2 )(x1, x2)
for all t1 and t2. Second-order distribution
function of a strictly stationary stochastic
process depends only on the time difference
between the sampling instants and not on time
sampled. Weakly Stationary Process A stochastic
process X(t) is said to be weakly
stationary(Wide-Sense Stationary) if its
second-order moments satisfy The mean of the
process X(t) is constant for all time t. The
autocorrelation function of the process X(t)
depends solely on the difference between any two
times at which the process is sampled.
8/28
47
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t. For k 2, we have FX(t1
),X(t2 )(x1, x2) FX(0),X(t1 -t2 )(x1, x2)
for all t1 and t2. Second-order distribution
function of a strictly stationary stochastic
process depends only on the time difference
between the sampling instants and not on time
sampled. Weakly Stationary Process A stochastic
process X(t) is said to be weakly
stationary(Wide-Sense Stationary) if its
second-order moments satisfy The mean of the
process X(t) is constant for all time t. The
autocorrelation function of the process X(t)
depends solely on the difference between any two
times at which the process is sampled. Summary of
Random Processes
8/28
48
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t. For k 2, we have FX(t1
),X(t2 )(x1, x2) FX(0),X(t1 -t2 )(x1, x2)
for all t1 and t2. Second-order distribution
function of a strictly stationary stochastic
process depends only on the time difference
between the sampling instants and not on time
sampled. Weakly Stationary Process A stochastic
process X(t) is said to be weakly
stationary(Wide-Sense Stationary) if its
second-order moments satisfy The mean of the
process X(t) is constant for all time t. The
autocorrelation function of the process X(t)
depends solely on the difference between any two
times at which the process is sampled. Summary of
Random Processes Wide-Sense Stationary processes
8/28
49
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t. For k 2, we have FX(t1
),X(t2 )(x1, x2) FX(0),X(t1 -t2 )(x1, x2)
for all t1 and t2. Second-order distribution
function of a strictly stationary stochastic
process depends only on the time difference
between the sampling instants and not on time
sampled. Weakly Stationary Process A stochastic
process X(t) is said to be weakly
stationary(Wide-Sense Stationary) if its
second-order moments satisfy The mean of the
process X(t) is constant for all time t. The
autocorrelation function of the process X(t)
depends solely on the difference between any two
times at which the process is sampled. Summary of
Random Processes Wide-Sense Stationary
processes Strictly Stationary Processes
8/28
50
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t. For k 2, we have FX(t1
),X(t2 )(x1, x2) FX(0),X(t1 -t2 )(x1, x2)
for all t1 and t2. Second-order distribution
function of a strictly stationary stochastic
process depends only on the time difference
between the sampling instants and not on time
sampled. Weakly Stationary Process A stochastic
process X(t) is said to be weakly
stationary(Wide-Sense Stationary) if its
second-order moments satisfy The mean of the
process X(t) is constant for all time t. The
autocorrelation function of the process X(t)
depends solely on the difference between any two
times at which the process is sampled. Summary of
Random Processes Wide-Sense Stationary
processes Strictly Stationary Processes Ergodic
Processes
8/28
51
Classes of Stochastic Process
Strictly Stationary and Weakly Stationary
Properties of Strictly Stationary Process For k
1, we have FX(t)(x) FX(tt )(x) FX (x) for
all t and t . First-order distribution function
of a strictly stationary stochastic process is
independent of time t. For k 2, we have FX(t1
),X(t2 )(x1, x2) FX(0),X(t1 -t2 )(x1, x2)
for all t1 and t2. Second-order distribution
function of a strictly stationary stochastic
process depends only on the time difference
between the sampling instants and not on time
sampled. Weakly Stationary Process A stochastic
process X(t) is said to be weakly
stationary(Wide-Sense Stationary) if its
second-order moments satisfy The mean of the
process X(t) is constant for all time t. The
autocorrelation function of the process X(t)
depends solely on the difference between any two
times at which the process is sampled. Summary of
Random Processes Wide-Sense Stationary
processes Strictly Stationary Processes Ergodic
Processes Non-Stationary processes
8/28
52
Mean, Correlation, and Covariance Functions of WSP
Mean
9/28
53
Mean, Correlation, and Covariance Functions of WSP
Mean Mean of real-valued stochastic process
X(t), is expectation of the random variable
obtained by sampling the process at some time t,
as shown by
9/28
54
Mean, Correlation, and Covariance Functions of WSP
Mean Mean of real-valued stochastic process
X(t), is expectation of the random variable
obtained by sampling the process at some time t,
as shown by µX (t) EX(t)
9/28
55
Mean, Correlation, and Covariance Functions of WSP
Mean Mean of real-valued stochastic process
X(t), is expectation of the random variable
obtained by sampling the process at some time t,
as shown by
µX (t) EX(t)
,
8
µ (t) xf
(x)dx
X
X(t)
-8
9/28
56
Mean, Correlation, and Covariance Functions of WSP
Mean Mean of real-valued stochastic process
X(t), is expectation of the random variable
obtained by sampling the process at some time t,
as shown by
µX (t) EX(t)
,
8
µ (t) xf
(x)dx
X
X(t)
-8
where f
is the first-order probability density function
of the
X(t)(x)
process X(t).
9/28
57
Mean, Correlation, and Covariance Functions of WSP
Mean Mean of real-valued stochastic process
X(t), is expectation of the random variable
obtained by sampling the process at some time t,
as shown by
µX (t) EX(t)
,
8
µ (t) xf
(x)dx
X
X(t)
-8
where f
is the first-order probability density function
of the
X(t)(x)
process X(t). µX (t) µX for weakly stationary
process
9/28
58
Mean, Correlation, and Covariance Functions of WSP
Mean Mean of real-valued stochastic process
X(t), is expectation of the random variable
obtained by sampling the process at some time t,
as shown by
µX (t) EX(t)
,
8
µ (t) xf
(x)dx
X
X(t)
-8
where f
is the first-order probability density function
of the
X(t)(x)
process X(t). µX (t) µX for weakly stationary
process Correlation
9/28
59
Mean, Correlation, and Covariance Functions of WSP
Mean Mean of real-valued stochastic process
X(t), is expectation of the random variable
obtained by sampling the process at some time t,
as shown by
µX (t) EX(t)
,
8
µ (t) xf
(x)dx
X
X(t)
-8
where f
is the first-order probability density function
of the
X(t)(x)
process X(t). µX (t) µX for weakly stationary
process Correlation Autocorrelation function of
the stochastic process X(t) is product of two
random variables, X(t1) and X(t2)
9/28
60
Mean, Correlation, and Covariance Functions of WSP
Mean Mean of real-valued stochastic process
X(t), is expectation of the random variable
obtained by sampling the process at some time t,
as shown by
µX (t) EX(t)
,
8
µ (t) xf
(x)dx
X
X(t)
-8
where f
is the first-order probability density function
of the
X(t)(x)
process X(t). µX (t) µX for weakly stationary
process Correlation Autocorrelation function of
the stochastic process X(t) is product of two
random variables, X(t1) and X(t2) MXX (t1, t2)
EX(t1)X(t2)
9/28
61
Mean, Correlation, and Covariance Functions of WSP
Mean Mean of real-valued stochastic process
X(t), is expectation of the random variable
obtained by sampling the process at some time t,
as shown by
µX (t) EX(t)
,
8
µ (t) xf
(x)dx
X
X(t)
-8
where f
is the first-order probability density function
of the
X(t)(x)
process X(t). µX (t) µX for weakly stationary
process Correlation Autocorrelation function of
the stochastic process X(t) is product of two
random variables, X(t1) and X(t2)
MXX (t1, t2) EX(t1)X(t2)
, ,
8 8
M (t , t )
x x f (x , x )dx dx
XX 1 2
1 2 1 2
1 2 X(t ),X(x )
1 2
where f -8 -8 is joint probability density
function of the
X(t1 ),X(x2 )(x1, x2)
process X(t) sampled at times t1 and t2. MXX (t1,
t2) is a second-order moment. It is depend only
on time difference t1 - t2 so that the process
X(t) satisfies the second condition of weak
stationarity and reduces to.
9/28
62
Mean, Correlation, and Covariance Functions of WSP
Mean Mean of real-valued stochastic process
X(t), is expectation of the random variable
obtained by sampling the process at some time t,
as shown by
µX (t) EX(t)
,
8
µ (t) xf
(x)dx
X
X(t)
-8
where f
is the first-order probability density function
of the
X(t)(x)
process X(t). µX (t) µX for weakly stationary
process Correlation Autocorrelation function of
the stochastic process X(t) is product of two
random variables, X(t1) and X(t2)
MXX (t1, t2) EX(t1)X(t2)
, ,
8 8
M (t , t )
x x f (x , x )dx dx
XX 1 2
1 2 1 2
1 2 X(t ),X(x )
1 2
where f -8 -8 is joint probability density
function of the
X(t1 ),X(x2 )(x1, x2)
process X(t) sampled at times t1 and t2. MXX (t1,
t2) is a second-order moment. It is depend only
on time difference t1 - t2 so that the process
X(t) satisfies the second condition of weak
stationarity and reduces to. MXX (t1, t2)
EX(t1)X(t2) RXX (t2 - t1)
9/28
63
Mean, Correlation, and Covariance Functions of
WSP Properties of Autocorrelation Function
10/28
64
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as
10/28
65
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t)
10/28
66
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2
10/28
67
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2 Properties
10/28
68
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2 Properties RXX (0)
ESX2(t)S(Mean-Square Value)
10/28
69
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2
Properties
S S
2
R (0) E X (t) (Mean-Square Value)
XX
RXX (t ) RXX (-t ) also RXX (t ) EX(t - t
)X(t) (Symmetry)
10/28
70
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2
Properties
S S
2
R (0) E X (t) (Mean-Square Value)
XX
RXX (t ) RXX (-t ) also RXX (t ) EX(t - t
)X(t) (Symmetry) RXX (t ) RXX (0) (Bound)
10/28
71
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2
Properties
S S
2
R (0) E X (t) (Mean-Square Value)
XX
RXX (t ) RXX (-t ) also RXX (t ) EX(t - t
)X(t) (Symmetry) RXX (t ) RXX (0) (Bound)
R (t )
? (t )
XX RXX (0)
(Normalization -1, 1)
XX
10/28
72
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2
Properties
S S
2
R (0) E X (t) (Mean-Square Value)
XX
RXX (t ) RXX (-t ) also RXX (t ) EX(t - t
)X(t) (Symmetry) RXX (t ) RXX (0) (Bound)
R (t )
? (t )
XX RXX (0)
(Normalization -1, 1)
XX Covariance
10/28
73
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2
Properties
S S
2
R (0) E X (t) (Mean-Square Value)
XX
RXX (t ) RXX (-t ) also RXX (t ) EX(t - t
)X(t) (Symmetry) RXX (t ) RXX (0) (Bound)
R (t )
? (t )
XX RXX (0)
(Normalization -1, 1)
XX Covariance
Autocovariance function of a weakly stationary
process X(t) is defined by
10/28
74
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2
Properties
S S
2
R (0) E X (t) (Mean-Square Value)
XX
RXX (t ) RXX (-t ) also RXX (t ) EX(t - t
)X(t) (Symmetry) RXX (t ) RXX (0) (Bound)
R (t )
? (t )
XX RXX (0)
(Normalization -1, 1)
XX Covariance
Autocovariance function of a weakly stationary
process X(t) is defined by CXX (t1, t2)
E(X(t1) - µx)(X(t2) - µx)
10/28
75
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2
Properties
S S
2
R (0) E X (t) (Mean-Square Value)
XX
RXX (t ) RXX (-t ) also RXX (t ) EX(t - t
)X(t) (Symmetry) RXX (t ) RXX (0) (Bound)
R (t )
? (t )
XX RXX (0)
(Normalization -1, 1)
XX Covariance
Autocovariance function of a weakly stationary
process X(t) is defined by CXX (t1, t2)
E(X(t1) - µx)(X(t2) - µx)
2
C (t , t ) R (t - t ) - µ
XX 1 2 XX 2 1
x
10/28
76
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2
Properties
S S
2
R (0) E X (t) (Mean-Square Value)
XX
RXX (t ) RXX (-t ) also RXX (t ) EX(t - t
)X(t) (Symmetry) RXX (t ) RXX (0) (Bound)
R (t )
? (t )
XX RXX (0)
(Normalization -1, 1)
XX Covariance
Autocovariance function of a weakly stationary
process X(t) is defined by CXX (t1, t2)
E(X(t1) - µx)(X(t2) - µx)
2
C (t , t ) R (t - t ) - µ
XX 1 2 XX 2 1
x
The autocovariance function of a weakly
stationary process X(t) depends only on the time
difference (t2 - t1)
10/28
77
Mean, Correlation, and Covariance Functions of WSP
Properties of Autocorrelation Function Autocorrela
tion function of a weakly stationary process X(t)
can also be represented as RXX (t ) EX(t t
)X(t) where t denotes a time shift that is,t
t2 and t t1 - t2
Properties
S S
2
R (0) E X (t) (Mean-Square Value)
XX
RXX (t ) RXX (-t ) also RXX (t ) EX(t - t
)X(t) (Symmetry) RXX (t ) RXX (0) (Bound)
R (t )
? (t )
XX RXX (0)
(Normalization -1, 1)
XX Covariance
Autocovariance function of a weakly stationary
process X(t) is defined by CXX (t1, t2)
E(X(t1) - µx)(X(t2) - µx)
2
C (t , t ) R (t - t ) - µ
XX 1 2 XX 2 1
x
The autocovariance function of a weakly
stationary process X(t) depends only on the time
difference (t2 - t1) The mean and autocorrelation
function only provide a weak description of the
distribution of the stochastic process X(t).
10/28
78
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Examples Show that the Random Process X(t)
Acos(?ct T) is wide sense stationary process,
where T is RV uniformly distributed in range (0,
2p)
Answer The ensemble consist of sinusoids of
constant amplitude A and constant frequency ?c,
but phase T is random.
11/28
79
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Examples Show that the Random Process X(t)
Acos(?ct T) is wide sense stationary process,
where T is RV uniformly distributed in range (0,
2p)
Answer The ensemble consist of sinusoids of
constant amplitude A and constant frequency ?c,
but phase T is random. The phase is equally
likely to any value in the range (0, 2p).
11/28
80
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Examples Show that the Random Process X(t)
Acos(?ct T) is wide sense stationary process,
where T is RV uniformly distributed in range (0,
2p)
Answer The ensemble consist of sinusoids of
constant amplitude A and constant frequency ?c,
but phase T is random. The phase is equally
likely to any value in the range (0, 2p). T is
RV uniformly distributed over the range (0, 2p).
11/28
81
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Examples Show that the Random Process X(t)
Acos(?ct T) is wide sense stationary process,
where T is RV uniformly distributed in range (0,
2p)
Answer The ensemble consist of sinusoids of
constant amplitude A and constant frequency ?c,
but phase T is random. The phase is equally
likely to any value in the range (0, 2p). T is
RV uniformly distributed over the range (0, 2p).
1
f (?) , 0 ? 2p
T
2p
11/28
82
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Examples Show that the Random Process X(t)
Acos(?ct T) is wide sense stationary process,
where T is RV uniformly distributed in range (0,
2p)
Answer The ensemble consist of sinusoids of
constant amplitude A and constant frequency ?c,
but phase T is random. The phase is equally
likely to any value in the range (0, 2p). T is
RV uniformly distributed over the range (0, 2p).
1
f (?) , 0 ? 2p
T
2p
0, elsewhere
11/28
83
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Because cos(?ct T) is function of RV T,
Mean of Random Process X(t) is
12/28
84
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Because cos(?ct T) is function of RV T,
Mean of Random Process X(t) is X(t) Acos(?ct
T)
12/28
85
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Because cos(?ct T) is function of RV T,
Mean of Random Process X(t) is
X(t)
Acos(?ct T) Acos(?ct T)
12/28
86
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2

• Answer
• Because cos(?ct T) is function of RV T, Mean of
  Random Process
• X(t) is
• X(t) Acos(?ct T)

Acos(?ct T)

2p
cos(? t T) cos(? t ?)f (?)d?
c c
T
0
12/28
87
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2

• Answer
• Because cos(?ct T) is function of RV T, Mean of
  Random Process
• X(t) is
• X(t) Acos(?ct T)

Acos(?ct T)

2p
cos(? t T) cos(? t ?)f (?)d?
c c
T
0

1
2p

cos(? t ?)d?
c
2p
0
12/28
88
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2

• Answer
• Because cos(?ct T) is function of RV T, Mean of
  Random Process
• X(t) is
• X(t) Acos(?ct T)

Acos(?ct T)

2p
cos(? t T) cos(? t ?)f (?)d?
c c
T
0

1
2p

cos(? t ?)d?
c
2p
0
0
12/28
89
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2

• Answer
• Because cos(?ct T) is function of RV T, Mean of
  Random Process
• X(t) is
• X(t) Acos(?ct T)

Acos(?ct T)

2p
cos(? t T) cos(? t ?)f (?)d?
c c
T
0

1
2p

cos(? t ?)d?
c
2p
0
0
X(t) 0
12/28
90
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2

• Answer
• Because cos(?ct T) is function of RV T, Mean of
  Random Process
• X(t) is
• X(t) Acos(?ct T)

Acos(?ct T)

2p
cos(? t T) cos(? t ?)f (?)d?
c c
T
0

1
2p

cos(? t ?)d?
c
2p
0
0
X(t) 0 Thus the ensemble mean of sample
function amplitude at any time instant t is zero.
12/28
91
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2

• Answer
• Because cos(?ct T) is function of RV T, Mean of
  Random Process
• X(t) is
• X(t) Acos(?ct T)

Acos(?ct T)

2p
cos(? t T) cos(? t ?)f (?)d?
c c
T
0

1
2p

cos(? t ?)d?
c
2p
0
0
X(t) 0 Thus the ensemble mean of sample
function amplitude at any time instant t is
zero. The Autocorrelation function RX X(t1, t2)
for this process can also be determined as RXX
(t1, t2) EX(t1)X(t2) A2cos(?ct1
T)cos(?ct2 T)
12/28
92
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . .
13/28
93
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . . A2cos(?ct1 T)cos(?ct2
T)
13/28
94
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . .
A2cos(?ct1 T)cos(?ct2 T)
2 S S
A
2
cos(?c(t2 - t1)) cos(?c(t2 t1) 2T)
13/28
95
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . .
A2cos(?ct1 T)cos(?ct2 T)
2 S S
A
cos(?c(t2 - t1)) cos(?c(t2 t1) 2T) 2
The term cos(?c(t2 - t1)) does not contain RV
Hence,
13/28
96
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . .
A2cos(?ct1 T)cos(?ct2 T)
2 S S cos(?c(t2 - t1)) cos(?c(t2
t1) 2T) 2
A
The term cos(?c(t2 - t1)) does not contain RV
Hence, cos(?c(t2 - t1)) cos(?c(t2 - t1))
13/28
97
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . .
A2cos(?ct1 T)cos(?ct2 T)
2 S S 2
A
cos(?c(t2 - t1)) cos(?c(t2 t1) 2T)
The term cos(?c(t2 - t1)) does not contain RV
Hence,
cos(?c(t2 - t1)) cos(?c(t2 - t1)) The term
cos(?c(t2 t1) 2T) is a function of RV T, and
it is
13/28
98
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . .
A2cos(?ct1 T)cos(?ct2 T)
2 S S cos(?c(t2 - t1)) cos(?c(t2
t1) 2T) 2
A
The term cos(?c(t2 - t1)) does not contain RV
Hence, cos(?c(t2 - t1)) cos(?c(t2 - t1)) The
term cos(?c(t2 t1) 2T) is a function of RV T,
and it is
1 2p

2p
cos(? (t t ) 2T)
cos(? (t t ) 2?)d?
c 2 1
c 2 1
0
13/28
99
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . .
A2cos(?ct1 T)cos(?ct2 T)
2 S S cos(?c(t2 - t1)) cos(?c(t2
t1) 2T) 2
A
The term cos(?c(t2 - t1)) does not contain RV
Hence, cos(?c(t2 - t1)) cos(?c(t2 - t1)) The
term cos(?c(t2 t1) 2T) is a function of RV T,
and it is

1
2p
cos(? (t t ) 2T)
cos(? (t t ) 2?)d?
c 2 1
c 2 1
2p
0
0
13/28
100
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . .
A2cos(?ct1 T)cos(?ct2 T)
2 S S cos(?c(t2 - t1)) cos(?c(t2
t1) 2T) 2
A
The term cos(?c(t2 - t1)) does not contain RV
Hence, cos(?c(t2 - t1)) cos(?c(t2 - t1)) The
term cos(?c(t2 t1) 2T) is a function of RV T,
and it is

1
2p
cos(? (t t ) 2T)
cos(? (t t ) 2?)d?
c 2 1
c 2 1
2p
0
0
A2
RXX (t1, t2)
cos(?c(t2 - t1)), 2
13/28
101
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . .
A2cos(?ct1 T)cos(?ct2 T)
2 S S cos(?c(t2 - t1)) cos(?c(t2
t1) 2T) 2
A
The term cos(?c(t2 - t1)) does not contain RV
Hence, cos(?c(t2 - t1)) cos(?c(t2 - t1)) The
term cos(?c(t2 t1) 2T) is a function of RV T,
and it is

1
2p
cos(? (t t ) 2T)
cos(? (t t ) 2?)d?
c 2 1
c 2 1
2p
0
0
A2
RXX (t1, t2)
cos(?c(t2 - t1)), 2 cos(?c(t )), t t2
- t1 2
A2
RXX (t )
13/28
102
Example Question
p
p
InSem 2014, InSem 2015 (Sin), InSem 2016 (
- T )
2
2
Answer Continue. . .
A2cos(?ct1 T)cos(?ct2 T)
2 S S cos(?c(t2 - t1)) cos(?c(t2
t1) 2T) 2
A
The term cos(?c(t2 - t1)) does not contain RV
Hence, cos(?c(t2 - t1)) cos(?c(t2 - t1)) The
term cos(?c(t2 t1) 2T) is a function of RV T,
and it is

1
2p
cos(? (t t ) 2T)
cos(? (t t ) 2?)d?
c 2 1
c 2 1
2p
0
0
A2
RXX (t1, t2)
cos(?c(t2 - t1)), 2 cos(?c(t )), t t2
- t1 2
A2
RXX (t )
A2 From X(t) 0 and RXX (t ) 2 cos(?c(t )) it
is clear that X(t) is Wide Sense Stationary
Process
13/28
103
Time Vs Ensemble Average and Ergodic Process
Ensemble Average
14/28
104
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process
14/28
105
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages
14/28
106
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
14/28
107
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
1 2T

T
Autocorrelation ? R (t, T )
x(t)x(t t )dt
XX
-T
14/28
108
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
1 2T

T
Autocorrelation ? R (t, T )
x(t)x(t t )dt
XX
-T
Ergodic Process
14/28
109
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
1 2T

T
Autocorrelation ? R (t, T )
x(t)x(t t )dt
XX
-T
Ergodic Process Ergodicity A random process is
called Ergodic if
14/28
110
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
1 2T

T
Autocorrelation ? R (t, T )
x(t)x(t t )dt
XX
-T
Ergodic Process Ergodicity A random process is
called Ergodic if it is ergodic in mean
14/28
111
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
1 2T

T
Autocorrelation ? R (t, T )
x(t)x(t t )dt
XX
-T
Ergodic Process Ergodicity A random process is
called Ergodic if it is ergodic in mean limT ?8
µX (T ) µX
14/28
112
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
1 2T

T
Autocorrelation ? R (t, T )
x(t)x(t t )dt
XX
-T
Ergodic Process Ergodicity A random process is
called Ergodic if it is ergodic in mean limT ?8
µX (T ) µX limT ?8 var µX (T ) 0
14/28
113
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
1 2T

T
Autocorrelation ? R (t, T )
x(t)x(t t )dt
XX
-T
Ergodic Process Ergodicity A random process is
called Ergodic if it is ergodic in mean limT ?8
µX (T ) µX limT ?8 var µX (T ) 0 it is
ergodic in autocorrelation
14/28
114
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
1 2T

T
Autocorrelation ? R (t, T )
x(t)x(t t )dt
XX
-T
Ergodic Process Ergodicity A random process is
called Ergodic if it is ergodic in mean limT ?8
µX (T ) µX limT ?8 var µX (T ) 0 it is
ergodic in autocorrelation limT ?8 RXX (t, T )
RXX (t )
14/28
115
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
1 2T

T
Autocorrelation ? R (t, T )
x(t)x(t t )dt
XX
-T
Ergodic Process Ergodicity A random process is
called Ergodic if it is ergodic in mean limT ?8
µX (T ) µX limT ?8 var µX (T ) 0 it is
ergodic in autocorrelation limT ?8 RXX (t, T )
RXX (t ) limT ?8 var RXX (t, T ) 0
14/28
116
Time Vs Ensemble Average and Ergodic Process
Ensemble Average Difficult to generate a number
of realizations of a random process Use time
averages

1
T
Mean ? µ (T )
x(t)dt
X
2T
-T
1 2T

T
Autocorrelation ? R (t, T )
x(t)x(t t )dt
XX
-T
Ergodic Process Ergodicity A random process is
called Ergodic if it is ergodic in mean limT ?8
µX (T ) µX limT ?8 var µX (T ) 0 it is
ergodic in autocorrelation limT ?8 RXX (t, T )
RXX (t ) limT ?8 var RXX (t, T ) 0 where
µX and RXX (t ) are the ensemble averages of the
same random process.
14/28
117
Transmission of a Weakly Stationary Process
through a LTI Filter
Linear Time Invariant Filter
15/28
118
Transmission of a Weakly Stationary Process
through a LTI Filter
Linear Time Invariant Filter Suppose that a
stochastic process X(t) is applied as input to a
linear time-invariant filter of impulse response
h(t), producing a new stochastic process Y (t)
at the filter output.
15/28
119
Transmission of a Weakly Stationary Process
through a LTI Filter
Linear Time Invariant Filter Suppose that a
stochastic process X(t) is applied as input to a
linear time-invariant filter of impulse response
h(t), producing a new stochastic process Y (t)
at the filter output.
15/28
120
Transmission of a Weakly Stationary Process
through a LTI Filter
Linear Time Invariant Filter Suppose that a
stochastic process X(t) is applied as input to a
linear time-invariant filter of impulse response
h(t), producing a new stochastic process Y (t)
at the filter output.
It is difficult to describe the probability
distribution of the output stochastic process
Y(t), even when the probability distribution of
the input stochastic process X(t) is completely
specified
15/28
121
Transmission of a Weakly Stationary Process
through a LTI Filter
Linear Time Invariant Filter Suppose that a
stochastic process X(t) is applied as input to a
linear time-invariant filter of impulse response
h(t), producing a new stochastic process Y (t)
at the filter output.
It is difficult to describe the probability
distribution of the output stochastic process
Y(t), even when the probability distribution of
the input stochastic process X(t) is completely
specified For defining the mean and
autocorrelation functions of the output
stochastic process Y (t) in terms of those of the
input X(t), assuming that X(t) is a weakly
stationary process.
15/28
122
Transmission of a Weakly Stationary Process
through a LTI Filter
Linear Time Invariant Filter Suppose that a
stochastic process X(t) is applied as input to a
linear time-invariant filter of impulse response
h(t), producing a new stochastic process Y (t)
at the filter output.
It is difficult to describe the probability
distribution of the output stochastic process
Y(t), even when the probability distribution of
the input stochastic process X(t) is completely
specified For defining the mean and
autocorrelation functions of the output
stochastic process Y (t) in terms of those of the
input X(t), assuming that X(t) is a weakly
stationary process. Transmission of a process
through a linear time-invariant filter is
governed by the convolution integral
15/28
123
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . .
16/28
124
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of the input
stochastic process X(t) as
16/28
125
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of
the input stochastic process X(t) as
,
8
Y (t) h(t )X(t - t )dt
1 1 1
-8
16/28
126
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of
the input stochastic process X(t) as
,
8
Y (t) h(t )X(t - t )dt
1 1 1
-8
where t is local time. Hence, the mean of Y (t) is
1
16/28
127
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of
the input stochastic process X(t) as
,
8
Y (t) h(t )X(t - t )dt
1 1 1
-8
where t is local time. Hence, the mean of Y (t) is
1
µY (t) E Y (t)
16/28
128
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of
the input stochastic process X(t) as
,
8
Y (t) h(t )X(t - t )dt
1 1 1
-8
where t is local time. Hence, the mean of Y (t) is
1
µY (t) E Y (t)
S S
,
8
µ (t) E
h(t )X(t - t ) dt
Y
1 1 1
-8
16/28
129
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of
the input stochastic process X(t) as
,
8
Y (t) h(t )X(t - t )dt
1 1 1
-8
where t is local time. Hence, the mean of Y (t) is
1
µY (t) E Y (t)
S S
,
8
µ (t) E
h(t )X(t - t ) dt
Y
1 1 1
-8
Provided that the expectation E X(t) is finite
for all t and the filter is stable.
16/28
130
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of
the input stochastic process X(t) as
,
8
Y (t) h(t )X(t - t )dt
1 1 1
-8
where t is local time. Hence, the mean of Y (t) is
1
µY (t) E Y (t)
S S
,
8
µ (t) E
h(t )X(t - t ) dt
Y
1 1 1
-8
Provided that the expectation E X(t) is finite
for all t and the filter is stable.
,
8
µ (t) h(t )E X(t - t ) dt
Y 1
1 1
-8
16/28
131
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of
the input stochastic process X(t) as
,
8
Y (t) h(t )X(t - t )dt
1 1 1
-8
where t is local time. Hence, the mean of Y (t) is
1
µY (t) E Y (t)
S S
,
8
µ (t) E
h(t )X(t - t ) dt
Y
1 1 1
-8
Provided that the expectation E X(t) is finite
for all t and the filter is stable.
,
8
µ (t) h(t )E X(t - t ) dt
Y 1
1 1
-8
,
8
µ (t) h(t )µ (t - t )dt
Y
1 X 1 1
-8
16/28
132
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of
the input stochastic process X(t) as
,
8
Y (t) h(t )X(t - t )dt
1 1 1
-8
where t is local time. Hence, the mean of Y (t) is
1
µY (t) E Y (t)
S S
,
8
µ (t) E
h(t )X(t - t ) dt
Y
1 1 1
-8
Provided that the expectation E X(t) is finite
for all t and the filter is stable.
,
8
µ (t) h(t )E X(t - t ) dt
Y 1
1 1
-8
,
8
µ (t) h(t )µ (t - t )dt
Y 1 X 1 1
-8
When the input stochastic process X(t) is weakly
stationary, the mean µX (t) is a constant µX
16/28
133
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of
the input stochastic process X(t) as
,
8
Y (t) h(t )X(t - t )dt
1 1 1
-8
where t is local time. Hence, the mean of Y (t) is
1
µY (t) E Y (t)
S S
,
8
µ (t) E
h(t )X(t - t ) dt
Y
1 1 1
-8
Provided that the expectation E X(t) is finite
for all t and the filter is stable.
,
8
µ (t) h(t )E X(t - t ) dt
Y 1
1 1
-8
,
8
µ (t) h(t )µ (t - t )dt
Y 1 X 1 1
-8
When the input stochastic process X(t) is weakly
stationary, the
mean µX (t) is a constant µX
,
8
µ µ
h(t )dt
Y X
1 1
-8
16/28
134
Transmission of a Weakly Stationary Process
through a LTI Filter
Continue. . . we may thus express the output
stochastic process Y (t) in terms of
the input stochastic process X(t) as
,
8
Y (t) h(t )X(t - t )dt
1 1 1
-8
where t is local time. Hence, the mean of Y (t) is
1
µY (t) E Y (t)
S S
,
8
µ (t) E
h(t )X(t - t ) dt