Random Processes and LSI Systems

1
Random Processes and LSI Systems
What happedns when a random signal is processed
by an LSI system? This is illustrated below,
where x(n) and y(n) are random signals, and h(n)
is a deterministic (i.e., nonrandom) LSI system.
x(n)
y(n)
h(n)
The input, x(n) is a random signal, so y(n) is,
too. Random in, random out.
2
Random Processes and LSI Systems
x(n)
y(n)
h(n)
The LSI system, h(n) does exactly the same thing
as it would if h(n) were deterministic, so
or, in the frequency domain,
3
Random Processes and LSI Systems
We can take the square of the magnitude,
Recall (from the previous lecture) that Sxx(w),
the spectral density of x(n), is related to
by
4
Random Processes and LSI Systems
so obviously
and this can be rewritten as
5
Random Processes and LSI Systems
so obviously
and this can be rewritten as
Since the LSI system is deterministic, we can
take it outside the expected value
6
Random Processes and LSI Systems
Whats left inside the expected value is Sxx(w),
so
isnt that nice?
7
Random Processes and LSI Systems
It can be shown that, if x(n) is a zero-mean
sequence (which it is) and h(n) is an LSI system
(which it is), then y(n) is also zero mean.
This means we can find the variance of y (the
average output power by
This means we can reduce the average power of a
random signal (i.e., reduce the noise power) by
attenuating parts of its spectral density
function.
8
Random Processes and LSI Systems
The above equation shows that we can attenuate
parts of the output noise spectral density by
making the system frequency response such that
the system rejects those parts. The power of a
wideband noise source can be reduced by lowpass
(or bandpass) filtering the noise.
9
Random Processes and LSI Systems
Suppose x(n) is a white noise sequence as shown.
Its average power is
Sxx(w)
-p
p
10
Random Processes and LSI Systems
Now, suppose h(n) is an ideal lowpass filter, wc
p/4
H(w)
-p/4
-p
p
-p/4
11
Random Processes and LSI Systems
So now the output spectral density is this
Syy(w)
Sxx(0)
-p/4
-p
p
-p/4
And the average power of the output sequence is
12
Random Processes and LSI Systems
And the average power of the output sequence is
So reducing the bandwidth by 75 also reduced the
noise power by 75.
13
Random Processes and LSI Systems
Suppose the input to our LSI system h(n) is the
sum of two random sequences, z(n) and g(n)
z(n)
x(n)
S
y(n)
h(n)
g(n)
If we know Szz(n) and Sgg(n), can we find Syy(n)?
Yes!
14
Random Processes and LSI Systems
We already know that
So we need to find Sxx(n) in terms of Szz(n) and
Sgg(n). recall that
and
15
Random Processes and LSI Systems
so
16
Random Processes and LSI Systems
Two random processes are independent if the
outcome of one does not influence the other. For
example, rolling two dice. Two random processes
are dependent if the outcome of one can influence
the other. Example drawing a 5-card poker hand.
17
Random Processes and LSI Systems
If the two processes that produce z(n) and g(n)
are independent, we can simplify the last
expression
and if theyre zero-mean processes, this further
reduces to
18
Random Processes and LSI Systems
and this means that the spectral density of x(n)
is
So, for two sequences z(n) and g(n), which are
generated by independent, zero-mean random
processes, summed together to form the input
sequence x(n) to an LSI system h(n), the output
noise spectral density is
19
Random Processes and LSI Systems
Finally, the average power of the output sequence
y(n) is
20
White Noise
A zero-mean white noise sequence x(n) has the
following proprerties Ex(n) 0 x(k) and
x(kn) are independent if n is nonzero. White
noise is independent with respect to any other
sequence.
21
White Noise
If two random variables x and y are independent,
So, for white noise,
But Ex(n) 0, so
22
White Noise
Weve previously seen that
So weve just shown that for white noise,
The power spectral density is the DTFT of the
autocorrelation function, so for zero-mean white
noise,
which is constant across the entire spectrum
(white).
23
White Noise
We can generate a zero-mean white noise sequence
x(n) by randomly choosing, for each n, a real
number between D/2 and D/2. Each number must be
independent of the choice of all others. This is
a uniformly distributed, zero mean, white noise
signal.
24
Quantization Noise and Oversampling
Heres a VERY practical example of exactly this
type of signal quantization noise. Suppose we
have an N-bit A/D converter with an input we
designate xa(t). xa(t) can range from a minimum
of -V volts to a maximum of V volts. The A/D
converter samples xa(t) every T seconds, and
produces an N-bit binary number which
approximates the quantized value of the sample
xa(nT)
25
Quantization Noise and Oversampling
An N-bit binary number is used to represent the
sampled input, xa(nT). This binary number has a
finite number of possible values 2N. Each of
these values represents a range of possible input
voltages, and there is one such range for each
possible N-bit number. Thus, the total input
range,
is divided into 2N smaller ranges.
26
Quantization Noise and Oversampling
Each of these smaller ranges can be expressed as
where M is the actual N-bit number representing a
particular sample. The following figure
illustrates these relationships for a 4-bit A/D
converter (N 4).
27
Quantization Noise and Oversampling
M
xa
M1
M0
4-bit A/D converter
28
Quantization Noise and Oversampling
If the output of the A/D converter is M, xa(nT)
is somwhere in the range
So, if the output of the A/D converter is M, this
says that the input is
29
Quantization Noise and Oversampling
If the output of the A/D converter is M, xa(nT)
is somwhere in the range
So, if the output of the A/D converter is M, this
says that the input is
30
Quantization Noise and Oversampling
If we let the A/D converter output be the center
of the range,
we can rewrite the previous relationship as
31
Quantization Noise and Oversampling
or
where x(n) is the actual input signal to the A/D
converter, and the A/D output is We can express
the quantization error as an error (or noise)
sequence, e(n), yielding this
32
Quantization Noise and Oversampling
Written this way, The A/D converters output can
be thought of as the sum of two sequences a
signal sequence equal to the input signal sampled
by an ideal, infinite-precision A/D converter,
and the quantization noise sequence, e(n). Note
that the quantization noise can have any value
between D/2 and D/2, and its probability density
function is uniform.
33
Quantization Noise and Oversampling
Consider the signal sequence, x(n). Its average
power is
The quantization noise sequence has this average
power
so the A/D converters signal to noise ratio
(SNR) is
34
Quantization Noise and Oversampling
We can write the SNR in terms of decibels
Naturally, we want the SNR to be as large as
possible. This means we make the signal power as
large as possible, by using the entire input
range of the A/D converter. We also minimize
the noise power. One way to do this is to
increase the number of bits, which may or may not
be practical. We would like to have another way.
35
Quantization Noise and Oversampling
To see if there is another way, lets investigate
e(n). We know that e(n) is a random variable
with values in the range
and its uniformly distributed in that range.
Positive and negative values are equally likely,
so its a zero-mean process.
36
Quantization Noise and Oversampling
Its average power is
37
Quantization Noise and Oversampling
With this knowledge, we can write the SNR as
Or, in dB
38
Quantization Noise and Oversampling
If N gt 8, we can use this approximation
substituting this in the previous expression for
SNR gives us
39
Quantization Noise and Oversampling
Which shows that each additional bit of precision
improves the SNR by about 6 dB. This makes
sense, since it cuts D in half. Its not too
much of a stretch to assume that quantization
errors occurring at different times are
independent. If this is assumed, then the
quantization noise sequence, e(n), satisfies the
conditions to be modeled as uniformly distributed
white noise.
40
Quantization Noise and Oversampling
Weve already seen that the power spectral
density of a white noise sequence x(n) is
so the power spectral density of e(n) is as shown
below
See(w)
se2
-p
p
41
Quantization Noise and Oversampling
So the average power of the quantization noise
sequence e(n) is spread over the discrete time
frequency range from p to p. Remember that p
radians per sample in the discrete time domain
maps to fs/2 in the continuous time domain, so
the quantizing noise power is spread over the
range fs/2 to fs/2.
See(2pfT)
se2
- fs/2
- fs/2
42
Quantization Noise and Oversampling
If, for example, we quadruple the sampling
frequency, but leave N alone, we take the
quantizing noise power and spread it over a band
four times as wide.
Suppose we have a bandlimited signal (bandwidth
B) with power spectral density shown in the next
slide
43
Quantization Noise and Oversampling
Sxx(f)
A
f
B
-B
If we sample it at the minimum sample rate, fs
2B, the spectral densities of the signal sequence
and the quantization noise sequence are as shown
next
44
Quantization Noise and Oversampling
Sxx(f)
2BA
se2
f
p
-p
If we double the sample rate (fs 4B), the
signal and noise spectral densities are as shown
45
Quantization Noise and Oversampling
Sxx(f)
4BA
se2
f
p
-p
p/2
-p/2
If we double the sample rate again (fs 8B), the
signal and noise spectral densities are as shown
46
Quantization Noise and Oversampling
Sxx(f)
8BA
se2
f
p
-p
p/4
-p/4
Note that the true signal power and the true
noise power are the same in all three, the
apparent difference merely serves to show that
the noise power is spread over a wider band.
47
Quantization Noise and Oversampling
This can be generalized for fs 2MB, as shown
below
Sxx(f)
2MBA
se2
f
p
-p
p/M
-p/M
Notice that if M gt 1 (i.e., if the signal is
oversampled) some of the noise power is outside
the signal bandwidth.
48
Quantization Noise and Oversampling
We can eliminate the portion of the noise power
which is outside the signal bandwidth by using a
digital lowpass filter to attenuate it. Since
this filter will not attenuate anything in the
signal band, the signal (and the information it
conveys) is unaffected. Unfortunately, the
in-band noise is also not affected, but getting
rid of the out-of-band noise is a good thing.
49
Quantization Noise and Oversampling
If M, the oversampling factor, is 1, we are
sampling at the minimum rate, and the SNR is
given by
If we let M 2 (oversampling by a factor of 2),
and use an ideal lowpass filter to eliminate
out-of-band noise, the result is as shown next
50
Quantization Noise and Oversampling
Sxx(f)
4BA
se2
f
p
-p
p/2
-p/2
This eliminates half the noise power, doubling
the signal to noise ratio. If we let M 4
(oversampling by 4), the SNR is doubled again
51
Quantization Noise and Oversampling
Sxx(f)
8BA
se2
f
p
-p
p/2
-p/2
In general, every time we double the sample rate,
we can double the SNR. This is the same as
increasing the SNR by 3 dB for every doubling of
the sample rate.
52
Quantization Noise and Oversampling
The resolution of an A/D converter is the
effective number of bits which, with M1, would
yield its signal to noise ratio. The SNR of an
A/D converter (with M 1) is given by
so oversampling by 4 effectively adds a bit to
the converters resolution. If a digital filter
eliminates the out-of-band quantization noise,
that is.
53
Quantization Noise and Oversampling
Our oversampling system is as shown below
A/D
H(ejw)
xa(t)
Lowpass wc p/M
fs 2MB
The filter output, g(n), can be thought of as the
sum of two components
54
Quantization Noise and Oversampling
The filter doesnt change the signal, but it does
affect the noise by eliminating the out-of-band
portion. Notice that the noise at the filter
input is denoted e(n), while the output noise is
Since the signal and noise components of the
filter input were assumed to be independent, the
spectral density of the filter output is given by
55
Quantization Noise and Oversampling
The signal power is still given by
The output noise power is
56
Quantization Noise and Oversampling
So the output SNR is
So oversampling by a factor of M multiplies the
output SNR by M. In terms of dB,
57
Quantization Noise and Oversampling
This shows that if M 2, we add 3 dB to the SNR,
and if M 4, we add 6 dB. 6 dB is also the SNR
improvement we get by adding one bit of
resolution. In other words, quadrupling the
sample rate is equivalent to adding one bit to
the A/D converter. If we call the converters
resolution in bits R, the relationship between R,
N and M is
58
Quantization Noise and Oversampling
If we have an N bit converter, and need R bits
of resolution, we can solve for M
The dynamic range of an A/D converter is the SNR
we get when the input is a full-scale sinusoid
59
Quantization Noise and Oversampling
The power of this signal is V2/2. by subsituting
this for sx2 into
we get
This shows that improving the resolution by 1 bit
also improves the dynamic range by 6 dB.
60
Quantization Noise and Oversampling
The dynamic range of an oversampling A/D
converter (and filter) is given by