Probability and Random Variables

1
Probability and Random Variables

• Why Probability in Communications
• Probability
• Random Variables
• Probability Density Functions
• Cumulative Distribution Functions

Huseyin Bilgekul EEE 461 Communication Systems
II Department of Electrical and Electronic
Engineering Eastern Mediterranean University
2
Why probability in Communications?

• Modeling effects of noise
• quantization
• Channel
• Thermal
• What happens when noise and signal are filtered,
  mixed, etc?
• Making the best decision at the receiver

3
Signals

• Two types of signals
• Deterministic know everything with complete
  certainty
• Random highly uncertain, perturbed with noise
• Which contains the most information?
  Information content is determined from the amount
  of uncertainty and unpredictability. There is no
  information in deterministic signals

Let x(t) be a radio broadcast. How useful is it
if x(t) is known? Noise is ubiquitous.
4
Need for Probabilistic Analysis

• Consider a server process
• e.g. internet packet switcher, HDTV frame
  decoder, bank teller line, instant messenger
  video display, IP phone, multitasking operating
  system, hard disk drive controller, etc., etc.

5
Probability Definitions

• Random Experiment outcome cannot be precisely
  predicted due to complexity
• Outcomes results of random experiment
• Events sets of outcomes that meet a criteria,
  roll of a die greater than 4
• Sample Space set of all possible outcomes, E
  (sometimes called the Universal Set)

6
Example

• Bx4, x5, x6
• Complement
• BCx1, x2, x3
• Union
• Intersection
• Null Set (f), empty set

E
Ao
1
3
5
B
2
4
6
Ae
7
Relative Frequency

• nA number of elements in a set, e.g. the number
  of times an event occurs in N trials
• Probability is related to the relative frequency
• For N small, fraction varies a lot usually gets
  better as N increases

8
Joint Probability

• Some events occur together
• Sum of two dice is 6
• Chance of drawing a pair of jacks
• Events can be
• mutually exclusive (no intersection) tossing a
  coin
• Intersect and have common elements
• The probability of a JOINT EVENT, AB, is

9
Bayes Theorem and Independent Events
10
Axioms of Probability

• Probability theory is based on 3axioms
• P(A) gt0
• P(E) 1
• P(AB) P(A) P(B) If P(AB) f

11
Random Variables

• Definition A real-valued random variable (RV) is
  a real-valued function defined on the events of
  the probability system

Event RV Value P(x)
A 3 0.2
B -2 0.5
C 0 0.1
D -1 0.2
12
Cumulative Density Function

• The cumulative density function (CDF) of the RV,
  x, is given by Fx(a)Px(xlta)

13
Probability Density Function

• The probability density function(PDF) of the RV x
  is given by f(x)
• Shows how probability is distributed across the
  axis

14
Types of Distributions

• Discrete-M discrete values at x1, x2, x3,. . . ,
  xm
• Continuous- Can take on any value in an defined
  interval

DISCRETE
Continuous
15
Properties of CDFs

• Fx(a) is a non decreasing function
• 0 lt Fx(a) lt 1
• Fx(-infinity) 0
• Fx(infinity) 1
• F(a) is right-hand continuous

16
PDF Properties

• fx(x) is nonnegative, fx(x) gt 0
• The total probability adds up to one

17
Calculating Probability

• To calculate the probability for a range of values

AREA F(b)- F(a)
F(b)
2
fx(x)
F(a)
b
a
1
-1
b
a
1
-1
0
18
Discrete Random Variables

• Summations are used instead of integrals for
  discrete RV.
• Discrete events are represented by using DELTA
  functions.

19
PDF and CDF of a Triangular Wave

• Calculate Probability that the amplitude of a
  triangle wave is greater than 1 Volt, if A2.
• Sweep a narrow window across the waveform and
  measure the relative frequency of occurrence of
  different voltages.

fx(x)
s(t)
A
A
-A
-A
20
PDF and CDF of a Triangular Wave

• Calculate Probability that the amplitude of a
  triangle wave is greater than 1 Volt, if A2.

FV(v)
1
fV(v)
3/4
1/4
-2
2
0
1
-2
0
1
2
21
PDF and CDF of a Triangular Wave

• Calculate Probability that the amplitude of a
  triangle wave is in the range 0.5,1 v, if A2.

FV(v)
1
fV(v)
3/4
5/8
1/4
-2
2
0
1
-2
0
1
2
22
PDF and CDF of a Square Wave

• Calculate Probability that the amplitude of a
  square wave is at A.
• Sketch PDF and CDF

s(t)
A
-A
23
PDF and CDF of a Square Wave

• Calculate Probability that the amplitude of a
  square wave is at A. 1/4
• Sketch PDF and CDF

s(t)
A
-A
24
Ensemble Averages

• The expected value (or ensemble average) of
  yh(x) is

25
Moments

• The r th moment of RV x about xxo is