3.1 Expectation

1
3.1 Expectation

• Expectation
• The process of averaging when a r.v. is involved
• Notation
• the mathematical expectation of X, the
  expected value of X, the mean value of X, the
  statistical average of X
• Example 3.1-1
• 90 people are randomly selected
• 8, 12, 28, 22, 15, and 5 people have 18, 45,
  64, 72, 77, and 95, respectively.
• -

...
These terms correspond to PDF
2
3.1 Expectation

• Expected Value of a Random Variable
• In the previous example, if X is the discrete
  r.v. fractional dollar value of pocket coins,
  it has 90 discrete values xi that occur with
  probabilities P(xi), and its expected value EX
  is
• xi the fractional dollar coin
• P(xi) the ratio of the number of people for the
  given dollar value to the total number of people

3
3.1 Expectation

• The expected value of any r.v. X
• If X happen to be discrete with N possible values
  xi having probabilities P(xi) of occurrence

4
3.1 Expectation

• Example 3.1-2
• Determine the mean value of the following
  continuous, exponentially distributed r.v.

5
3.1 Expectation

• Expected Value of a Function of a r.v.
• The expected value of a real function g() of X
• If X is a discrete r.v.
• Example 3.1-3
• A particular random voltage V having Rayleigh
  r.v., a 0, b 5
• The power Y g(V) V2
• Find the average power

6
3.1 Expectation

• Example 3.1-4 Entropy

7
3.1 Expectation

• g(X) a sum of N functions gn(X), n 1,2,,N
• The expected value of the sum of N functions of a
  r.v. X
• the sum of the N expected values of the
  individual function of the r.v.
• Conditional Expected Value
• Conditional expected value of X

8
3.1 Expectation

• Conditional pdf mean (rolling a die)

9
3.2 Moments

• Moments about the Origin
• The nth moment denoted by mn
• The function g(X) Xn, n 0,1, in
• m0 1 the area of the function fX(x)
• m1 EX the expected value of X
• The 1st order moment is very similar to the
  center of gravity

10
3.2 Moments

• Consider a stick with a uniform density of 1 kg/m
  and length of 4 m ? total weight 1 kg/m x 4 m
  4 kg
• Where is the center of gravity ?
• If the density is not uniform, then fx(x) shall
  be a function of the density of stick.

0
5
1
11
3.2 Moments

• Central Moments
• Moments about the mean value of X
• The function g(X) (X-EX)n, n 0,1,
• ?0 1 the area of fX(x)
• ?1 0

12
3.2 Moments

• Variance and Skew
• Variance
• The 2nd central moment ?2
• Standard deviation
• The positive square root of variance
• A measure of the spread in the function fX(x)
  about the mean
• Variance can be found from a knowledge of the 1st
  and 2nd moments

13
3.2 Moments

• Example 3.2-1
• Let X have the exponential density function
• Find the variance of X

14
3.2 Moments

• The third central moment
• A measure of the asymmetry of fX(x) about
• The skew of the density function
• Zero skew if a density is symmetric about
  
• The normalized third central moment
  
• is known as the skewness of the density
  function, or, alternatively, as the coefficient
  of skewness

15
3.2 Moments

• Example 3.2-2
• Continue Example 3.2-1.
• Exponential density
• Compute the skew and coefficient of skewness

16
3.2 Moments

• Chebychevs Inequality
• For a r.v. X with mean and variance

17
3.2 Moments

• The Week Law of Large Numbers
• If X1, X2, ..., Xn is a sequence of independent
  r. v. with identical PDFs and EXi?,
  VarXi?2,
• Ex) How many samples should be taken if we want
  to have a prob. of at least 0.95 that the sample
  mean will not deviate by more than ?/10 from the
  true mean ?

18
3.2 Moments

• Example 3.2-3
• Find the largest probability that any r.v.s
  values are smaller than its mean by 3 standard
  deviation or larger than its mean by the same
  amount

19
3.2 Moments

• An alternative form of Chebychevs inequality
• If for a r.v., then
  for any
• If the variance of a r.v. X approaches zero, the
  probability approaches 1, i.e. X will equal its
  mean value
• Markovs Inequality
• For a nonnegative r.v. X

20
3.4 Transformations of a R.V.

• Transformations of a r.v.
• We may wish to transform (change) one r.v. X into
  a new r.v. Y by means of a transformation
• The density function fX(x) or distribution
  function FX(x) of X is known, and the problem is
  to determine the density function fY(y) or
  distribution function FY(y) of Y.
• The problem viewed as a black box with input
  X, output Y, and transfer characteristic Y
  T(X).
• X discrete, continuous, or mixed r.v.
• T linear, nonlinear, segmented, staircase, etc

21
3.4 Transformations of a R.V.

• A R.V. and an increasing transformation

22
3.4 Transformations of a R.V.

• Monotonic Transformations of a Continuous Random
  Variable
• A transformation T is called
• Monotonically increasing if T(x1)ltT(x2) for any
  x1ltx2
• Monotonically decreasing if T(x1)gtT(x2) for any
  x1ltx2
• Consider the increasing transformation
• T continuous and differentiable at all values
  of x which fX(x)?0
• Let Y have a value y0 corresponding to the value
  x0 of X in shown Figure
• T-1 the inverse of the transformation T

23
3.4 Transformations of a R.V.

• The one-to-one correspondence between X and Y
• The probability of the event Y?y0 must equal
  the probability of the event X?x0
• Differentiate both side w.r.t y0

24
3.4 Transformations of a R.V.

• A R. V. and a decreasing transformation

A B C D E
1 2 3 4 5
-1 -4 -9 -16 -25
Sample Space
Y
X(s)
25
3.4 Transformations of a R.V.

• Consider the decreasing transformation

26
3.4 Transformations of a R.V.

• Example 3.4-1
• Y T(X) aXb, a,b any real constant
• X T-1(Y) (Y-b)/a, dx/dy 1/a
• X Gaussian r.v.
• A linear transformation of a Gaussian r.v.
  produces another Gaussian r.v

See also that Ex.5-1 in Papoulis (pp. 124)
27

• Note) a2, b1

28
3.4 Transformations of a R.V.

• A R. V. and a non-monotonic transformation

A B C D E
1 2 3 4 5
5 8 9 8 5
Sample Space
Y
X(s)
29
3.4 Transformations of a R.V.

• Nonmonotonic Transformations of a Continuous r.v.
  A transformation may not be monotonic
• There may now be more than one interval of values
  of X that corresponds to the event PY?y0
• For the values y0 shown in Figure, the event
  Y?y0 corresponds to the event X?x1 and x2 ? X
  ? x3
• The probability of the event Y?y0 now equals
  the probability of the event x values yielding
  Y?y0 ? xY?y0

poof) see pp. 130, Papoulis
30
3.4 Transformations of a R.V.

• Example 3.4-2
• Find fY(y) for the square-law transformation
  YT(X)cX2, cgt0

(Sol-1) cdf ? differentiate cdf ? pdf
See also that Ex. 5-2 on pp. 125, Papoulis
31
3.4 Transformations of a R.V.
(Sol-2) using the formula
32
3.4 Transformations of a R.V.

• Transformation of a Discrete r.v.
• If X is a discrete r.v. while Y T(X) is a
  continuous transformation
• If the transformation is monotonic, there is
  one-to-one correspondence between X and Y so that
  a set yn corresponds to the set xn through
  the equation ynT(xn)
• The probability P(yn) equals P(xn)
• If T is not monotonic, the above procedure
  remains valid except there now exists the
  possibility that more than one value xn
  corresponds to a value yn
• In such a case P(yn) will equal the sum of the
  probabilities of the various xn which yn T(xn)

33
3.4 Transformations of a R.V.

• Example 3.4-3
• Consider a discrete r.v. X having values x -1,
  0, 1, and 2 with respective probabilities 0.1,
  0.3, 0.4, and 0.2
• Consider Y 2-X2(X3/3)
• Find the density of Y
• The values of X map to respective values of Y
  given by y 2/3, 2, 4/3 and 2/3
• The two values x -1 and x 2 map to one value
  y 2/3
• The probability of Y 2/3 is the sum of
  probabilities PX -1 and PX 3

34
3.5 Computer Generation of One Random Variable

• How to generate the samples with the distribution
  we want
• Suppose that we have the samples following
  uniform distribution
• How do we do for it?
• linear transformation T(x)

35
3.5 Computer Generation of One Random Variable

• We assume initially that T(X) is a monotonically
  nondecreasing functions so that
  applies
• For uniform X, FX(x) x where 0ltxlt1
• We solve for the inverse the above eq.
• Since any distribution function FY(y) is
  nondecreasing, its inverse is nondecreasing, and
  the initial assumption is always satisfied
• Given a specified distribution FY(y) for Y, we
  find the inverse function by solving FY(y) x
  for y ? this result is T(x)

36
3.5 Computer Generation of One Random Variable

• Example 3.5-1
• Find the transformation required to generate the
  Rayleigh random variable with a 0
• We solve for y and find

37
3.5 Computer Generation of One Random Variable

• Example 3.5-2
• Find the required transformation to convert (0,1)
  uniform r.v. to a r.v. with the arcsine
  distribution

38
3.5 Computer Generation of One Random Variable

• Example 3.5-3

39

• Homework assignment
• Programming
• 2-1) Generate 10000 random numbers on 0,1 and
  plot their density function
• Hint) c-functions rand(), srand()
• 2-2) Transform the 10000 random numbers of 1)
  using the linear transform obtained in
  Prob)3.5-1, and plot their density function
• Due next Tuesday