STATISTICAL%20INFERENCE%20PART%20III - PowerPoint PPT Presentation

About This Presentation
Title:

STATISTICAL%20INFERENCE%20PART%20III

Description:

* REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFS Random Sample from Regular ... Find an EE for . Find CRLB for any UE of exp{-2 ... ESTIMATION PART II Author: – PowerPoint PPT presentation

Number of Views:142
Avg rating:3.0/5.0
Slides: 36
Provided by: Window872
Category:

less

Transcript and Presenter's Notes

Title: STATISTICAL%20INFERENCE%20PART%20III


1
STATISTICAL INFERENCEPART III
  • EXPONENTIAL FAMILY, LOCATION AND SCALE PARAMETERS

2
EXPONENTIAL CLASS OF PDFS
  • X is a continuous (discrete) rv with pdf f(x?),
    ???. If the pdf can be written in the following
    form

then, the pdf is a member of exponential class of
pdfs of the continuous (discrete) type. (Here, k
is the number of parameters)
3
REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFS
  • We have a regular case of the exponential class
    of pdfs of the continuous type if
  • Range of X does not depend on ?.
  • c(?) 0, w1(?),, wk(?) are real valued
    functions of ? for ???.
  • h(x) 0, t1(x),, tk(x) are real valued
    functions of x.

If the range of X depends on ?, then it is called
irregular exponential class or range-dependent
exponential class.
4
EXAMPLES
  • XBin(n,p), where n is known. Is this pdf a
    member of exponential class of pdfs? Why?

Binomial family is a member of exponential family
of distributions.
5
EXAMPLES
  • XCauchy(1,?). Is this pdf a member of
    exponential class of pdfs? Why?

Cauchy is not a member of exponential family.
6
REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFS
  • Random Sample from Regular Exponential Class

is a css for ?.
If Y is an UE of ?, Y is the MVUE of ?.
7
EXAMPLES
  • Recall XBin(n,p), where n is known.

This family is a member of exponential family of
distributions.
is a CSS for p.
is UE of p.
is MVUE of p.
8
EXAMPLES
  • XN(?,?2) where both ? and ?2 is unknown. Find a
    css for ? and ?2 .

9
FISHER INFORMATION AND INFORMATION CRITERIA
  • X, f(x?), ???, x?A (not depend on ?).

Definitions and notations
10
FISHER INFORMATION AND INFORMATION CRITERIA

The Fisher Information in a random variable X
The Fisher Information in the random sample
Lets prove the equalities above.
11
FISHER INFORMATION AND INFORMATION CRITERIA

12
FISHER INFORMATION AND INFORMATION CRITERIA

13
FISHER INFORMATION AND INFORMATION CRITERIA

The Fisher Information in a random variable X
The Fisher Information in the random sample
Proof of the last equality is available on
Casella Berger (1990), pg. 310-311.
14
CRAMER-RAO LOWER BOUND (CRLB)
  • Let X1,X2,,Xn be sample random variables.
  • Range of X does not depend on ?.
  • YU(X1,X2,,Xn) a statistic not containing ?.
  • Let E(Y)m(?).
  • Z?'(x1,x2,,xn?) is a r.v.
  • E(Z)0 and V(Z)In(?) (from previous slides).
  • Let prove this!

15
CRAMER-RAO LOWER BOUND (CRLB)
  • Cov(Y,Z)E(YZ)-E(Y)E(Z)E(YZ)

16
CRAMER-RAO LOWER BOUND (CRLB)
  • E(Y.Z)m(?)
  • -1?Corr(Y,Z)?1?
  • 0 ?Corr(Y,Z)2?1 ?

The Cramer-Rao Inequality (Information Inequality)
17
CRAMER-RAO LOWER BOUND (CRLB)
  • CRLB is the lower bound for the variance of the
    unbiased estimator of m(?).
  • When V(Y)CRLB, Y is the MVUE of m(?).
  • For a r.s., remember that In(?)n I(?), so,

18
EFFICIENT ESTIMATOR
  • T is an efficient estimator (EE) of ? if
  • T is UE of ?, and,
  • Var(T)CRLB
  • Y is an efficient estimator (EE) of its
    expectation, m(?), if its variance reaches the
    CRLB.
  • An EE of m(?) may not exist.
  • The EE of m(?), if exists, is unique.
  • The EE of m(?) is the unique MVUE of m(?).

19
ASYMPTOTIC EFFICIENT ESTIMATOR
  • Y is an asymptotic EE of m(?) if

20
EXAMPLES
  • A r.s. of size n from XPoi(?).
  • Find CRLB for any UE of ?.
  • Find UMVUE of ?.
  • Find an EE for ?.
  • Find CRLB for any UE of exp-2?. Assume n1, and
    show that is UMVUE of exp-2?. Is this a
    reasonable estimator?

21
EXAMPLE
  • A r.s. of size n from XExp(?). Find UMVUE of ?,
    if exists.

22
LIMITING DISTRIBUTION OF MLEs
  • MLE of ?
  • X1,X2,,Xn is a random sample.

23
LIMITING DISTRIBUTION OF MLEs
  • Let be MLEs of ?1, ?2,, ?m.
  • If Y is an EE of ?, then ZabY is an EE of
    abm(?) where a and b are constants.

24
LOCATION PARAMETER
  • Let f(x) be any pdf. The family of pdfs f(x??)
    indexed by parameter ? is called the location
    family with standard pdf f(x) and ? is the
    location parameter for the family.
  • Equivalently, ? is a location parameter for f(x)
    iff the distribution of X?? does not depend on ?.

25
Example
  • If XN(?,1), then X-?N(0,1) ? distribution is
    independent of ?. ? ? is a location parameter.
  • If XN(0,?), then X-?N(-?,?) ? distribution is
    NOT independent of ?. ? ? is NOT a location
    parameter.

26
LOCATION PARAMETER
  • Let X1,X2,,Xn be a r.s. of a distribution with
    pdf (or pmf) f(x ?) ???. An estimator
    t(x1,,xn) is defined to be a location
    equivariant iff
  • t(x1c,,xnc) t(x1,,xn) c
  • for all values of x1,,xn and a constant c.
  • t(x1,,xn) is location invariant iff
  • t(x1c,,xnc) t(x1,,xn)
  • for all values of x1,,xn and a constant c.

Invariant does not change
27
Example
  • Is location invariant or equivariant
    estimator?
  • Let t(x1,,xn) . Then,
  • t(x1c,,xnc) (x1cxnc)/n (x1xnnc)/n
    c t(x1,,xn) c
  • ? location equivariant

28
Example
  • Is s² location invariant or equivariant
    estimator?
  • Let t(x1,,xn) s²
  • Then,
  • t(x1c,,xnc)
  • t(x1,,xn)? Location invariant

(x1,,xn) and (x1c,,xnc) are located at
different points on real line, but spread among
the sample values is same for both samples.
29
SCALE PARAMETER
  • Let f(x) be any pdf. The family of pdfs f(x/?)/?
    for ?gt0, indexed by parameter ?, is called the
    scale family with standard pdf f(x) and ? is the
    scale parameter for the family.
  • Equivalently, ? is a scale parameter for f(x) iff
    the distribution of X/? does not depend on ?.

30
Example
  • Let XExp(?). Let YX/?.
  • You can show that f(y)exp(-y) for ygt0
  • Distribution is free of ?
  • ? is scale parameter.

31
SCALE PARAMETER
  • Let X1,X2,,Xn be a r.s. of a distribution with
    pdf (or pmf) f(x ?) ???. An estimator
    t(x1,,xn) is defined to be a scale equivariant
    iff
  • t(cx1,,cxn) ct(x1,,xn)
  • for all values of x1,,xn and a constant cgt0.
  • t(x1,,xn) is scale invariant iff
  • t(cx1,,cxn) t(x1,,xn)
  • for all values of x1,,xn and a constant cgt0.

32
Example
  • Is scale invariant or equivariant estimator?
  • Let t(x1,,xn) . Then,
  • t(cx1,,cxn) c(x1xn)/n c ct(x1,,xn)
  • ? Scale equivariant

33
LOATION-SCALE PARAMETER
  • Let f(x) be any pdf. The family of pdfs
  • f((x??) /?)/? for ?gt0, indexed by parameter
    (?,?), is called the location-scale family with
    standard pdf f(x) and ? is a location parameter
    and ? is the scale parameter for the family.
  • Equivalently, ? is a location parameter and ? is
    a scale parameter for f(x) iff the distribution
    of (X??)/? does not depend on ? and?.

34
Example
  • 1. XN(µ,s²). Then, Y(X- µ)/s N(0,1)
  • Distribution is independent of µ and s²
  • µ and s² are location-scale paramaters
  • 2. XCauchy(?,ß). You can show that the p.d.f. of
    Y(X- ß)/ ? is f(y) 1/(p(1y²))
  • ? ß and ? are location-and-scale parameters.

35
LOCATION-SCALE PARAMETER
  • Let X1,X2,,Xn be a r.s. of a distribution with
    pdf (or pmf) f(x ?) ???. An estimator
    t(x1,,xn) is defined to be a location-scale
    equivariant iff
  • t(cx1d,,cxnd) ct(x1,,xn)d
  • for all values of x1,,xn and a constant cgt0.
  • t(x1,,xn) is location-scale invariant iff
  • t(cx1d,,cxnd) t(x1,,xn)
  • for all values of x1,,xn and a constant cgt0.
Write a Comment
User Comments (0)
About PowerShow.com