Title: STATISTICAL%20INFERENCE%20PART%20III
1STATISTICAL INFERENCEPART III
- EXPONENTIAL FAMILY, LOCATION AND SCALE PARAMETERS
2EXPONENTIAL CLASS OF PDFS
- X is a continuous (discrete) rv with pdf f(x?),
???. If the pdf can be written in the following
form
then, the pdf is a member of exponential class of
pdfs of the continuous (discrete) type. (Here, k
is the number of parameters)
3REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFS
- We have a regular case of the exponential class
of pdfs of the continuous type if - Range of X does not depend on ?.
- c(?) 0, w1(?),, wk(?) are real valued
functions of ? for ???. - h(x) 0, t1(x),, tk(x) are real valued
functions of x.
If the range of X depends on ?, then it is called
irregular exponential class or range-dependent
exponential class.
4EXAMPLES
- XBin(n,p), where n is known. Is this pdf a
member of exponential class of pdfs? Why?
Binomial family is a member of exponential family
of distributions.
5EXAMPLES
- XCauchy(1,?). Is this pdf a member of
exponential class of pdfs? Why?
Cauchy is not a member of exponential family.
6REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFS
- Random Sample from Regular Exponential Class
is a css for ?.
If Y is an UE of ?, Y is the MVUE of ?.
7EXAMPLES
- Recall XBin(n,p), where n is known.
This family is a member of exponential family of
distributions.
is a CSS for p.
is UE of p.
is MVUE of p.
8EXAMPLES
- XN(?,?2) where both ? and ?2 is unknown. Find a
css for ? and ?2 .
9FISHER INFORMATION AND INFORMATION CRITERIA
- X, f(x?), ???, x?A (not depend on ?).
Definitions and notations
10FISHER INFORMATION AND INFORMATION CRITERIA
The Fisher Information in a random variable X
The Fisher Information in the random sample
Lets prove the equalities above.
11FISHER INFORMATION AND INFORMATION CRITERIA
12FISHER INFORMATION AND INFORMATION CRITERIA
13FISHER INFORMATION AND INFORMATION CRITERIA
The Fisher Information in a random variable X
The Fisher Information in the random sample
Proof of the last equality is available on
Casella Berger (1990), pg. 310-311.
14CRAMER-RAO LOWER BOUND (CRLB)
- Let X1,X2,,Xn be sample random variables.
- Range of X does not depend on ?.
- YU(X1,X2,,Xn) a statistic not containing ?.
- Let E(Y)m(?).
- Z?'(x1,x2,,xn?) is a r.v.
- E(Z)0 and V(Z)In(?) (from previous slides).
- Let prove this!
15CRAMER-RAO LOWER BOUND (CRLB)
- Cov(Y,Z)E(YZ)-E(Y)E(Z)E(YZ)
16CRAMER-RAO LOWER BOUND (CRLB)
- E(Y.Z)m(?)
- -1?Corr(Y,Z)?1?
- 0 ?Corr(Y,Z)2?1 ?
The Cramer-Rao Inequality (Information Inequality)
17CRAMER-RAO LOWER BOUND (CRLB)
- CRLB is the lower bound for the variance of the
unbiased estimator of m(?). - When V(Y)CRLB, Y is the MVUE of m(?).
- For a r.s., remember that In(?)n I(?), so,
18EFFICIENT ESTIMATOR
- T is an efficient estimator (EE) of ? if
- T is UE of ?, and,
- Var(T)CRLB
- Y is an efficient estimator (EE) of its
expectation, m(?), if its variance reaches the
CRLB. - An EE of m(?) may not exist.
- The EE of m(?), if exists, is unique.
- The EE of m(?) is the unique MVUE of m(?).
19ASYMPTOTIC EFFICIENT ESTIMATOR
- Y is an asymptotic EE of m(?) if
20EXAMPLES
- A r.s. of size n from XPoi(?).
- Find CRLB for any UE of ?.
- Find UMVUE of ?.
- Find an EE for ?.
- Find CRLB for any UE of exp-2?. Assume n1, and
show that is UMVUE of exp-2?. Is this a
reasonable estimator?
21EXAMPLE
- A r.s. of size n from XExp(?). Find UMVUE of ?,
if exists.
22LIMITING DISTRIBUTION OF MLEs
- MLE of ?
- X1,X2,,Xn is a random sample.
23LIMITING DISTRIBUTION OF MLEs
- Let be MLEs of ?1, ?2,, ?m.
- If Y is an EE of ?, then ZabY is an EE of
abm(?) where a and b are constants.
24LOCATION PARAMETER
- Let f(x) be any pdf. The family of pdfs f(x??)
indexed by parameter ? is called the location
family with standard pdf f(x) and ? is the
location parameter for the family. - Equivalently, ? is a location parameter for f(x)
iff the distribution of X?? does not depend on ?.
25Example
- If XN(?,1), then X-?N(0,1) ? distribution is
independent of ?. ? ? is a location parameter. - If XN(0,?), then X-?N(-?,?) ? distribution is
NOT independent of ?. ? ? is NOT a location
parameter.
26LOCATION PARAMETER
- Let X1,X2,,Xn be a r.s. of a distribution with
pdf (or pmf) f(x ?) ???. An estimator
t(x1,,xn) is defined to be a location
equivariant iff - t(x1c,,xnc) t(x1,,xn) c
- for all values of x1,,xn and a constant c.
- t(x1,,xn) is location invariant iff
- t(x1c,,xnc) t(x1,,xn)
- for all values of x1,,xn and a constant c.
Invariant does not change
27Example
- Is location invariant or equivariant
estimator? - Let t(x1,,xn) . Then,
- t(x1c,,xnc) (x1cxnc)/n (x1xnnc)/n
c t(x1,,xn) c - ? location equivariant
28Example
- Is s² location invariant or equivariant
estimator? - Let t(x1,,xn) s²
- Then,
- t(x1c,,xnc)
- t(x1,,xn)? Location invariant
(x1,,xn) and (x1c,,xnc) are located at
different points on real line, but spread among
the sample values is same for both samples.
29SCALE PARAMETER
- Let f(x) be any pdf. The family of pdfs f(x/?)/?
for ?gt0, indexed by parameter ?, is called the
scale family with standard pdf f(x) and ? is the
scale parameter for the family. - Equivalently, ? is a scale parameter for f(x) iff
the distribution of X/? does not depend on ?.
30Example
- Let XExp(?). Let YX/?.
- You can show that f(y)exp(-y) for ygt0
- Distribution is free of ?
- ? is scale parameter.
31SCALE PARAMETER
- Let X1,X2,,Xn be a r.s. of a distribution with
pdf (or pmf) f(x ?) ???. An estimator
t(x1,,xn) is defined to be a scale equivariant
iff - t(cx1,,cxn) ct(x1,,xn)
- for all values of x1,,xn and a constant cgt0.
- t(x1,,xn) is scale invariant iff
- t(cx1,,cxn) t(x1,,xn)
- for all values of x1,,xn and a constant cgt0.
32Example
- Is scale invariant or equivariant estimator?
- Let t(x1,,xn) . Then,
- t(cx1,,cxn) c(x1xn)/n c ct(x1,,xn)
- ? Scale equivariant
33LOATION-SCALE PARAMETER
- Let f(x) be any pdf. The family of pdfs
- f((x??) /?)/? for ?gt0, indexed by parameter
(?,?), is called the location-scale family with
standard pdf f(x) and ? is a location parameter
and ? is the scale parameter for the family. - Equivalently, ? is a location parameter and ? is
a scale parameter for f(x) iff the distribution
of (X??)/? does not depend on ? and?.
34Example
- 1. XN(µ,s²). Then, Y(X- µ)/s N(0,1)
- Distribution is independent of µ and s²
- µ and s² are location-scale paramaters
- 2. XCauchy(?,ß). You can show that the p.d.f. of
Y(X- ß)/ ? is f(y) 1/(p(1y²)) - ? ß and ? are location-and-scale parameters.
35LOCATION-SCALE PARAMETER
- Let X1,X2,,Xn be a r.s. of a distribution with
pdf (or pmf) f(x ?) ???. An estimator
t(x1,,xn) is defined to be a location-scale
equivariant iff - t(cx1d,,cxnd) ct(x1,,xn)d
- for all values of x1,,xn and a constant cgt0.
- t(x1,,xn) is location-scale invariant iff
- t(cx1d,,cxnd) t(x1,,xn)
- for all values of x1,,xn and a constant cgt0.