The Bootstrap

1
The Bootstraps Finite Sample Distribution An
Analytical Approach
Lawrence C. Marsh
Department of Economics and Econometrics Universit
y of Notre Dame
Midwest Econometrics Group (MEG)
Northwestern University
October 15 16, 2004
2
This is the first of three papers
(1.) Bootstraps Finite Sample Distribution (
today !!! )
(2.) Bootstrapped Asymptotically Pivotal
Statistics
(3.) Bootstrap Hypothesis Testing and Confidence
Intervals
3
traditional approach in econometrics
Analogy principle (Manski) GMM (Hansen)
Empirical process
Analytical problem
?
?
approach used in this paper
Analytical solution
Bootstraps Finite Sample Distribution
Empirical process
?
?
4
bootstrap procedure
Start with a sample of size n
Xi i 1,,n
Bootstrap sample of size m
Xj j 1,,m
m lt n or m n or
m gt n
Define Mi as the frequency of drawing each Xi
.
5
.
.
.
6
for i ? k
7
Applied Econometrician
The bootstrap treats the original sample as if
it were the population and induces multinomial
distributed randomness.

8
Econometric theorist what does this buy you?
Find out under joint distribution of
bootstrap-induced randomness and randomness
implied by the original sample data

9
Applied Econometrician
For example,
Econometric theorist

10
The Wild Bootstrap
Multiply each boostrapped value by plus one or
minus one each with a probability of one-half
(Rademacher Distribution).
Use binomial distribution to impose Rademacher
distribution
Wi number of positive ones out of Mi which,
in turn, is the number of Xis drawn in m
multinomial draws.
11
The Wild Bootstrap
Applied Econometrician

Econometric Theorist
under zero mean assumption
12
.
.
.
13
where X is a p x 1 vector.
nonlinear function of ?.
Horowitz (2001) approximates the bias of
14

15
Horowitz (2001) uses bootstrap simulations to
approximate the first term on the right hand
side.
Exact finite sample solution


16
Separability Condition
Definition Any bootstrap statistic, ,
that is a function of the elements of the set
f(Xj) j 1,,m and satisfies the
separability condition
where g(Mi ) and h( f(Xi )) are independent
functions and where the expected value EM
g(Mi) exists, is a directly analyzable
bootstrap statistic.
17
X is an n x 1 vector of original sample values.
X is an m x 1 vector of bootstrapped sample
values.
X HX where the rows of H are all zeros
except for a one in the
position corresponding to the
element of X that was randomly drawn.
EHH (1/n) 1m1n
where 1m and 1n are column vectors of ones.
?m g(X ) g(HX )
Taylor series expansion
Xo Ho X
Setup for empirical process
?m g(Xo) G1(Xo)(X -Xo) (1/2)
(X -Xo)G2(Xo)(X -Xo) R
18
Taylor series expansion
X HX where the rows of H are all
zeros except for a one in the position
corresponding to the element of X that was
randomly drawn.
?m g(X ) g(HX )
Taylor series
Xo Ho X
Ho EHH (1/n) 1m1n
?m g((1/n)1m1nX )
G1((1/n)1m1nX )(H-(1/n)1m1n) X (1/2)X
(H-(1/n)1m1n)G2((1/n)1m1nX )(H-(1/n)1m1n)
X R
Setup for analytical solution
Now ready to determine exact finite moments, et
cetera.
19
e ( In X (XX)-1X)?

, , . . ., ?
e H e
, , . . ., ?
A ( In (1/n)1n1n )
EHH (1/n) 1n1n
No restrictions on covariance matrix for
errors.
20
Applied Econometrician

.
A In
or
where
A1n1n 0
1n1nA 0
A ( In (1/n)1n1n )
and
so
21
Econometric theorist

No restrictions on
where
22
This is the first of three papers
?
(1.) Bootstraps Finite Sample Distribution (
today !!! )
basically done.
(2.) Bootstrapped Asymptotically Pivotal
Statistics
almost done.
(3.) Bootstrap Hypothesis Testing and Confidence
Intervals
Thank you !