Estimating the Variance of the Least Squares Estimator

1
Estimating the Variance of the Least Squares
Estimator

• Based on Greenes Note 7

2
Context

• The true variance of b is ?2E(X?X)-1 We
  consider how to use the sample data to estimate
  this matrix. The ultimate objectives are to form
  interval estimates for regression slopes and to
  test hypotheses about them. Both require
  estimates of the variability of the distribution.
  

3
Estimating ?2

• Using the residuals instead of the disturbances
  
• The natural estimator e?e/n as a sample
  surrogate for ???/n
• Imperfect observation of ?i ei (? -
  b)?xi
• Downward bias of e?e/n. We obtain the result
  Ee?e (n-K)?2

4
Expectation of ee
5
Proof
6
Estimating s2

• The unbiased estimator is s2 e?e/(n-K).
  Degrees of freedom.
• Therefore, the unbiased estimator is
• s2 e?e/(n-K) ??M?/(n-K).

7
Estimating VarbX

• Estimating the Covariance Matrix for bX
• The true covariance matrix is ?2 (XX)-1
• The natural estimator is s2(XX)-1
• Standard errors of the individual
  coefficients.
• The standard error of bk is the square root of
  the kth diagonal element of s2(XX)-1 .
• How does the conditional variance ?2(XX)-1
  differ from the unconditional one, ?2E(XX)-1?

8
Example

• Estimate the U.S. gasoline demand functionG b0
  b1PG b2Y b3PNC b4PUC e
• Verify the estimated variance-covariance matrix
  of the regression estimator
• y Xb e
• s ee/(N-K)
• s2(XX)-1