Autocorrelation

1
Autocorrelation

• Outline
• 1) What is it?
• 2) What are the consequences for our Least
  Squares estimator when we have an autocorrelated
  error?
• 3) How do we test for an autocorrelated error?
• 4) How do we correct a model that has an
  autocorrelated error?

2
What is Autocorrelation?
Review the assumption of Gauss-Markov

• Linear Regression Model
  y ?1 ?2x e
• Error Term has a mean of zero E(e) 0 ? E(y)
  ?1 ?2x
• Error term has constant variance Var(e) E(e2)
  ?2
• Error term is not correlated with itself (no
  serial correlation) Cov(ei,ej) E(eiej) 0
  i?j
• Data on X are not random and thus are
  uncorrelated with the error term Cov(X,e)
  E(Xe) 0

This is the assumption of a serially uncorrelated
error. The error is assumed to be independent of
its past it has no memory of its past values. It
is like flipping a coin.
A a serial correlated error (a.k.a.
autocorrelated error) is one that has a memory of
its past values. It is correlated with itself.
Autocorrelation is more commonly a problem for
time-series data.
3

• An example of an autocorrelated error
• Here we have ? 0.8. It means that 80 of the
  error in period t-1
• is still felt in period t. The error in period
  t is comprised of 80
• of last periods error plus an error that is
  unique to period t. This is sometimes called an
  AR(1) model for autoregressive of the first
  order
• The autocorrelation coefficient must lie between
  1 and 1
• -1 lt ? lt 1
• Anything outside this range is unstable and very
  unlikely for economic models

4

• Autocorrelation can be positive or negative
• if ? gt 0 ? we say that the error has positive
  autocorrelation.
• A graph of the errors shows a tracking pattern
• if ? lt 0 ? we say that the error has negative
  autocorrelation.
• A graph of the errors shows an oscillating
  pattern
• In general ? measures the strength of the
  correlation between the errors at time t and
  their values lagged one period.
• There can be higher orders such as a second order
  AR(2) model

5
The mean, variance and covariance for an AR(1)
error
6
What are the Implications for Least Squares?
We have to ask where did we used the
assumption? Or why was the assumption needed in
the first place? We used the assumption in the
derivation of the variance formulas for the least
squares estimators, b1 and b2. For b2 this was
The assumption of a serially uncorrelated error
is made when we say that the variance of a sum is
equal to the sum of the variances. This is true
only if the random variables are uncorrelated.
See Chapter 2, pg. 31.
7

• The proof that the least squares estimators is
  unbiased did not use the assumption of serially
  uncorrelated errors therefore, this property of
  least squares continues to hold even in the
  presence of a autocorrelated error.
• The B in BLUE of the Gauss-Markov Theorem no
  longer holds.
• The variance formulas for the least squares
  estimators are incorrect? invalidates hypoth
  tests and confidence intervals.

The correct variance formula
The large term in brackets shows how the Var(b2)
formula changes to allow for an autocorrelated
error.
8

• If ? gt 0 which is typically the case for economic
  models, it can be shown that the incorrect
  Var(b2) lt correct Var(b2).
• If we ignore the problem and use the incorrect
  Var(b2) we will overstate the reliability of the
  estimates, because we will report a standard
  error that is too small. The t-statistics will
  be falsely large, leading to a false sense of
  precision.

9
How to Test for Autocorrelation

• We test for autocorrelation similar to how we
  test for a heteroskedastic error estimate the
  model using least squares and examine the
  residuals for a pattern.
• Visual Inspection Plot residuals against time.
  Do they have a systematic pattern that indicates
  a tracking pattern (for positive autocorrelation)
  or an oscillating pattern (for negative
  autocorrelation)?
• Example a model of Job Vacancies and the
  Unemployment Rate
• Page 278, Exercise 12.3
• ln(JV)t ?1 ?2 ln(U)t et
• Where JV are job vacancies, U is the
  unemployment rate.

10
Sum of
Mean Source DF
Squares Square F Value Pr gt
F   Model 1 8.72001
8.72001 107.36 lt.0001 Error
22 1.78687
0.08122 Corrected Total 23
10.50688   Root MSE
0.28499 R-Square 0.8299
Dependent Mean 0.63427 Adj R-Sq
0.8222 Coeff Var
44.93266 Parameter
Estimates Parameter
Standard Variable DF Estimate
Error t Value Pr gt t  
Intercept 1 3.50270 0.28288
12.38 lt.0001 lu 1
-1.61159 0.15554 -10.36 lt.0001

ln(JV)t 3.503 1.612 ln(U)t
11

• 2) Formal Test Durbin-Watson Test
• This test is based on the residuals from the
  least squares regression.
• (remember that our test for heteroskedasticity
  was also based on the residuals from a least
  squares regression)
• If the error term has first-order serial
  correlation, et ?et-1 vt
• The residuals at t and t-1 ought to be
  correlated.
• Ho ? 0
• H1 ? gt 0 (positive autocorrelation is more
  likely in economics)
• The Durbin-Watson test statistic is used to test
  this hypothesis. It is constructed using the
  least squares residuals. Specifically

12
The d statistic can be simplified into an
expression involving The sample correlation ?
between the residuals at t and t-1


Note that if there is no autocorrelation, then ?
0, so that ? should also be around 0, implying
a d-statistic of 2. If ? 1 ? d 0 If ?
-1 ? d 4 The question then becomes How far
below 2 must the d-statistic be to say that
there is positive autocorrelation? and How far
above 2 must the d-statistic be to say that
there is negative autocorrelation?
13
Typically we want to compare our test statistic
to a critical value to determine whether or not
the data reject the null hypothesis. The
probability distribution for the d-statistic have
some convenient well-known form such as the t or
the F. Instead, its distribution depends on the
values of the explanatory variables. For this
reason, the best we can do is tie down a lower
and upper bound for the critical d values. See
Table 5, pg 393-396. Suppose T24 observations
used to estimate a model with one
independent variable and an intercept k 2.

• 1) The test Ho ? 0
• H1 ? gt 0
• 2) Calculate the d-statistic according
• to the formula on slide 12.11
• Conduct the test
• If d lt 1.273 ? reject Ho
• If d gt 1.446 ? fail to reject Ho
• If 1.273 lt d lt 1.446 ?
• inconclusive

2
d
0
4
dL 1.273
dU 1.446
14
Example Test the model of job vacancies. For
this model T24, and k2 ? we can use the dL and
dU critical values from slide 12.13. To
calculate the durbin-watson d-statistic, we get
SAS to do so by adding the dw option to the
model statement Proc reg model ljv lu /
dw Run
The REG Procedure Model MODEL1 Dependent
Variable ljv Durbin-Watson D
1.090 Number of Observations 24 1st
Order Autocorrelation 0.432
Conclusion reject Ho because d 1.09 lt 1.273
15
How to Correct for Autocorrelation

• It is quite possible that the error in a
  regression equation appears to be autocorrelated
  due to an omitted variable. Recall that omitted
  variables end up in the error term. If the
  omitted variable is correlated over time (which
  is true of many economic time-series), then the
  residuals will appear to track ? Correct the
  problem by reformulating the model (include the
  omitted variable)
• 2) Generalized Least Squares
• Similar to the problem of a heteroskedastic
  error, we will take our model that has an
  autocorrelated error and transform it into a
  model that has a well-behaved (serially
  uncorrelated) error.

16
The original model where
vt is a well-behaved error that is serially
uncorrelated
Algebraic manipulations
17
Construct new variables
These variables are sometimes called generalized
differences. We will then estimate this model
using the new variables
Note that x1 is really a constant, not a
variable. The intercept ?1 has always been
multiplied by 1 and now it is multiplied by (1-?)
18

• The problem is, what to use for ? because it is
  unknown?
• There are many different ways of estimating ?.
• All methods begin with the residuals from least
  squares,
• the same residuals used to construct the
  durbin-watson test statistic
• 2) Use this estimate of ? to construct the
  generalized differences
• according to the formulas on the previous
  slide for y, x1 and x2
• 3) Run Least Squares using these generalized
  differences
• 4) (Cochrane-Orcutts Iterative Procedure) a.k.a
  Yule-Walker Method
• From step 3), take the residuals from this
  regression
• and repeat steps 1) 3). Each time you get new
  estimates of ?1 and ?2.
• Continue to iterative until the values of the
  estimates converge.

19
The AUTOREG Procedure
Dependent Variable
ljv   Ordinary Least
Squares Estimates SSE
1.78686627 DFE 22
MSE 0.08122 Root MSE
0.28499 SBC
12.1229868 AIC 9.76687918
Regress R-Square 0.8299 Total R-Square
0.8299 Durbin-Watson
1.0896
Standard Approx Variable
DF Estimate Error t Value Pr
gt t Intercept 1 3.5027
0.2829 12.38 lt.0001 lu
1 -1.6116 0.1555 -10.36
lt.0001   Estimates of
Autocorrelations Lag Covariance Correlation
-1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
0 0.0745 1.000000
1 0.0322
0.431840
Preliminary MSE
0.0606   Estimates of
Autoregressive Parameters
Standard Lag
Coefficient Error t Value
1 -0.431840 0.196822
-2.19   Yule-Walker
Estimates   SSE
1.4379184 DFE 21
MSE 0.06847 Root MSE
0.26167 SBC
10.2930273 AIC 6.75886582
Regress R-Square 0.8853 Total R-Square
0.8631 Durbin-Watson
2.0166 The AUTOREG
Procedure
Standard Approx Variable
DF Estimate Error t Value Pr
gt t Intercept 1 3.5138
0.2437 14.42 lt.0001 lu
1 -1.6162 0.1269 -12.73
lt.0001
These results will be discussed in class.
20
options ls78 options formdlim'' goptions
resetall data one infile 'c\my
documents\classes\UE\datafiles\vacan.dat'
firstobs2 input jv u time_n_   ljv
log(jv) lu log(u) symbol1 inone cred vdot
h.5 symbol2 cblack ijoin l1
  proc gplot plot ljv lu 1   proc
autoreg model ljv lu / dwprob output
outstuff residual ehat predictedljv_hat run  
proc gplot datastuff plot ljvmortg1
ljv_hatmortg 2 / overlay legend plot
ehattime1 / vref0   proc autoreg model ljv
lu / nlag1 run
use PROC AUTOREG with DWPROB to get p-values for
the DW statistic
Using PROC AUTOREG with the / nlag1 option in
the model statement will estimate the model and
correct for first-order autocorrelation in the
errors.