Additional Topics in Regression Analysis

1
Chapter 12

• Additional Topics in Regression Analysis

2
The Stages of Model Building
Model Specification
Coefficient Estimation
Model Verification
Interpretation and Inference
3
Experimental Design

• Dummy variable regression can be used as a tool
  in experimental design work. The experiments
  have a single outcome variable, which contains
  all of the random error. Each experimental
  outcome is measured at discrete combinations of
  experimental (independent) variables, Xj.
• There is an important difference in philosophy
  for experimental designs in comparisons to most
  of the problems that have been considered.
  Experimental design attempts to identify causes
  for the changes in the dependent variable. This
  is done by pre-specifying combinations of
  discrete independent variables at which the
  dependent variable will be measured. An
  important objective is to choose experimental
  points, defined by independent variables that
  provide minimum variance estimators. The order
  in which the experiments are performed is chosen
  randomly to avoid biases from variables not
  included in the experiment.

4
Example Dummy Variable Specification for
Treatment and Blocking Variables(Table 12.1)
5
Regressions Involving Lagged Dependent Variables

• Consider the following regression model linking a
  dependent variable, Y, and K independent
  variables
• Where ?0, ?1, . . . ,?K, ? are fixed
  coefficients. If data are generated by this
  model
• An increase of 1 unit in the independent variable
  xj in the time period t, will with all other
  independent variables held fixed, lead to an
  expected increase in the dependent variable of ?j
  in period t, ?j ? in period (t1), ?j?2 in period
  (t2), ?j?3 in period (t3), and so on. The
  total expected increase over all current and
  future time periods is ?j/(1-?).
• The coefficients ?0, ?1, . . . ,?K, ? can be
  estimated by least squares in the usual manner.
• That a subset of regression parameters are
  simultaneously equal to 0 against the alternative
  hypothesis

6
Regressions Involving Lagged Dependent
Variables(continued)

• Confidence intervals and hypothesis tests for the
  regression coefficients can be computed precisely
  as for the ordinary multiple regression model.
  (Strictly speaking, when the regression equation
  contains lagged variables, these procedures are
  only approximately valid. The quality of the
  approximation improves, all other things being
  equal, as the number of sample observations
  increases.)
• Caution should be used when using confidence
  intervals and hypothesis tests with time series
  data. There is a possibility that the equation
  errors ?i are no longer independent from one
  another. When errors are correlated the
  coefficient estimates are unbiased, but not
  efficient. Thus confidence intervals and
  hypothesis tests are no longer valid.
  Econometricians have developed procedures for
  obtaining estimates under these conditions.

7
Specification Bias

• When significant predictor variables are omitted
  from the model, the least squares estimates will
  usually be biased, and the usual inferential
  statements from hypothesis test or confidence
  intervals can be seriously misleading. In
  addition the estimated model error will include
  the effect of the missing variable(s) and thus
  will be larger. In the rare case where omitted
  variables are uncorrelated with the independent
  variables included in the regression model, this
  will not occur.

8
Multicollinearity

• Multicollinearity refers to the situation when
  high correlation exists between two independent
  variables. This means the two variables
  contribute redundant information to the multiple
  regression model. When highly correlated
  independent variables are included in the
  regression model, they can adversely affect the
  regression results.

9
Multicollinearity
Two Designs with Perfect Multicollinearity (Figure
12.8)
.
.
x2i
x2i
.
.
.
.
7,900
7,900
.
.
.
.
7,700
7,700
.
.
7,500
7,500
3.0
3.2
3.4
3.0
3.2
3.4
(a)
(b)
10
Tests for Heteroscedasticity

• Consider a regression model
• Linking a dependent variable to K independent
  variables and based on n sets of observations.
  Let b0, b1,. . . , bK be the least squares
  estimates of the model coefficients, with
  predicted values
• And the residuals from the fitted model are
• To test the null hypothesis that the error terms,
  ?I, all have the same variance against the
  alternative that their variances depend on the
  expected values

11
Tests for Heteroscedasticity(continued)

• We estimate a simple regression. In this
  regression, the dependent variable is the square
  of the residuals that is ei2 and the
  independent variable is the predicted value,
  yi-hat
• Let R2 be the coefficient of determination of
  this auxiliary regression. Then for a test of
  significance level ?, the null hypothesis is
  rejected if nR2 is bigger than ?21,? where ?21,?
  is the critical value of the chi-square random
  variable with 1 degree of freedom and probability
  of error ?.

12
Autocorrelated Errors

• Consider the regression model
• based on sets of n observations. We are
  interested in determining if the error terms are
  autocorrelated and follow a first-order
  autoregressive model
• where ut is not autocorrelated.
• The test of the null hypothesis of no
  autocorrelation
• is based on the Durbin-Watson statistic

13
Autocorrelated Errors(continued)

• Where et are the residuals when the regression
  equation is estimated by least squares. When the
  alternative hypothesis is of positive
  autocorrelation in errors, that is
• the decision rule is as follows
• Where dL and dU are tabulated for values of n and
  K and for significance levels of 1 and 5 in
  Table 10 of the Appendix.
• Occasionally, one wants to test against the
  alternative of negative autocorrelation that is

14
Estimation of Regression Models with
Autocorrelated Errors

• Suppose that we want to estimate the coefficients
  of the regression model
• where the error term ?t is autocorrelated.
• This can be accomplished in two stages, as
  follows
• (i) Estimate the model by least squares,
  obtaining the Durbin-Watson statistic, d, and
  hence the estimate
• of the autocorrelation parameter
• (ii) Estimate by least squares a second
  regression in which the dependent variable is (Yt
  rYt-1) and the independent variables are (x1t
  rx1,t-1) , (x2t rx2,t-1) , . . ., (xk1t
  rxk,t-1) . The parameters ?1, ?2, . . ., ?k are
  estimated regression coefficients from the second
  model. An estimate of ?0 is obtained by dividing
  the estimated intercept for the second model by
  (1-r). Hypothesis tests and confidence intervals
  for the regression coefficients can be carried
  out using the output from the second model.

15
Key Words

• Autocorrelated Errors
• Autocorrelated Errors with Lagged Dependent
  Variables
• Bias from Excluding Significant Predictor
  Variables
• Coefficient Estimation
• Dummy Variables
• Durbin-Watson Test

• Estimation of Regression Models with
  Autocorrelated Errors
• Experimental Design
• Heteroscedasticity
• Model Interpretation and Inference
• Model Specification
• Model Verification
• Multicollinearity

16
Key Words(continued)

• Regression Involving Lagged Dependent Variables
• Test for Heteroscedasticity