Multiple Regression Analysis: Inference

1
Multiple Regression Analysis Inference
2
Readings

• Lecture notes
• Chapter 4, Introductory Econometrics, 2nd ed. by
  Jeffrey Wooldridge, PP. 123-175

3
Topics

• Topics
• t test
• Confidence interval
• F test

4
Assumptions of the Classical Linear Model (CLM)

• Given the Gauss-Markov assumptions, OLS is BLUE
• Beyond the Gauss-Markov assumptions, we need
  another assumption to conduct tests of hypotheses
  (inference)
• Assume that u is independent of x1, x2,, xk and
  u is normally distributed with zero mean and
  variance s2 u N(0,s2)

5
CLM Assumptions (cont)

• Under CLM, OLS is BLUE OLS is THE minimum
  variance unbiased estimator
• yx N(b0 b1x1 bkxk, s2)

6
Normal Sampling Distributions
7
The t Test
8
t distribution
9
The t Test

• Knowing the sampling distribution for the
  standardized estimator allows us to carry out
  hypothesis tests
• Start with a null hypothesis
• Example H0 bj0
• If we accept the null hypothesis, then we
  conclude that xj has no effect on y, controlling
  for other xs

10
Steps of the t Test

• Form hypothesis
• One-sided hypothesis
• Two-sided hypothesis
• Calculate t statistic
• Find the critical value, c
• Given a significance level, a, we look up the
  corresponding percentile in a t distribution with
  nk1 degrees of freedom and call it c, the
  critical value
• Apply rejection rule to determine whether or not
  to accept the null hypothesis

11
Types of Hypotheses and Significance Levels

• Hypothesis null vs. alternative
• One-sided H0 bj 0 and H1 bj lt 0 or H1 bj gt0
• Two-sided H0 bj 0 and H1 bj ? 0
• Significance level (a)
• If we want to have only a 5 probability of
  rejecting H0 if it is really true, then we say
  our significance level is 5
• a values are generally 0.01, 0.05, or 0.10
• a values dictated by sample size

12
Critical Value c

• What do you need to find c
• t-distribution table (Appendix Table B.3, p. 723
  Hirschey)
• Significance level
• Degrees of freedom
• n-k-1 where n is the of observations, k is the
  of RHS variables, and 1 is for the constant

13
One-Sided Alternatives
yi b0 b1x1i bkxki ui H0 bj 0
H1 bj gt 0
Fail to reject
reject
(1 - a)
a
c
0
Critical value c the (1a)th percentile in a
t-dist with n k 1 DF. t-statistic Results
Reject H0 if t-statisticgtc Fail to reject H0 if
t-statisticltc
14
One-Sided Alternatives
yi b0 b1x1i bkxki ui H0 bj
0 H1 bj lt 0
Fail to reject
reject
(1 - a)
a
-c
0
Critical value c the (1a)th percentile in a
t-dist with n k 1 DF. t-statistic Results
Reject H0 if t-statisticlt-c Fail to reject H0
if t-statisticgt-c
15
Two-Sided Alternative
yi b0 b1X1i bkXki ui H0 bj 0
H1
Critical value the (1a/2)th percentile in a
t-dist with n k 1 DF. t-statistic Results
Reject H0 if t-statisticgtc Fail to reject H0
if t-statisticltc
16
Summary for H0 bj 0

• Unless otherwise stated, the alternative is
  assumed to be two-sided
• If we reject the null hypothesis, we typically
  say xj is statistically significant at the a
  level
• If we fail to reject the null hypothesis, we
  typically say xj is statistically insignificant
  at the a level

17
Testing Other Hypotheses

• A more general form of the t statistic recognizes
  that we may want to test something like H0 bj
  aj
• In this case, the appropriate t statistic is

18
t-Test Example

• Tile Example
• Q 17.513 0.296P 0.066I 0.036A
• (-0.35) (-2.91) (2.56) (4.61)
• t-statistics are in parentheses
• Questions
• (a) How do we calculate the standard errors?
• (b) Which coefficients are statistically
  different from zero?

19
Confidence Intervals

• Another way to use classical statistical testing
  is to construct a confidence interval using the
  same critical value as was used for a two-sided
  test
• A (1 - a) confidence interval is defined as

20
Confidence Interval (cont)
21
Computing p-values for t tests

• An alternative to the classical approach is to
  ask, what is the smallest significance level at
  which the null hypothesis would be rejected?
• Compute the t statistic, and then obtain the
  probability of getting a larger value than this
  calculated value.
• The p-value is this probability

22
EXAMPLE Regression Relation Between Units Sold
and Personal Selling Expenditures for Electronic
Data Processing (EDP),Inc.

• Units sold -1292.3 0.09289 PSE
• (396.5) (0.01097)
• What are the associated t-statistics for the
  intercept and slope parameter estimates?
• t-stat for -3.26 p-value 0.009
• t-stat for 8.47 p-value 0.000
• If p-value lt a, then reject H0 bi 0.
• If p-value gt a, then fail to reject H0 bi 0.
• (c) What conclusion about the statistical
  significance of the estimated parameters do you
  reach given these p-values?

23
Testing a Linear Combination of Parameter
Estimates

• Suppose instead of testing whether b1 is equal to
  a constant, you want to test if it is equal to
  another parameter, that is H0 b1 b2
• Use same basic procedure for forming a t
  statistic

24
NOTE
25
Overall Significance

• H0 b1 b2 bk 0
• Use of F-statistic

26
F Distribution with 4 and 30 Degrees of Freedom
(for a regression model with four X variables
based on 35 observations)
27
The F statistic

• Reject H0 at a
• significance level
• if F gt c

fail to reject
Appendix Tables B.2, pp.720-722. Hirschey
reject
a
(1 - a)
0
c
F
28
EXAMPLE
UNITSt -117.513 - 0.296Pt0.036ADt0.066PSEt (-
0.35) (-2.91) (2.56)
(4.61)

• Pt Price ADt Advertising
• PSEt Selling Expenses UNITSt of Units
  Sold
• s standard error of the regression is
  123.9
• R2 0.97 n 32 0.958
• Calculate the F-statistic.
• What are the degrees-of-freedom associated with
  the F-statistic?
• What is the cutoff value of this F-statistic when
  a .05? When a .01?

29
General Linear Restrictions

• The basic form of the F statistic will work for
  any set of linear restrictions
• First estimate the unrestricted (UR) model and
  then estimate the restricted (R) model
• In each case, make note of the SSE.

30
Test of General Linear Restrictions

• This F-statistic is measuring the relative
  increase in SSE when moving from the unrestricted
  (UR) model to the restricted (R) model
• q number of restrictions

31
EXAMPLE

• Unrestricted Model
• Restricted Model (under ). Note q 1

32
F Statistic Summary

• Just as with t statistics, p-values can be
  calculated by looking up the percentile in the
  appropriate F distribution
• If q 1, then F t2, and the p-values will be
  the same

33
Summary Inferences

• t-test
• One-sided vs. two-sided hypotheses
• Tests associated with a constant value
• Tests associated with linear combinations of
  parameters
• P-values of t tests
• Confidence intervals for estimated coefficients
• Confidence intervals for predictions
• F-test
• P-values of F tests