The Simple Regression Model II

1
The Simple Regression Model II
2
Recap

• Two variable regression model
• y ß0 ß1x u or E(yx) ß0 ß1x
• Least squares estimation leads to

3
Recap (continued)

• The fitted regression line
• A residual
• A measure of goodness of fit
• R2 SSE/SST 1 SSR/SST
• where

4
Stata Output

• . regress wage educ
• Source SS df MS
  Number of obs 526
• -------------------------------------------
  F( 1, 524) 103.36
• Model 1179.73204 1 1179.73204
  Prob gt F 0.0000
• Residual 5980.68225 524 11.4135158
  R-squared 0.1648
• -------------------------------------------
  Adj R-squared 0.1632
• Total 7160.41429 525 13.6388844
  Root MSE 3.3784
• --------------------------------------------------
  ----------------------------
• wage Coef. Std. Err. t
  Pgtt 95 Conf. Interval
• -------------------------------------------------
  ----------------------------
• educ .5413593 .053248 10.17
  0.000 .4367534 .6459651
• _cons -.9048516 .6849678 -1.32
  0.187 -2.250472 .4407687
• --------------------------------------------------
  ----------------------------
• .

5
Lecture 4 Outline

• Consider functional form (or how to incorporate
  some non-linearity).
• Derive the sampling distribution of the least
  squares estimators.
• The sampling distribution is the basis of
• statistical inference
• evaluation of estimators.

6
Functional Form

• Fitted Linear wage equation model

• ?wage 0.541?educ
• One extra year of education leads to an
  increased wage of 54 cents.
• Constant change in wage whatever the level of
  education.

7
Functional Form

• Suppose instead a constant percentage change in
  the wage for a unit increase in education.
• A model which captures this is
• log(wage) ß0 ß1educ u
• In this model
• ?wage ? (100.ß1)?educ
• Mathematically this is because
• log(1r) ? r for small r (Appendix A)
• It is more important, however, to understand the
  practical implications.

8
Stata Example

• . use wage3, clear
• . generate logwagelog(wage)
• . regress logwage educ
• Source SS df MS
  Number of obs 526
• -------------------------------------------
  F( 1, 524) 119.58
• Model 27.5606288 1 27.5606288
  Prob gt F 0.0000
• Residual 120.769123 524 .230475425
  R-squared 0.1858
• -------------------------------------------
  Adj R-squared 0.1843
• Total 148.329751 525 .28253286
  Root MSE .48008
• --------------------------------------------------
  ----------------------------
• logwage Coef. Std. Err. t
  Pgtt 95 Conf. Interval
• -------------------------------------------------
  ----------------------------
• educ .0827444 .0075667 10.94
  0.000 .0678796 .0976091
• _cons .5837727 .0973358 6.00
  0.000 .3925563 .7749891
• --------------------------------------------------
  ----------------------------

9
Stata Example interpretation

• The fitted regression line is

• A one year increase in education leads to
  (approximately) an 8.3 increase in the
  (expected) wage.
• This measures economists idea of the returns
  to education.
• The estimated intercept (0.584) is not
  meaningful.
• The R-squared value of 0.19 refers to variation
  in log(wage).

10
A Constant Elasticity Model

• Consider the model
• log(y) ß0 ß1log(x) u
• Mathematically, ?y (ß1) ?x.
• In other words, ß1 is the constant elasticity of
  y with respect to x.

11
Example

• Consider the model
• log(salary) ß0 ß1log(sales) u
• Estimating the model yields

n 209, R-squared 0.211. We estimate that a
1 increase in sales leads to a 0.26 increase in
CEO salary.
12
Other possibilities (e.g)

• y ß0 ß1log(x) u
• y ß0 ß1(1/x) u

• Many non-linear forms can be given a useful
  linear representation.
• Interpretation of estimated parameters depends on
  precise functional form.
• Choice should be based on views of underlying
  data generation process (statistical tests,
  convenience).

13
Sampling Distribution of OLS Estimators

• What are the expected values of the least
  squares estimators?
• Are they unbiased?
• What are the variances of the least squares
  estimators?
• Are they efficient?
• Do they have a normal distribution?
• How do we use this for statistical inference?

14
Unbiasedness of OLS

• Assume the population model is linear in
  parameters as y b0 b1x u SLR.1
• Assume we can use a random sample of size n,
  (xi, yi) i1, 2, , n, from the population
  model. Thus yi b0 b1xi ui SLR.2
• Assume E(ux) 0 and thus E(uixi) 0 SLR.3
• Assume there is variation in the xi SLR.4

15
Unbiasedness of OLS (cont)

• In order to think about unbiasedness, we need to
  rewrite our estimator in terms of the population
  parameter
• Start with a simple rewrite of the formula as

16
Unbiasedness of OLS (cont)
17
Unbiasedness of OLS (cont)
18
Unbiasedness of OLS (cont)
19
Unbiasedness Summary

• The OLS estimates of b1 and b0 are unbiased (see
  p. 51 for b0 proof)
• Proof of unbiasedness depends on our 4
  assumptions if any assumption fails, then OLS
  is not necessarily unbiased
• Remember unbiasedness is a description of the
  estimator in a given sample we may be near or
  far from the true parameter

20
Variance of the OLS Estimators

• Now we know that the sampling distribution of
  our estimator is centered around the true
  parameter
• Want to think about how spread out this
  distribution is
• Much easier to think about this variance under
  an additional assumption, so
• Assume Var(ux) s2 (Homoscedasticity)

21
Variance of OLS (cont)

• Var(ux) E(u2x)-E(ux)2
• E(ux) 0, so s2 E(u2x) E(u2) Var(u)
• Thus s2 is also the unconditional variance,
  called the error variance
• s, the square root of the error variance is
  called the standard deviation of the error
• Can say E(yx)b0 b1x and Var(yx) s2

22
Homoskedastic Case
y
f(yx)
.
E(yx) b0 b1x
.
x1
x2
23
Heteroskedastic Case
f(yx)
y
.
.
E(yx) b0 b1x
.
x
x1
x2
x3
24
Variance of OLS (cont)
25
Variance of OLS Summary

• The larger the error variance, s2, the larger
  the variance of the slope estimate
• The larger the variability in the xi, the
  smaller the variance of the slope estimate
• As a result, a larger sample size may decrease
  the variance of the slope estimate
• Problem that the error variance is unknown

26
Variance of OLS (cont.)

• There is a similar expression for the variance
  of the intercept estimator

• There is a non-zero covariance between the slope
  and intercept estimators

27
Normality

• To complete the derivation of the sampling
  distribution we need to assume that
• ux Normal(0, s2)
• Since the slope estimator is a linear function
  of the us,

28
Inference?

• It would seem that
• could form the basis of statistical inference.
• However s is unobservable.
• Hence

29
Estimating the Error Variance

• We dont know what the error variance, s2, is,
  because we dont observe the errors, ui
• What we observe are the residuals, ûi
• We can use the residuals to form an estimate of
  the error variance

30
Error Variance Estimate (cont)
31
Error Variance Estimate (cont)
32
Inference Again
does not have a normal distribution but
rather has a t-distribution with n-2 degrees of
freedom.

• The t-distribution
• has fatter tails than normal
• is characterised by degrees of freedom
• gets more like normal as df increase

33
Next Week

• Chapter 3 of Wooldridge
• Estimation of the multiple regression model
• y b0 b1x1 b2x2 bkxk u
• E(yx) b0 b1x1 b2x2 bkxk