Review II: Ordinary Least Square (OLS) Regressions

1
Review II Ordinary Least Square (OLS) Regressions

• Econ6001 Applied Econometrics
• Lecturer Zhigang Li

2
Questions to check your understanding of the
materials last class

• Economic theory predicts that introducing minimum
  wage law would reduce the employment. Could you
  write down an econometric model about this
  hypothesis?
• What is the dependent variable here?
• What is the causing variable?
• Which parameter is of interest to us to test the
  hypothesis?
• How to estimate this parameter?
• How do you test whether this theory is supported?

3
A Linear Multivariate Model

• Yiß0,iß1X1,iß2X2,iß3X3,ißkXk,iu,i
• This model tells us what the relationship between
  Y and the Xs are.
• We want to consistently estimate some of the
  parameters (or coefficients) in (ß0 , , ßk).
• Consistency When the sample size becomes large,
  our estimates converge to the true values of (ß0
  , , ßk).
• The error term u contains all factors affecting
  Y but are not included in the Xs. It reflects
  the ignorance of researchers. The statistical
  property of u is the key to the consistent
  estimation of the parameters.

4
Unbiased or consistent

• Unbiased On average the estimate is the same as
  the true parameter estimated, no matter whether
  the sample size is large or small.
• Consistent When sample size becomes large, on
  average the estimate converges to the true
  parameter.
• How large is large enough?

5
Why do we need more than one independent
variables?

• The Xs are called the independent variables, of
  covariates.
• Why we need more than one covariates?
• We may be interested in the effect of more than
  one variables.
• Adding more covariates that are relevant may
  improve the precision of the estimates of the
  coefficients that we are interested in.

6
The OLS Estimate of ßk

• Solve the following k1-equation-k1-unknowns
  system for estimates of ßs
• This estimation is called OLS because it actually
  minimizes the sum of squared prediction errors.

7
Properties of the Estimate (Chapter 3)

• Given sufficient conditions, we can prove that
  when the sample size (or the number of
  observations) is large enough, the estimates are
  consistent and follows the following particular
  distribution

Mean
Standard Deviation
Variation of Xj Explained by Other Xs
Sample Variation of u
Total Variation In Xj
8
Law of large numbers(Appendix C)

• Under certain conditions (e.g. Y1, Y2, , Yn are
  independent, identically distributed random
  variables), then the average of Ys is a
  consistent estimate of the true mean.

9
Central Limit Theorem

• Under certain assumptions, the average of Ys
  converges to a standard normal distribution as
  the sample size becomes large.

10
Key Assumptions for Consistent Estimates of
Coefficients

• For correct estimates of parameters
• MLR.2 Random (or representative) sampling
• MLR.3 Exogeneity (Xj is uncorrelated with u)
• Xs can be correlated but not perfectly
  correlated.
• For correct estimates of the variances of
  parameter estimates
• Spherical Disturbances
• MLR.5 Homoskedasticity (The variances of the
  error term u are not affected by Xs)
• Nonautocorrelation (The error term of different
  observations are not correlated)

11
Testing H0ßj0, H1ßjgt0

• T test
• Calculate
• Reject H0 if tgtc, where c is the critical value
  decided by your chosen confidence level and by
  the distribution of your estimate.
• P-values
• P-value, often reported automatically by most
  statistical package, is defined as P(Tgtt) in this
  case, where T is a t-distribution.

12
Some Convenient Techniques I

• Using logarithmic functional forms
• Easy to interpret (elasticity) and compare
• Decrease the effect of outliers
• Changing the distribution of positive variables
  to have a more normal shape
• Variables related to dollar amounts and
  population often take the log form
• Variables measured in years often take original
  form
• Variables representing percentage often take
  original form

13
Some Convenient Techniques II

• Quadratic Terms
• Interpretation
• Turning point
• Interaction Terms
• Yß0ß1X1ß2X2ß3X1X2u
• Adding regressors to reduce error variance
• Especially true if the regressor is uncorrelated
  with other regressors.

14
Some Convenient Techniques III Binary
Independent (Dummy) Variables

• Everything is as usual. The coefficient of the
  dummy (variable) reflects the differential
  (partial) effect of the two characteristics on
  the dependent variable.
• The dummy approach can be used for discrete
  variables with more than two categories. For
  example, if there are g categories, we need g-1
  dummies (when an intercept is in the model), each
  of which for one of the g category.

15
Some Convenient Techniques IV A Binary
Dependent Variable

• When the dependent variable is a Bernoulli
  variable (i.e. the value is either 0 or 1), then
  the linear regression model actually predict the
  probability for 1 to happen. Therefore, we also
  call this model a linear probability model.
• Heteroskedasticity
• Consistency of estimates not affected
• Variances of estimates affected, but likely small.