Interpretation and Evaluation of the Simple Regression Model

1
Interpretation and Evaluation of the Simple
Regression Model
2
Interpretation of the Coefficients

• It is important to know what the calculated
  coefficients mean
• b0 is the intercept of the estimated regression
  line. It is the estimated value of y when x is 0.
  It is meaningful only if 0 is a reasonable value
  for x to have.
• b1 is the slope of the estimated regression line.
  It represents the change in the estimated value
  of y when x changes by one unit

3
Evaluation of the Simple Regression Model

• The computer will estimate a regression line for
  any data that is input
• It is up to the person doing the analysis to make
  a judgment about the estimated model and whether
  it provides meaningful information
• We might want to know whether the speculated
  relationship between x and y exists and how good
  a fit the line is

4
Methods of Evaluation

• Standard Error of the Estimate
• Coefficient of Determination
• Hypothesis Tests
• t-test about coefficients
• F-test about the model
• Residual Analysis

5
Standard Error of the Estimate

• The variance of the ys at any particular level
  of x is ?2 and represents the scatter of the ys
  around the regression line ?????x
• This variance can be estimated from the sample
  and its square root is called the standard error
  of the estimate

6
Standard Error of the Estimate

• The standard error of the estimate represents the
  scatter of the data around the estimated
  regression line
• If syx is large there is a lot of residual
  variation and the fit is not good
• If syx is small there is little residual
  variation and the fit is good
• The amount of residual variation helps us decide
  how good a model we have for prediction

7
Coefficient of Determination (r2)

• The problem in interpreting syx is that its
  difficult to make a judgment about large vs.
  small
• The coefficient of determination allows us to
  make a more common sense evaluation of the fit
  of the model
• The coefficient of determination is the percent
  of the variation in the ys thats explained by
  the regression line

8
Coefficient of Determination

• When doing a regression an ANOVA is computed
  that partitions the variation in the ys between
  the variation explained by the xs (SSR) and the
  variation that is unexplained(SSE)

9
Coefficient of Determination

• If r2 is large (98, 89, etc.), the model is
  providing a good fit and we have confidence in
  its ability to predict
• If r2 is small (10, 13, etc.), the model is not
  providing a good fit and we have less confidence
  in its ability to predict
• Note r2 is the square of the correlation
  coefficient, r

10
Hypothesis Tests

• While r2 is more interpretable than syx there is
  still room for ambiguity. (i.e.. How large does
  r2 have to be to be considered a good fit?)
• To make a more cut and dried evaluation of the
  regression model we can use hypothesis tests
• We use t-tests to make judgments about the slope
  coefficient and F-tests to make judgments about
  the model as a whole

11
t-test about ?

• A t-test is used to make a judgment about the
  relationship between x and y
• No relationship between the variables is the null
  hypothesis
• If the null is rejected, we conclude that a
  relationship between the two variables exists

12
F-test for Model Judgment

• The ANOVA produced with the regression procedure
  can be used to do a test about model suitability
• The null hypothesis is that the model has no
  explanatory power
• If we reject the null hypothesis, we are
  concluding that the x(s) has explanatory power
• For simple regression the F-test conclusions are
  the same as the t-test conclusion

13
Residual Analysis

• The residuals may be plotted against the x values
  to make a judgment about whether the model
  conforms to the assumptions of the model
• If the assumptions of the model are met, the
  residual plot should look like a random
  scattering of dots
• If the plot has any discernible pattern, the
  assumptions may be violated
• If the model is violated, we may not be confident
  in our predictions and interpretations