Chapter 9: Regression Wisdom

1
Chapter 9Regression Wisdom

• Prediction is difficult, especially about the
  future.
• Niels Bohr,
• Danish Physicist

2
Sifting Residuals for Groups

• No regression analysis is complete without a
  display of the residuals.
• Is the linear model reasonable?
• Residuals are what is left-over after the model
  describes the relationship.
• Reveal subtleties
• Additional details that confirm
• Reveal violations of regression conditions
• Look a histogram of the residuals

3
Subsets

• Another condition for fitting models
• All data must come from the same group.
• If you discover that there is more than 1 group
  in a regression
• Analyze the groups separately.
• Use a different model for each group.
• OR, use the original model and note the different
  groups.

4
Getting the Bends

• Fundamental assumption for working with linear
  models
• The relationship modeled is in fact linear
• Sometimes it is hard to tell from the
  scatterplot.
• Plot regression residuals against predicted
  values to see if there is a bend.
• The residual plot should have NO pattern if a
  linear model is appropriate.

5
Extrapolation Reaching Beyond the Data

• Linear models give a predicted value for each
  case in the data.
• Put a new x-value into the equation.
• The equation gives a predicted y-value.
• The farther the x-values lie from the data we
  used to build the regression, the less we should
  trust the predicted y-value.

6
Extrapolation

• Extrapolation
• The use of a regression line for prediction
  outside the domain of values of x.
• NOT to be trusted!
• Extrapolation requires the assumption that the x,
  y relationship never changes, even at extreme
  values of y.

7
Predicting the Future

• When the x-variable in a linear model is time,
  extrapolation is an attempt to predict the
  future.
• If you must extrapolate into the future, be wary!
• Dont believe that the prediction will come true!!

8
Outliers

• Outliers strongly influence a regression.
• Outlier any point that stands away from the
  other data points.
• Model outliers
• Point that falls far from the regression line.
• Compare the regression models (with and without
  outlier).
• x-Outliers
• Can especially influence regression model.
• High Leverage

9
Influential Points

• Influential points can hide in residual plots.
• Points with high leverage pull the line close to
  them.
• Small residuals
• To find influential points
• Look at the original scatterplot.
• Find the regression model with and without the
  point(s).

10
Lurking Variables and Causation

• With observational data, there is no way to be
  sure that a lurking variable is not the cause of
  any apparent association.
• Do NOT infer causality from a regression.
• Resist the temptation to conclude that x causes y
  from a regression, no matter how obvious that
  conclusion seems to you.

11
Working with Summary Values

• Scatterplots of statistics summarized over groups
  tend to show less variability than would be seen
  if we measured the same variable on individuals.
• Show less scatter
• Can give a false impression of how well a line
  summarizes the data.

12
What Can Go Wrong?!?!

• Make sure the relationship is straight.
• Check the residual plot!!
• Beware of extrapolating.
• Be cautious of extrapolating beyond the x-values
  of the data.
• Dont trust predicted values when x-variable is
  time.
• Be on guard for different groups in your
  regression.
• Look at a histogram of the residual values.

13
What Can Go Wrong?!?!

• Look for outliers.
• Beware of high leverage points, especially
  influential ones.
• Consider comparing two regressions.
• One with outlier and one without.
• Treat outliers honestly.
• Beware of lurking variables.
• Watch out for summary data.