Multiple Regression

1
Chapter 11

• Multiple Regression

2
Introduction

• So far, we have only examined a single variable
  or the relationship between two variables.
• In the real world, how often does one variable
  explain all of the change in another variable
  (perfect correlation)?
• (Hint Not very often!)

3
Expanding Regression

• We still need quantitative variables to perform
  regression.
• In the past, we had n observed values for both x
  and y (e.g., a person would give both height and
  weight).
• Now we have n observed values for x1, x2, x3, ,
  xp and a value of y.

4
Old Methods Return

• As before, we must look at the
• Shape
• Center
• Spread
• Outliers (if any)
• We look at each variable separately (i.e., look
  at the histograms, means, etc.)

5
Pairs of Variables

• After examining the variables one at a time, we
  can examine the variables in pairs.
• Examine the simple linear regression of
• y and x1
• y and x2
• y and xp

6
Pairs of Variables

• If each pair shows some relationship, then we can
  include the variables in the multiple regression
  model.
• Next, we need to expand our previous models to
  accommodate the extra variables.

7
True Multiple Regression

• We have the statistical model for each
  observation
• The Population Model
• The Prediction Model

8
Interpreting Beta

• Consider each Beta in terms of holding all other
  Beta coefficients constant.
• In other words, if you get a model
• Where x1 measures promotional spending and x2
  measures a competitors promotional spending and y
  measures sales income.
• Interpretation
• x1? Holding all else constant, for every dollar
  we spend on promotion, we gain 2 dollars in
  sales.
• x2 ? Holding all else constant, for every dollar
  our competitor spends on promotion, we lose 3
  dollars in sales.

9
True Multiple Regression

• The observed variability is measured as a
  function of the residuals
• Residual for observation i
• The regression standard error ?

10
Inference Techniques

• Confidence Intervals for Beta and Hypothesis test
  are feasible.
• See page 653 for an excellent graphic showing the
  nature of the CI and Hypothesis Tests.

11
Selling Price of a Home

• The results in JMP after a step-wise selection of
  variables that will prove significant in the
  model. Selection from 4 possible variables. JMP
  found 3 were significant.

12
More Bedrooms Reduces the Sale Price?!

• According to the model, having more bedrooms will
  reduce the price of the house. Hmm. Lets look
  at the model with just sale price and bedrooms.

13
Analyzing the Model

• So, Bedrooms should add to the sale price of a
  house when bedrooms are the only x-variable.
• What we found is likely due to an interaction
  effect as well as not making sure we fit the
  right variable transformations/levels of
  variables. Have fun in 326!

14
Quasi-Multiple Regression

• We can also have what appears to be multiple
  regression but is in fact non-linear regression
  on one x-variable.
• The polynomial model (p-th order)
• Still only one x-variable even though the model
  seems complex.
• Handles non-linear models.
• Consider each Beta in terms of holding all other
  Beta coefficients constant.

15
Quasi-Multiple Regression

• The Scatterplot Linear Fit

16
Quasi-Multiple Regression

• The linear fit model is good, right? Actually
  not. We violated the prime assumption of
  linearity! A better fit
• With X2 up to X5

17
Quasi-Multiple Regression

• As you can see, the models improve in fit as we
  added more terms. The numerical results
• Up to X2 ?
• Up to X5 ?