Multiple Regression

1
Multiple Regression

• Y might depend on several X vars
• Almost everything for simple regr carries over
• Still use \ in Matlab to find slopes
• Have to watch the size and shape of matrices

2
Multiple Regression

• Df for Regr vars
• H0 Y does not depend on any of the Xs
• Ha Y depends on some of the Xs
• Use F in ANOVA to test

3
Multiple Regression

• Can get confidence on individual slopes
• Have to modify Sxx to be a matrix
• sdcoeffsqrt(diag(mseinv(xx'xx)))
• Where xx is the matrix with 1s on the left
• SD(slope) ? MSE diag elt of Sxx-1

4
Multiple Regression

• If we reject H0, then how do we know which vars
  affect Y?
• Partial F
• Let SSE1 be SSE for a set of vars
• Let SSE2 be SSE for a subset of these vars

5
Multiple Regression

• Note that SSE2gtSSE1 because SSE2 uses fewer vars
• Let dfe1 and dfe2 be associated df
• Partial F (?SSE /?df) /MSE
• Where MSE is from model with more vars
• F distn and numerator df ?df

6
Multiple Regression

• If we wanted to test if a particular coeff 0,
  we would use t distn
• Let pt (coeff 0)/ SD(coeff)
• For the special case where the subset of vars
  omits only this single var
• Partial F (pt)2

7
Multiple Regression

• Step down procedure
• 1. Begin with all vars in the model
• 2. Compute (pt)2 for all vars
• 3. Drop var with smallest value of partial F (in
  2)
• Note that we must recompute everything after
  dropping a variable

8
Dummy vars

• Suppose we use a var that is only 0/1
• Use as an indicator, say for Male/Female
• Effect is to say that other slopes are the same
  for both sexes, but different intercepts

9
Dummy vars

• If we use 0/1 only, then we get 2 sample t test

10
Dummy vars

• How to do different slopes for Male/Female?
• Multiply X by the dummy var
• Can also allow for both diff slopes and diff
  intercepts
• (Does assume that MSE is the same for both sexes)

11
Interaction

• When we mult X by dummy, we are finding the
  interaction between the two vars
• If we mult two X vars, then we find the
  interaction between the two vars
• Think of drug interactions
• Effect of both drugs is different from the
  separate effects of each drug

12
ANOVA

• Using a single dummy var leads to the 2 sample t
  test
• To do ANOVA for G groups, use G-1 dummy vars
• Coefficients are the difference between a groups
  avg and the avg of the group that was left out

13
ANOVA

• The ANOVA we studied earlier is one way ANOVA
• One category variable with several levels
• Two way ANOVA is when we have 2 category vars,
  each with several levels

14
ANOVA

• Use two sets of dummy vars, leaving out one level
  for each var
• Use partial F to see if each category var matters
• Also use interactions

15
ANACOVA

• ANACOVA uses both category and quantitative vars
• Is there a difference among the categories,
  after allowing for the effect of the Xs?
• Again, use partial F to see if categories matter
  (including X vars)