MULTIPLE REGRESSION TOPICS

1
LECTURE 5

• MULTIPLE REGRESSION TOPICS
• SQUARED MULTIPLE CORRELATION
• B AND BETA WEIGHTS
• HIERARCHICAL REGRESSION MODELS
• SETS OF INDEPENDENT VARIABLES
• SIGNIFICANCE TESTING SETS
• POWER
• ERROR RATES

2
SQUARED MULTIPLE CORRELATION

• Measure of variance accounted for by predictors
• Always increases (or stays same) with additional
  predictors
• Always gt 0 in OLS
• More stable than individual predictors
  (compensatory effect across samples)

3
Multiple regression analysis

• The test of the overall hypothesis that y is
  unrelated to all predictors, equivalent to
• H0 ?2y?123 0
• H1 ?2y?123 0
• is tested by
• F R2y?123 / p / ( 1 - R2y?123) / (n p
  1)
• F SSreg / p / SSe / (n p 1)

4
SSreg
ssx1
SSy
SSe
ssx2
Fig. 8.4 Venn diagram for multiple regression
with two predictors and one outcome measure
5
SSreg
ssx1
SSy
SSe
ssx2
Fig. 8.4 Venn diagram for multiple regression
with two predictors and one outcome measure
6
Type I ssx1
SSx1
SSy
SSe
SSx2
Type III ssx2
Fig. 8.5 Type I and III contributions
7
B and Beta Weights

• B weights
• are t-distributed under multinormality
• Give change in y per unit change in predictor x
• raw or unstandardized coefficients

8
B and Beta Weights

• Beta weights
• are NOT t-distributed- no correct significance
  test
• Give change in y in standard deviation units per
  standard deviation change in predictor x
• standardized coefficients
• More easily interpreted

9
PATH DIAGRAM FOR REGRESSION Beta weight form
? .5
X1
.387
r .4
Y
e
X2
? .6
R2 .742 .82 - 2(.74)(.8)(.4)
? (1-.42) .85
10
Depression
e
.471
?.4
LOC. CON.
-.345
-.448
DEPRESSION
-.317
SELF-EST
R2 .60
.399
-.186
SELF-REL
11
PATH DIAGRAM FOR REGRESSIONS Beta weight form
X1
? .2
.387
r .35
R2y .2
Y1
e1
? .3
X2
? .2
? .5
Y2
e2
? .3
R2y .6
12
HIERARCHICAL REGRESSION

• Predictors entered in SETS
• First set either causally prior, existing
  conditions, or theoretically/empirically
  established structure
• Next set added to decide if model changes
• Mediation effect
• Independent contribution to R-square

13
HIERARCHICAL REGRESSION

• Sample-focused procedures
• Forward regression
• Backward regression
• Stepwise regression
• Criteria may include R-square change in sample,
  error reduction

14
STATISTICAL TESTING Single additional predictor

• R-square change F-test for increase in SS per
  predictor in relation to MSerror for complete
  model
• F (1,dfe) (SSAB SSA )/ MSeAB

A
B
Y
A
byB
SSe
B
Y
t byB / sebyB
15
STATISTICAL TESTING Sets of predictors

• R-square change F-test for increase in SS per p
  predictors in relation to MSerror for complete
  model
• F (p,dfe) ((SSAB SSA )/p)/ MSeAB

Y
A
B is a set of p predictors
SSe
B
16
Experimentwise Error Rate

• Bonferroni error rate ptotal lt p1 p2 p3
• Allocate error differentially according to
  theory
• Predicted variables should have liberal error for
  deletion (eg. .05 to retain in model)
• Unpredicted additional variables should have
  conservative error to add (eg. .01 to add to
  model)