Collinearity

1
Collinearity

• The Problem of Large Correlations Among the
  Independent Variables

2
Skill Set

• What is collinearity?
• Why is it a problem?

• How do I know if Ive got it?
• What can I do about it?

3
Collinearity Defined

• Within the set of IVs, one or more IVs are
  (nearly) totally predicted by the other IVs.
• In such a case, the b or beta weights are poorly
  estimated.
• Problem of the Bouncing Betas.

4
Diagnostics
1. Variance Inflation Factor (VIF).
Standard error of the b weight with 2 IVs
Sampling Variance of b weight
VIF
5
VIF (2)
Standard Error with k predictors
Large values of VIF are trouble. Some say values
gt 10 are high.
6
Tolerance
Tolerance is
Small values are trouble. Maybe .10?
7
Condition Index
Lambda is an eigenvalue.
 
Number refers to a linear combination of the
predictors. Eigenvalue refers to the variance of
that combination.
Collinearity is spotted by finding 2 or more
variables that have large proportions of variance
(.50 or more) that correspond to large condition
indices. A rule of thumb is to label as large
those condition indices in the range of 30 or
larger. No apparent problem here.
8
Condition Index (2)
The last condition index (15.128) is highly
associated with X2 and X3. The b weights for X2
and X3 are probably not well estimated.
9
Dealing with Collinearity

• Lump it. Admit ambiguity SE of b weights.
  Refer also to correlations.
• Select or combine variables.
• Factor analyze set of IVs.
• Use another type of analysis (e.g., path
  analysis).
• Use another type of regression (ridge
  regression).
• Unit weights (no longer regression).

10
Review

• What is collinearity?
• Why is collinearity a problem?
• What is the VIF?
• What is Tolerance?
• What is a condition index?
• What are some things you can do to deal with
  collinearity?