Using Principal Component Analysis

1
Using Principal Component Analysis

• Jennifer Kahle
• University of South Florida

2
What is PCA?

• Variable reduction procedure
• Use with several correlated variables (measuring
  the same construct)
• Sets of similar items, representing the same
  underlying factor are meaningfully formed
• Research uses
• Scale development (e.g. advocacy scale- Johnson
  1993 Mason and Levy 2002)
• Use of factor loadings in subsequent analysis
  (Cuccia and McGill 2000)

3
PCA vs. FA

• Both PCA and FA are data reduction techniques
• Similar process and generally, similar results
• PCA analyzes all the variance in the observed
  variables (uses the full correlation matrix)
• Unique, shared, and error variance
• Used primarily for decisions about the number of
  variables
• FA analyzes the shared variance
• Assumes underlying latent variable exerts causal
  influence on the observed variables

4
PCA

• A PC provides a linear combination of optimally
  weighted observed variables
• PC1 b1(X1) b2(X2)bp(Xp)
• Each successive factor (component)...
• accounts for maximum available variance remaining
• is orthogonal (uncorrelated) with all prior
  factors

5
Steps in PCA

• Extract a full components solution
• Eigenvalues represent total variance accounted
  for by each PC
• Standardized, so total variance total of
  variables
• Determine of factors to retain
• Retain factors that extract meaningful variance

6
Kaisers gt1 criterion
7
Cattells scree criterion
8
Factor Pattern Matrix

• Bivariate correlation of each item with other
  variables and with factors identified
• Highly correlated variables should load on the
  same factor
• Use to determine variables to keep/drop within
  each factor

9
Factor Pattern Matrix
If gt.40 Retain
10
Rotation

• When more than one factor, rotate the components
• Linear transformation-forces the solution to be
  more interpretable
• Tries to attain Simple Structure
• Each variable loads highly on only one factor and
  loads lowly on the others
• Most common rotation for PCA is varimax
• VARIance MAXimization within a factor, (forces to
  simple structure)
• Orthogonal axes of rotation.

11
Factor Pattern MatrixMore than one component
12
Interpret

• Decisions about where a variable best fits, if at
  all
• Interpret (name) our factors
• Guidelines
• At least 3 variables load on the component
• When creating scales, start with more!
• Variables share conceptual meaning
• Simple structure

13
Apply the solution

• Apply components solution
• Theoretically -- understand the meaning of the
  data reduction
• Advocacy Scale (Johnson 1993, Mason and Levy
  2002)
• Statistically -- use the component scores in
  other analyses
• Cuccia and McGill (2000)
• Used PC factor score (18 item advocacy scale) as
  a covariate