Exploratory Factor Analysis

1
Exploratory Factor Analysis
2
Review FA vs. PCA

• Principal components analysis seeks linear
  combinations that best capture the variation in
  the original variables.
• Factor analysis seeks linear combinations that
  best capture the correlations among the original
  variables.

3
Review FA vs. PCA
Principal Components Analysis The correlation
matrix contains ones on the main diagonal,
accounting for all of the variance in Xthe goal
of principal components analysis.
Factor Analysis Seeks to identify common factors
that influence correlations among items, it is
not a correlation matrix that is analyzed ? thus,
the main diagonal is replaced by the
communalities - the variances in X that are due
only to the common factors. But, communalities
are not known in advance of the factor analysis,
giving rise to the communality problem and the
need to solve for the common factors iteratively.
4
FA Principal Axes Factoring
In the principal axes approach to factor
analysis, the only difference compared to
principal components is that the matrix being
analyzed is a correlation matrix (which is also a
variance-covariance matrix for standardized
variables) in which the main diagonal contains
the communalities rather than the variances.
5
Simple Structure

• Most of the loadings on any given factor are
  small and a few loadings are large in absolute
  value
• Most of the loadings for any given variable are
  small, with ideally only one loading being large
  in absolute value.
• Any pair of factors have dissimilar patterns of
  loadings.

6
The ideal loadings are the simple structure,
which might look like this for the factor
loadings To get closer to the ideal, we rotate
the factors
7
Varimax One way to approach
this ideal pattern is to find the rotation that
maximizes the variance of the loadings in the
columns of the factor structure matrix. This
approach was suggested by Kaiser and is called
varimax rotation.
8
Quartimax A second way to
approach this ideal pattern is to find the
rotation that maximizes the variance of the
loadings in the rows of the factor structure
matrix. This approach is called quartimax
rotation.
9
Equimax For those who want
the best of both worlds, equimax rotation
attempts to satisfy both goals. Varimax is the
most commonly used and the three rarely produce
results that are very discrepant.
10
Example Principal Components Analysis
Data on drug use reported by 1634 students in Los
Angeles. Participants rated their use on a
5-point scale 1 never tried, 2 only once, 3
a few times, 4 many times, 5 regularly.

11
Syntax for Factor Analysis
FACTOR /VARIABLES cigs beer wine liquor cocaine
tranqs drugstr heroin marijuan hashish inhale
hallucin amphets /PRINT INITIAL KMO EXTRACTION
ROTATION /PLOT EIGEN ROTATION /CRITERIA
MINEIGEN(1) ITERATE(25) /EXTRACTION PC
/CRITERIA ITERATE(25) /ROTATION VARIMAX
/METHODCORRELATION .
KMO asks for a test of multicollinearity this
will tell whether there is enough shared
variance to warrant a factor analysis
PC Principal Components PAF Principal Axis
Factoring
12
Should the matrix be analyzed?
Bartlett's test indicates the correlation matrix
is clearly not an identity matrix
This can vary between 0 and 1, indicating whether
there is sufficient multicollinearity to warrant
an analysis. Higher values indicate the
desirability of a principal components analysis.
13
Two principal components account for nearly half
of the information in the original variables
14
Initial Extraction All principal components are
extracted so all of the variance in the variables
is accounted for.
15
Part of the loading matrix. How should the first
two principal components be interpreted?
16
Only two principal components are indicated by
the scree test.
17
Only two principal components are indicated by
the Kaiser rule as well.
18
With only two principal components, much less of
the variance in each variable is accounted for.
19
The loadings for the first two components do not
change when they are the only ones extracted.
20
Common Factor Analysis
Drug use data by 1634 students. Participants
rated their use on a 5-point scale 1 never
tried, 2 only once, 3 a few times, 4 many
times, 5 regularly.
21
The analysis begins in the same way as principal
components analysis. It would make little sense
to search for common factors in an identity
matrix
22
Unlike principal components analysis, factor
analysis will not attempt to explain all of the
variance in each variable. Only common factor
variance is of interest. This creates the need
for some initial estimates of communalities.
23
The number of factors to extract is guided by the
size of the eigenvalues, as it was in principal
components analysis. But, not all of the variance
can be accounted for in the variables.
24
Remember, this is because PCA assumes no error of
measurement, whereas FA assumes there is
measurement error.
To the extent there is random error in the
measures, the eigenvalues for factor analysis
will be smaller than the corresponding
eigenvalues in principal components analysis.
25
The location of the factors might be rotated to a
position that allows easier interpretation. This
will shift the variance, but preserve the total
amount accounted for.
26
Two factors appear to be sufficient. The
attenuated eigenvalues will generally tell the
same story.
27
The loadings will be reduced in FA compared to
PCA because not all of the variance in X is due
to common factor variance.
PCA
FA
28
Rotating the factors to simple structure makes
the interpretation easier. The first factor
appears to be minor recreational drug use. The
second factor appears to be major abusive drug
use.
29
New Example FA
A sample of 303 MBA students were asked to
evaluate different makes of cars using 16
different adjectives rated on a 5-point agreement
scale (1 strongly disagree, 5 strongly
agree) This car is an exciting car.
30
Are all 16 individual ratings required to
understand product evaluation, or, is there a
simpler measurement model?
31
The analysis must begin with some idea of the
proportion of variance in each variable that can
be attributed to the common factors. The most
common initial estimate is the squared multiple
correlation between a given measure and all the
remaining measures.
32
Three common factors appear to underlie the 16
evaluative ratings.
33
The three common factors account for two-thirds
of the common factor variance.
34
Without rotation, the interpretation of the
factors is not readily apparent, especially the
second and third factors.
35
The varimax rotation makes the interpretation a
bit easier. What would you name these factors?
Factor 1?
Factor 2?
Factor 3?
36
It is rare for the different rotational criteria
to produce different results
37
PCA vs. FA

• Advocates of factor analysis often claim that it
  is inappropriate to apply principal components
  procedures in the search for meaning or latent
  constructs.
• But, does it really matter all that much?
• To the extent that the communalities for all
  variables are high? the two procedures should
  give very similar results.
• When commonalities are very low ? factor
  analysis results may depart from principal
  components.

38
PCA
FA
39
PCA
FA
40
PCA
FA
41
FA Oblique Rotation

• Factor analysis offers a more realistic model of
  measurement than principal components analysis by
  admitting the presence of random and systematic
  error.
• Another way to make the model more realistic is
  to relax the restriction that factors be
  orthogonal. Allowing oblique factors has two
  potential benefits
• It allows the model to better match the actual
  data
• It allows the possibility of higher order
  factors

42
FA Example, Oblique Rotation
A sample of 303 MBA students were asked to
evaluate different makes of cars using 16
different adjectives rated on a 5-point agreement
scale (1 strongly disagree, 5 strongly
agree) This car is an exciting car.
43
(No Transcript)
44

• Oblique rotation relaxes the requirement that
  factors be independent. This requires the
  addition of a new matrixthe factor pattern
  matrix.
• With orthogonal rotation, because the factors
  are independent, the weights are simply
  correlations---just the elements of the factor
  loading (structure) matrix.
• When the factors are not orthogonal, the
  correlations are not the only weights considered
  (structure matrix) and a separate matrix
  containing partial weights is necessary (pattern
  matrix).

45
Because the factor pattern matrix represents the
unique contribution of each factor to the
reconstruction of any variable in X, it provides
a better basis for judging simple
structure. Available techniques for achieving
simple structure for oblique rotation (e.g.,
promax, direct oblimin) confront an additional
problem---the specification for the amount of
correlation among the factors. Unlike orthogonal
rotation in which these correlations are fixed at
0, in oblique rotation, simple structure can be
sought for any correlations among the factors.
46
Hypothetical data (N 500) were created for
individuals completing a 12-section test of
mental abilities. All variables are in standard
form.
47
The correlation matrix is not an identity matrix
48
The scree test clearly shows the presence of
three factors
49
On average, the three factors extracted can
account for about half of the variance in the
individual subtests.
50
Factor analysis accounts for less variance than
principal components and rotation shifts the
variance accounted for by the factors.
Why does the sum of the squared rotated loadings
not equal the sum of the squared unrotated
loadings?
51
The initial extraction . . .
52
Oblique rotation is much clearer in the pattern
matrix than in the structure matrix
53
The correlations among the factors suggest the
presence of a higher order factor
Why are some of the correlations negative when
one would expect all mental abilities to be
positively correlated?
54
An alternative oblique rotation---Promax---provide
s much the same answer
The order of the factors may vary, and one gets
reflected, but the essential interpretation is
the same.
55
The correlations among the factors are similar
for the two procedures.
56
Exploratory factor analytic methods are sometimes
used as a crude way to confirm hypotheses about
the latent structure underlying a data set. As a
first pass, these methods do just fine. But, more
powerful confirmatory factor analytic procedures
exist that can better address questions about
data that are strongly informed by theory.