How to Fool Yourself with SEM

1
How to Fool Yourself with SEM

• James G. Anderson, Ph.D
• Purdue University

2
Tripping at the Starting Line Specification

• Specify the model after the data are collected
  rather than before.
• Omit causes that are correlated with other
  variables in a structural model.
• Fail to have sufficient numbers of indicator
  latent variables.
• Use psychometrically inadequate measures.

3
Tripping at the Starting Line Specification (2)

• Fail to give careful consideration to the
  question of directionality.
• Specify feedback effects in structural models as
  a way to mask uncertainty about directionality.
• Overfit the model (e.g., forget the goal of
  parsimony.

4
Tripping at the Starting Line Specification (3)

• Add disturbance or measurement error correlations
  without substantive reason.
• Specify that indicators load on more than one
  factor without substantive reason.

5
Improper Care and Feeding Data

• 10. Dont check the accuracy of data input or
  coding.
• Ignore whether the pattern of data loss is random
  or systematic.
• Fail to examine distributional characteristics.
• Dont screen for outliers.
• Assume that all relations are linear.

6
Checking Critical Judgment at the Door Analysis
and Re-specification

• Re-specify a model based entirely upon
  statistical criteria.
• Fail to check the accuracy of your programming.
• Analyze a correlation matrix when it is
  inappropriate.
• Analyze variables so highly correlated that the
  solution is unstable.

7
Checking Critical Judgment at the Door Analysis
and Re-specification (2)

• Estimate a very complex model with a small
  sample.
• Set scales for latent variables inappropriately.
• Ignore the problem of starting values or provide
  grossly inaccurate ones.
• When identification status is uncertain, fail to
  conduct tests of solution uniqueness.

8
Checking Critical Judgment at the Door Analysis
and Re-specification (3)

• 23.Fail to recognize empirical under-identificatio
  n.
• 24. Fail to separately evaluate the measurement
  and structural portions of a hybrid model.

9
The Garden Path Interpretation

• Look only at indices of overall fit and ignore
  other types of fit information.
• Interpret goodness-of-fit as meaning that the
  model is proved.
• Interpret goodness-of-fit as meaning that the
  endogenous variables are strongly predicted.
• Rely too much upon significance tests.

10
The Garden Path Interpretation (2)

• Interpret the standardized solution in
  inappropriate ways.
• Fail to consider equivalent models.
• Fail to consider alternative models.
• Reify Factors

11
The Garden Path Interpretation(3)

• 33. Believe that a strong analytical method like
  SEM can compensate for poor study design or poor
  ideas.
• 34. As the researcher, fail to report enough
  information so that your readers can reproduce
  your results.
• 35. Interpret estimates of large direct effects
  from a structural model as proof of causality.

12
Reference

• R.B. Kline, Principles and Practice of Structural
  Equation Modeling, NY Guilford Press, 1998.