All Things in Moderation, Including Tests of Mediation

1
All Things in Moderation, Including Tests of
Mediation

• Jeffrey B. Vancouver
• Charlie M. Thompson
• Bruce Carlson

2
Abstract

• Covariance tests of mediation are questioned
  regarding their internal validity. Specifically,
  eight alternative explanations for a finding of
  mediation are described. An alternative method
  using a moderation design is proposed.
  Implications regarding the potential use, and
  advantages and disadvantages of this approach are
  described.

3
Exogenous/
Endogenous/
Mediator
Independent
Dependent
a
Exogenous/
Endogenous/
Independent
Dependent
Moderator
b
Figure 1. Mediator and moderator models.
4
Introduction

• Few concepts confuse researchers more readily
  than the distinction between mediation and
  moderation.
• Mediation refers to a process or mechanism
  through which one variable (i.e., exogenous)
  causes variation in another variable (i.e.,
  endogenous) (see Figure 1a).
• Moderation refers to the influence of a process
  or mechanism on the degree or kind of
  co-variation between the exogenous and endogenous
  variables (see Figure 1b).
• Conceptually and analytically important
• Mediation is analyzed via differences in partial
  and whole covariances.
• Moderation is analyzed via interactions.

5

• We argue that studies designed to test for
  moderation may provide stronger tests of
  mediation than the partial and whole covariance
  approaches typically used (e.g., Baron Kenny,
  1986 Bing, Davison, LeBreton, LeBreton, 2002
  James Brett, 1984).
• Problem
• The fundamental issue is the weak internal
  validity of passive observational methods (Runkel
  McGrath, 1972 Shadish, Cook, Campbell,
  2002).
• Specifically, the issue is that the covariance
  methods for testing mediation are susceptible to
  numerous alternative explanations (Shadish, et
  al., 2002).
• Figure 2 shows alternative models when the
  exogenous/independent variable is manipulated.
• Figure 3 shows alternative models when all
  variables are passively observed.

6

• Incidence of problem
• Use of the covariance methods are widespread,
  particularly in the applied psychological
  literature (Bing, et al., 2002 MacKinnon et al.,
  2002),
• whereas use of the moderation method, either
  experimentally or passively, to assess mediation
  is rare
• Figure 4 shows the results of a review of the
  Journal of Applied Psychology from 1999 through
  2002.
• Potential solution
• Use moderation (especially manipulated) to assess
  hypothetical mediators (see Figure 5).
• Consider automotive science example.

7
Alternatives Explanations for Positive Test of
Mediation
Exogenous/
Endogenous/
Mediator
Independent
Dependent
A. "Endogenous" variable causes "mediator"
8
Mediator
Exogenous/
Independent
Endogenous/
Dependent
B. "Exogenous" variable causes "mediator" and
endogenous variable independently.
Mediator
Exogenous/
Independent
Endogenous/
Dependent
C. "Exogenous" variable causes "mediator" more
quickly then it causes the endogenous
variable.
Figure 2. Alternative models when the
exogenous/independent variable is manipulated.
9
Endogenous/
Exogenous/
Mediator
Dependent
Independent
A. "Endogenous" variable causes the "exogenous"
variable, which causes the "mediator".
Endogenous/
Exogenous/
Mediator
Dependent
Independent
B. The mediator mediates effect of the
"endogenous" variable on the "exogenous" variable.
Endogenous/
Exogenous/
Mediator
Dependent
Independent
C. The mediator causes the "endogenous"
variable, which causes the "exogenous variable.
10
Exogenous/
Endogenous/
Mediator
Independent
Dependent
D. The mediator causes "exogenous" variable,
which causes the "endogenous" variable.
Mediator
Exogenous/
4th variable
Independent
Endogenous/
Dependent
E. Some fourth variable causes two or more of
the others in some order other than the one
hypothesized.
Figure 3. Additional alternative models when no
variable is manipulated.
11
280 articles
99 (35) tested for moderation
76 (27) tested for mediation
19
mentioned the moderator implied a mediator
Figure 4. Incidence of tests of mediation and
moderation in the Journal of Applied Psychology
(volumes 84-86).
12
The Automotive Science Example
Research Question Why does pressing the gas
pedal increase speed of car?
Hypothesis 1 Drive train mediates relationship
between pedal and speed.
Hypothesis 2 Alternator mediates relationship
between pedal and speed.
Results of standard covariance approach to tests
of mediation
Both hypotheses are supported.
Moderation research designs
Hypothesis 3 Wrench in drive train will affect
relationship between pedal and speed.
Hypothesis 4 Disconnecting alternator from car
will affect relationship.
Results of moderation tests
Hypothesis 3 was supported but Hypothesis 4 was
not!
Conclusion Drive train is a mediator but
alternator is not.
13
Exogenous/
Endogenous/
Mediator
Independent
Dependent
Moderator
Figure 5. Test of mediation using moderation.
14
Issues

• Tests of mediation through manipulated moderation
  are not fool-proof.
• Confounds in manipulation can lead to invalid
  interpretations.
• No manipulation possible or possible manipulation
  weak.
• Tests of mediation through passive observation
  are likely to be . . .
• heavily confounded or
• analytically suspect (power, analysis issues).
• Nonetheless,
• manipulation studies are generally more
  internally valid,
• require clever designs (e.g., task load paradigm
  to moderate attentional resources available), and
• represent many of the issues I/O Psychologists
  find important
• E.g., why do context/job dimensions matter?
• Or, when are they likely to matter given the
  underlying mechanisms?

15
Conclusions

• We are not suggesting that covariance techniques
  for assessing mediators should be abandoned.
  There will always be cases where it is the only
  viable choice or other considerations (e.g.,
  external validity) suggest favoring the approach.
  Rather, our objectives are threefold
• First, we wanted to add some humility to the
  discussion sections of covariance mediator
  studies.
• Second, we wanted to motivate researchers to
  develop manipulations or measures of moderators
  that might allow for cleaner inferences of
  mediation.
• Finally, we wanted to increase the likelihood
  that researchers will consider the understandings
  their findings of moderation might provide
  regarding mediating mechanisms.

16
References

• Baron, R. M. Kenny, D. A. (1986). The
  moderator-mediator variable distinction in social
  psychological research Conceptual, strategic,
  and statistical considerations. Journal of
  Personality and Social Psychology, 51, 1173-1182.
• Bing, M. N., Davison, H. K., LeBreton, D. L.,
  LeBreton, J. M. (2002, April). Issues and
  improvements in tests of mediation. Poster
  presented at the annual meeting of the Society
  for Industrial and Organizational Psychology,
  Toronto, Canada.
• James, L. R. Brett, J. M. (1984). Mediators,
  moderators and tests of mediation. Journal of
  Applied Psychology, 69, 307-321.
• MacKinnon, D. P.., Lockwood, C. M., Hoffman, J.
  M., West, S. G., Sheets, V. (2002). A
  comparison of methods to test mediation and other
  intervening variable effects. Psychological
  Methods, 7, 83-104.
• Runkel, P. J. McGrath, J. E. (1972). Research
  on human behavior A systematic guide to method.
  New York Holt, Rinehart, and Winston.
• Shadish, W, R, Cook, T. D. Campbell, D. T.
  (2002). Experimental and Quasi-Experimentation
  Designs for generalized causal inference.
  Boston Houghton Mifflin.