Method Variance

1
Method Variance

• Lauralee Flores

2
What is method variance?

• Method variance concerns variance in measurement
  attributed to the particular instrumentation
  rather than to the construct of interest
  (Spector 1987).
• Instrument or method-specific bias
• all self-report measures inter-correlate at
  .30 (Spector 1987)

3
Background

• In 1959 Campbell and Fiske developed the
  Multitrait-Multimethod Matrix.
• This matrix allows one to determine the
• Content validity (and thus the convergent and
  discriminant validity)
• Reliability
• and Method variance of a set of measures in a
  study.

4
Background, cont.

5
Background, cont.

• In the MMTM Method Variance can be detected when
• Multiple Traits
• Mono-method
• Producing higher correlation than
• Multiple Traits
• Multiple Methods

6
The three views

• Spector (1987)
• There is little evidence of method variance
• Williams et al. (1987)
• Method variance is present and accounts for
  substantial variance in the measures originally
  examined by Spector.
• Bogazzi and Yi (1990)
• Method variance in these studies is more
  prevalent than Spector concluded but less
  prevalent than Williams et al. asserted.

7
Spectors perspective

• Method variance may be a result of
• Halo bias in performance appraisals raters tend
  to rate each ratee the same across different
  dimensions of performance.
• Acquiescence the tendency for a respondent to
  agree (or disagree) with items, regardless of
  content.

8
Spectors perspective, cont.

• Method variance may also be a result of
• Social Desirability the tendency for a
  respondent to choose the socially desirable
  response, regardless of the veracity of that
  response.
• Confirmational reasoning

9
Bogazzi and Yis perspective

• First investigate the important and often
  ignored issues of method variance.
• Second correct any eroneous conclusions about
  method variance in and construct validity of the
  data examined by Spector (1987) and Williams et
  al. (1987).

10
Bogazzi and Yis perspective, cont.

• Third Show that the analysis and interpretation
  of MTMM data are not straightforward endeavors
  but require a careful, detailed consideration of
  many criteria for model specification, goodness
  of fit, and other statistical findings.
• Seems to be much more interested in the truth
  rather than have their own agenda.

11
Tepper and Teppers perspective

• Tepper and Tepper (1993) looked specifically at
  method variance in within-subject research.
• Their basic conclusion was that method variance
  can largely affect research conclusions. Despite
  the inflammation of correlations in research,
  method variance is often ignored.
• They in turn give some suggestions for scale
  development to help control for method variance.

12
Conclusion

• Research interpretation and conclusions are, in
  large part, a product of the researcher.
• Spector and Williams et al. interpreted basically
  the same data in two completely different ways.
• Method variance isnt a big issue largely due
  to the fact that it increases the correlation
  coefficient and thus significant research
  findings.

13
Conclusion,cont.

• Reaffirmed the notion that training and prior
  research affect future research.
• So much research has been done that completely
  ignores method variance and its consequences.
  Therefore, its still an issue today.
• Properly training a future scientist is the first
  step in increasing the likelihood that they will
  control for method variance in their research.

14
The end

• .