Quantitative versus Qualitative Is a Distraction: Variations on a Theme by Brewer

1
Quantitative versus Qualitative Is a Distraction
Variations on a Theme by Brewer Hunter (2006)

• Methods _at_ Plymouth, 2007
• W. Paul Vogt

2
The Theme

• Focusing on the distinction between quantitative
  and qualitative, and on methods of mixing them,
  ignores a wider range of methodological problems
  and opportunities to solve them.

3
The Variations (Overture)

• To think in terms of quant and qual designs is a
  category mistake.
• The q-q distinction diverts attention from other
  kinds of multi-method research.
• It leads to stereotyping and tribalism among
  researchers.
• It encourages us to accept our weaknesses.
• It ignores the relation between indicators and
  concepts.
• It is based on confusion about the nature of
  thinking.
• It diverts attention from the nature of the
  phenomena being studied.
• It underemphasizes the extent to which
  researchers routinely alternate between
  categorical and continuous variables.
• It distracts us from other ways of thinking and
  handling data, particularly graphic ones.
• Mixed methods, by treating the q-q distinction
  as though it were the most important one, may
  have the paradoxical effect of reinforcing
  categories better abandoned or deemphasized.

4
1. Thinking in terms of quant and qual designs is
a category mistake.

• All designs can accommodate quant and/or qual
  data. This is an empirical claim. It is easy to
  find examples of quant or qual data collected
  using each of the main design types document
  analysis, secondary analysis, naturalistic
  observation, surveys, interviews, experiments,
  participant observation.

5
2. The quant-qual distinction deflects attention
from other multi-method work.

• Document analysis and interviews (both often
  verbal)
• Field observations coded numerically and
  laboratory experiments coded numerically
• Surveys and focus groups

6
3. It leads to stereotyping and tribalism among
researchers.

• Correlations and tendencies are confused with
  logical entailment.
• Tashakkori Teddlie (1998) can illustrate.
• Researchers using quantitative methods make
  time-free and context-free generalizations. The
  number of exceptions is huge, e.g., sociologists
  who use quant data to study social context, or
  social historians who use quant data to show
  change over time.
• Researchers using qualitative methods do not
  believe it is possible to discuss causes. Again,
  the number of counter examples is huge. Causal
  analysis that does not rely on quant data is
  venerably old.
• Gibbons Decline and Fall
• Max Webers Protestant Ethic
• Durkheims Division of Labor

7
4. It encourages us to accept our weaknesses
or provides an excuse for not correcting them

• Social psychologists have shown how easy it is
  for biased side-taking to emerge in social
  situations.
• Ignorance of, say, grounded theory or of
  regression analysis can be a mark of cultural
  status among researchers.

8
5. The important relation of indicators to
concepts is not addressed

• Is the indicator a symptom? a cause? a
  component? a predictor?
• Law and morality a la Durkheim
• Democracy ? fair elections, separation of
  religion and state, free speech
• Principal components analysis and factor analysis
  (e.g., health and intelligence)

9
6. It leads to confusion about the nature of
thought.

• Numbers are words.
• One of the most basic of language distinctions,
  in all languages, is the distinction between
  singular and plural, one and many, which is
  fundamentally a quantitative distinction.
• Even the most verbal-intensive research contains
  relations of quantitysuch as more or less, mild
  or strong, and so on.
• Even the most number-intensive research contains
  qualitative distinctions cause, influence,
  predict, present/absent, and so on.
• Rank order relationships are the somewhat
  unrecognized meeting ground of qualitative and
  quantitative data.
• Researchers using quantitative data often split
  continua into categories, such as high, medium,
  or low on the self-efficacy scale.

10
7. It diverts attention from the nature of the
phenomena being studied.

• Height of Mendels peas (cat or cont?).
• Could evolution could accommodate continuous
  variation or did it require discontinuous
  variation?
• The battle mattered because it concerned the
  nature of the phenomena it had to be resolved
  on substantive grounds. The issue was the nature
  of reality, not how to code it.

11
8. Researchers often alternate between
categorical and continuous variables

• Falklands War and First Gulf War and suicide
  rates in the U.K. and U.S.
• recessions and the ability to afford university
  tuition in the U.S.
• Quantifying subjective judgments Pain
• Gleason scoring of tumors. Here we have the
  subjective judgments of physicians about objects.
  
• interrater reliabilityinter-subjective.
• Quantified subjective judgments are used to make
  a categorical decision about treatment.
• To talk about whether this kind of analysis and
  decision making is quant or qual is irrelevant.
  It must inevitably be both.

12
9. It distracts from other ways of thinking
about data, particularly graphic.

• Venn diagrams are essential for thinking about
  overlapping categories.
• Change over time in quantitative data is often
  hard to describe without line graphs.
• SEM without graphic means of depicting the
  measurement and causal models.
• Causal models are almost inevitably graphicnot
  quant, not qual, but graph.

13
10. Mixed methods may not help

• Mixed methods may have the paradoxical effect of
  reinforcing the categories they were meant to
  bridgecategories that in many contexts are
  better abandoned.

14
Conclusions

• The characteristics of ones evidence and how one
  codes it will constrain ones analysis
  strategiesat some point. This point should come
  relatively later, not earlier. It should be one
  of the later, not one of the earlier, branches in
  the decision tree.
• What should come earlier?

15
Prior Questions 1

• How can I best gather the kind of evidence I need
  to answer my research question? (Design)
• Who or what do I sample or select for study using
  that design? (Sampling)
• What are the ethical implications of those design
  and sampling choices? (Ethics)
• Only then Should I code my data using words,
  numbers, pictures, or all three?

16
Prior Questions 2

• Questions about research questions that have
  priority over the quant-qual distinction include
• Does the problem involve making causal
  inferences?
• Is it necessary to generalize from the cases
  studied (sample) to a broader group (population)?
  
• Does the problem include the study change over
  time?
• Is it necessary to interact with
  subjects/participants?
• Must one find ones own data sources and/or
  generate ones own data?
• Is it necessary to use more than one design?