Why do we need mixed-methods? Should we differentiate integration versus mixed-methods?

1
Why do we need mixed-methods? Should we
differentiate integrationversus mixed-methods?

• Alan Bryman,
• Management Centre,
• University of Leicester, UK.

2
The current situation

• Exciting times
• Paradigm wars
• Epistemological arguments
• Pragmatism
• Washing machines and questionnaires!
• Not addressing epistemological issues today

3
My Research on Multi-Strategy Research

• Funded by Economic Social Research Council
  Research Methods Programme
• 2 main strands today
• Traditional content analysis of journal articles
  using mixed-methods research
• Interviews with social researchers whove used it

4
Research methods

• Wanted to map general characteristics of
  mixed-methods research content analysis
• Wanted perspectives of mixed-methods researchers
  contingencies involved semi-structured
  interviews

5
Content analysis

• Searched Social Sciences Citation Index for
  quantitative and qualitative triangulation
  multi(-)method mixed method in titles and
  abstracts of English language articles
• 5 disciplines sociology social psychology
  organizational behaviour human and cultural
  geography media and cultural studies
• 1994-2003
• 232 articles

6
Content analysis

• Articles foregrounded mixed-methods
• Emphasis on mixed-methods in terms of data
  collection and analysis
• Major focus on rationales for mixed-methods
  research

7
Classifying Forms of Mixed-Methods Research

• Distinction between rationale and practice
• Rationale stated purpose(s) of integrating
  quantitative and qualitative research
• Practice actual use(s) made of integrating
  quantitative and qualitative research
• Used both Greene et al. scheme and my own
  grounded scheme

8
Classifying Forms of Mixed-Methods Research

• Greene et al. (triangulation complementarity
  development initiation expansion)
• Parsimonious but only 2 rationales coded (primary
  secondary)
• Developed alternative scheme

9
Alternative Scheme

• Triangulation
• Offset
• Completeness
• Process
• Different research questions
• Explanation
• Unexpected results
• Instrument development

• Sampling
• Credibility
• Context
• Illustration
• Utility
• Confirm discover
• Diversity of views
• Enhancement
• Other/unclear/not stated

10
Highlights of Findings Rationale

• No rationale in 27 of all articles
• Main categories in terms of rationale
• Enhancement 32
• Completeness 13
• Sampling 13
• Triangulation 13

11
Highlights of Findings Practice

• Main categories in terms of practice
• Enhancement 52 (rationale 32)
• Triangulation 35 (rationale 13)
• Completeness 29 (rationale 13)
• Illustration 23 (rationale 2)
• Sampling 19 (rationale 13)

12
Highlights of Findings Rationale and Practice

• Rationale and practice not always in line
• Rationale often not reflected in how
  multi-strategy research actually used
• Practice often doesnt chime with rationales
  given
• Examples from contingency table analysis

13
Highlights of Findings Rationale and Practice
The Case of Triangulation

• Of the 29 articles citing triangulation as a
  rationale, 19 used it that way, i.e. one-third of
  articles citing triangulation as rationale didnt
  use multi-strategy research that way or didnt
  report doing so. Other prominent uses of articles
  citing triangulation were enhance (13)
  completeness (10) and illustration (8).

14
Highlights of Findings Rationale and Practice
The Case of Triangulation

• Other way around
• 80 articles used a triangulation approach but
  only 19 of them gave it as a rationale, i.e.
  three-quarters of articles using triangulation
  didnt cite it as a rationale
• Suggests triangulation hard to resist when
  opportunity arises

15
Highlights of Findings Rationale and Practice
The Case of Completeness

• Completeness was a rationale for 31 articles and
  84 of them used it that way
• But when practice is examined, 61 of all
  articles using a completeness approach didnt
  specify it as a rationale

16
Highlights of Findings Rationale and Practice
The Case of Enhancement

• 73 articles specified enhance as rationale, a
  quarter of them didnt use multi-strategy
  research this way
• 121 articles used multi-strategy research this
  way, but over half of them hadnt specified it as
  a rationale
• Several other examples of mismatches

17
Highlights of Findings Rationale and Practice

• Often mismatch between rationale and practice
• Mixed-methods research a moveable feast

18
Minority StrategyThe Gatling Gun Approach
19
Gatling Gun Strategy

• 4 or more rationales
• 6 articles in terms of rationale
• 33 articles in terms of practice

20
Themes from Semi-Structured Interviews

• Similar to content analysis
• Mixed-methods research increasingly expected
• Concern for many
• Research questions important
• Particularistic versus universalistic discourses
• Not due to confusion lack of guidelines about
  mixed-methods issues textbook account too
  simple ambivalence about role of research
  questions lack of prescription
• ? Uncertainty

21
What do we mean by mixing, etc.?

• Mixing vs. integration
• Use of verbs
• What does bringing together of quantitative and
  qualitative research entail?
• Mixed-methods research or multi-methodology/multi-
  method research?

22
Is Integration Occurring?Content analysis
findings

• Content analysis findings
• Genuine integration 18 of articles
• Parallel presentation 47 of articles
• Looked for evidence of findings being brought
  together ? comprehensive picture interweaving both

23
Is Integration Occurring?Semi-structured
interviews

• Most expressed concern. Main themes
• Different audiences
• Greater faith in one also familiarity
• Design issues
• Time-lines differ
• Skill specialisms
• One more striking or interesting
• Objectivist vs. constructionist accounts
• Journal publication issues

24
Bryman Goes Reflexive

• Mine was a mixed-methods project
• Justified using both content analysis and
  qualitative interviewing quite well
• Outcomes consistent with rationales
• Mixed-methods research linked to my research
  questions
• Integration of data not adequately achieved

25
Back to the Title
26
Why do we need mixed-methods?

• We often dont need it
• But difficult to decide when we do
• Good deal of uncertainty about when to use a
  mixed-methods approach

27
Should we differentiate integration vs.
mixed-methods?

• Dont get preoccupied with the verbs
• Interviewees saw problem
• Mixing findings like a conversation
• Forging an overall account
• Lack of templates

28
Should we differentiate integration vs.
mixed-methods?

• Tended to emphasize doing mixed-methods research
• Need more attention to representation of
  mixed-methods findings in publications
• Quality criteria