Marrying Qualitative With Quantitative Analysis

1
Marrying Qualitative With Quantitative Analysis

• John J. Usera, Ph.D.
• Institute for Educational Leadership Evaluation
• Chiesman Center
• Rapid City ? Pierre ? Sioux Falls
• Jusera_at_chiesman.org

2
TALKING POINTS

• What is Mixed Methods Evaluation?
• Four Types of Mixed Methods
• Analyzing Data
• Questions About Mixed Methods

3
MIXED METHODS DEFINED

• Involves philosophical assumptions that guide the
  direction of the collection and analysis of data
• Considers the mixture of qualitative and
  quantitative approaches in the different phases
  of an evaluation process.
• Focuses on the idea that a mixed approach will
  provide a better understanding and solution to
  the evaluation questions and hypotheses.

4
MIXING THE DATA
Merging the data
Qualitative Data
Results
Quantitative Data
Connecting the data
Qualitative Data
Quantitative Data
Results
5
MIXING THE DATA
Embedding the data
Quantitative Data
Results
Qualitative Data
Phasing the data
Quantitative Data
Qualitative Data
Phase 1
Phase 2
Results
6
MIXED METHODS DESIGNS

• Triangulation
• Embedded
• Explanatory
• Exploratory

View complete resume as a Word document
7
TRIANGUALATION DESIGNConvergence Model
Quantitative
Data Collection
Data Analysis
Results
Compare Contrast
Data Collection
Data Analysis
Results
Interpretation
Qualitative
8
TRIANGUALATION DESIGNTransformation Model
Quantitative
Compare Contrast As two quantitative Data sets
Data Collection
Data Analysis
Interpretation
Data Collection
Data Analysis
Transform into quantitative data
Qualitative
9
TRIANGUALATION DESIGNValidation Model
Quantitative
Data Collection (Survey)
Data Analysis
Results
Validate results Supporting evidence
Data Collection(Open-ended Survey items)
Data Analysis
Results
Interpretation
Qualitative
10
EMBEDDED DESIGNExperimental Model
Qualitative During Intervention
Qualitative data before intervention
Qualitative Data after intervention
Intervention
Quantitative postmeasure
Quantitative premeasure
Interpretation based On results
11
EMBEDDED DESIGNCorrelational Model
Quantitative Predictors
Qualitative Data collection processes
Quantitative Outcome
Quantitative Predictors
Quantitative Predictors
Interpretation based On the results
12
EXPLANATORY DESIGN

• Two phase mixed methods design
• Qualitative data helps explain or build upon
  initial quantitative results (Creswell, Plano,
  Clark, et al., 2003)
• Helpful in explaining outlier or surprising
  results (Morse, 1991)
• Use quantitative participant characteristics to
  guide purposeful sampling for a qualitative phase

13
EXPLORATORY DESIGN

• The results of the first method (qualitative) can
  help develop or form the second method
  (quantitative) (Greene et al., 1989)
• Exploration is needed because
• Measures or instruments are not available
• The variables are unknown
• There is no guiding framework or theory
• This design is excellent for phenomenological
  type of evaluation or for identifying variables
  to study quantitatively.

14
EXPLORATORY DESIGNInstrument Development Model
Quantitative
Data Collection
Data Analysis
Results
Develop Instrument
Interpretation
Data Collection
Data Analysis
Results
Qualitative
15
ANALYZING DATA

• Preparation of the data for analysis
• Exploring the data
• Analyzing the data
• Representing data analysis
• Validating the data

16
PREPARATION

• Coding
• Cleaning
• Recoding
• Establishing the codebook

• Transcribing
• Coding
• Categorization
• Labeling

Qualitative
Quantitative
17
EXPLORING

• Visual inspection
• Descriptive analysis
• Checking for trends and distributions

• Reading through the data
• Making notes
• Develop a qualitative codebook

Quantitative
Qualitative
18
ANALYZING THE DATA

• Statistical tests
• Answering evaluation questions
• Testing hypotheses
• Report effect sizes, alphas, p values, etc.

• Find interrelating themes
• Abstract into smaller set of themes if necessary
• Qualitative software wherever possible

Quantitative
Qualitative
19
REPRESENTING DATA

• Represent results in statements of results
• Provide appropriate tables
• Use graphics where feasible

• Represent findings in discussions of themes or
  categories
• Present visual models, figures and tables

Qualitative
Quantitative
20
VALIDATION

• Use external standards
• Validate using other studies and sources
• Check for reliability of scores and current data

• Use appropriate standards
• Use the tested validation strategies especially
  triangulation, peer review, etc.

Qualitative
Quantitative