Current Methodological Issues in Single Case Research

1
Current Methodological Issues in Single Case
Research

• David Rindskopf, City University of New York
• Rob Horner, University of Oregon

2
Purposes

• Inclusion of Single-case Designs in education
  research literature.
• Interpretation of Single-case Research
• Visual Analysis
• Statistical Analysis
• Impact of Single-case Research
• Identification of Evidence-based Practices
• Inclusion of Single-case Research in Meta-analyses

3
Toward a Comprehensive Analysis of Single-case
Designs

• Rob Horner, George Sugai,
• Hariharon Swaminathan, Tom Kratochwill, and Joel
  Levin

4
Focusing the Discussion

• We are concerned here only with single-case
  experimental designs
• ABAB Reversal/withdrawal Designs
• Multiple Baseline Designs
• Alternating Treatment Designs
• Note these designs are sometimes conceptualized
  as within, combined, and between-series,
  respectively.
• Excluded are AB designs, ABA designs, Interrupted
  Time Series Case Studies.
• The three types of designs can be extended into
  randomized experiments (Kratochwill Levin,
  2007).

5
Single Case Research Logic

• Individual is unit of analysis
• Dependent Variable
• Operationally defined
• Measured directly
• Repeated measurement
• Independent Variable
• Measured for implementation fidelity
• Replication of effect within an experimental
  design
• To control for threats to internal validity

6
Analysis Needs
Analysis Questions Group Designs Single-case Designs
Experimental Control (Statistical Significance) Design Analysis RCT, RD Statistical MBL, ABAB, etc Visual Analysis
Effect Size Cohens d, r, etc PND, ?
Social Validity
X
X
X
7
Visual Analysis

• Documenting Experimental Control
• Three demonstrations of an effect at three
  different points in time.
• An effect is a predicted change in the
  dependent variable when the independent variable
  is actively manipulated.
• Visual Analysis includes simultaneous assessment
  of
• Level, Trend, Variability, Immediacy of Effect,
  Overlap across Adjacent Phases, Similarity of
  Data Pattern in Similar Phases.
• (Parsonson Baer, 1978 Kratochwill Levin,
  1992)

8
Visual Analysis

• Interpreting experimental control always involves
  assessment of data from the whole study (all
  phases), not just assessment of two adjacent
  phases.
• Assessment of an effect is done with 2 adjacent
  phases.
• Assessment of experimental control, however,
  requires evaluation of all data in all phases.

9
First Demonstration of Effect
Third Demonstration of Effect
Second Demonstration of Effect
Visual Analysis 1. Change in Level 2. Change
in Trend 3. Change in Variability 4. Immediacy
of Effect 5. Overlap 6. Similarity of Data
Patterns and Phases
Parsonson Baer, 1978 Kratochwill Levin, 1992
10
First Demonstration of Effect
Third Demonstration of Effect
Second Demonstration of Effect
Comparison of actual against projected data
(Analysis of Transition States versus Analysis
of Steady States)
11
First Demonstration of Effect
Third Demonstration of Effect
Second Demonstration of Effect
12
First Demonstration of Effect
Second Demonstration of Effect
Third Demonstration of Effect
13
(No Transcript)
14
Statistical Analyses of Single-case Research

• Documenting Experimental Control/ Statistical
  Significance
• No statistical model currently replicates the
  conceptual logic used in visual analysis.
• Effect Size
• Not a variable traditionally addressed in
  Single-case Analyses.
• It needs to be.

15
Documenting Evidence-based Practices via
Single-case Research

• Define practice
• Build professional consensus

16
Defining a Practice

• A practice is a procedure, or set of
  procedures, designed for use in a specific
  context, by individuals with certain
  skills/features, to produce specific outcomes for
  specific individuals.
• Operationally defined procedures
• Target population/ Context
• Implementer Characteristics
• Defined outcomes
• Large practicessmall practices

17
Toward a Professional Consensus

• Five studies documenting experimental control
• Conducted in at least three different locations
  by at least three different researchers
• At least 20 different participants
• Documenting an effect size of ??

18
Example of using Single-Case Research to document
Evidence-based Practice

• Functional Communication Training to reduce
  problem behavior in school, home and community
• At least Five Studies
• Bird, Dores, Moniz Robinson (1989)
• Brown et al., (2000)
• Carr Durand (1985)
• Durand Carr (1987)
• Hagopian, Fisher, Sullivan, Acquisto LeBlanc
  (1998)
• Mildon, Moore, Dixon (2004)
• Wacker et al., (1990)
• At least three settings /scholars
• Six locations, Research Groups
• At least 20 participants
• 42 different participants document effect

19
Implications for Single-Case Research

• Measures
• Define assumptions about trajectory
• Baseline
• Document predictable patterns
• More data points if trend
• More data points if higher variability
• Data points per phase
• At least 5 points per phase
• More data when trend and variability is high
• Combination of visual and statistical analysis
• Need for effect size measures in single-case
  designs
• For individual studies
• For meta-analyses

20
(No Transcript)
21
Summary

• The inclusion of single-case research findings in
  a larger body of education research will require
  attention to added strategies for interpretation,
  and added strategies for application
• Interpretation
• Better definition of visual analysis procedures
• Agreement on effect size index
• Application
• Definition of evidence-based practices
• Protocol for inclusion of single-case research in
  meta-analyses.