Interrupted Time-Series Designs

1
Interrupted Time-Series Designs

• Biglan and colleagues
• Komro and colleagues
• Flay and colleagues

2
The Value of Interrupted Time-Series Experiments
for Community Intervention Research

• Biglan, T., Ary, D., Wagenaar, A.C. (2000).
  Prevention Science, 1(1), 31-49.

3
Limitations of RCTs

• High cost
• Not good vehicle for identifying principles about
  variables that influence community practices
• May obscure important relationships unique to
  subset of communities
• Preventing diffusion of intervention into control
  communities

4
Repeated time-series experiments

• Repeated measurement of a process
• Manipulation of an independent variable
• Need to consider
• variability of the
• outcome
• Limitations change could be due to numerous
  other co-occuring factors

5
(No Transcript)
6
Multiple Baseline Designs

• IV is manipulated at different points in time for
  multiple time series.
• 2 basic types
• Across cases
• A phenomenon is measured repeatedly in 2 cases.
• Manipulation of IV at different times for
  different cases.
• Within cases
• Measure 2 phenomena repeatedly within a single
  case.
• IV is applied to one of the phenomena at a time.

7
(No Transcript)
8
Internal Validity (multiple baseline)

• What are some threats to internal validity in
  multiple baseline designs?

9
Internal Validity (multiple baseline)

• History only historical events occurring across
  all communities.
• Testing multiple assessments could influence the
  processes under study.
• Instrumentation changes in observer or measure
  over time.
• Instability greater variability in processes
  under study makes it harder to find intervention
  effects.
• Statistical regression regression to the mean of
  extreme scores with high single baseline
  assessment.
• Selection can be a problem if characteristics of
  cases are confounded with intervention.
• Control of the implementation of the IV if
  implementation of IV cannot be controlled,
  difficult to make statements about the
  intervention effect.

10
External Validity (multiple baseline)

• What are some threats to external validity in
  multiple baseline designs?
• Failure to replicate a relationship, clue about
  moderators
• Generalizability
• Multiple baseline not really meant for testing
  broad generalizability
• Use in conjunction with RCTs
• Can use to help eliminate extraneous ineffective
  components of an intervention

11
Statistical Analyses

• Examination of intercept, slope and non-linear
  patterns and change (changes in cycles, trends,
  variance)
• ARIMA modeling
• ITSE
• ITSACORR
• Simontons method
• Velicer McDonalds method
• LGM
• Individual cases?

12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
Small group activity

• Does your intervention have a uniform effect
  across all cases?
• What individual cases might you want to look at?
• Can you think of examples of studies/phenomena
  that looked at individual cases? Or,
  studies/phenomena that might benefit from looking
  at individual cases? (Or, cases where youve
  done this in your own work?)

16
PROPOSALPrevention Trial in the Cherokee Nation
Interactive Effects of Environment SBIRT

• PI Kelli A. Komro
• Co-PI B.J. Boyd, Misty L. Boyd
• Co-I Levi Keehler, T.K. Kominsky,
• Mildred Maldonado-Molina, Amy L. Tobler,
• Alexander C. Wagenaar

17
Proposed study

• Controlled randomized time-series experimental
  design
• Build on prior foundational research
• Community based participatory research
• Environmental interventions
• Brief interventions
• Rural high-risk underserved communities
• Northeastern Oklahoma
• Challenges

18
Randomized controlled interrupted time-series
design
19
Two parts to the intervention

• Environmental interventions
• CMCA
• Enforcement Checks
• Hot-spot Policing
• Media Campaign
• Brief interventions
• SBIRT
• Gatekeeper training
• Peer leader training
• Family postcard campaign
• Community-wide media campaign

20
Research Design Issues for Evaluating Complex
Multicomponent Interventions in Neighborhoods and
Communities

• Flay, B.R., Biglan, A., Komro, K.A., Wagenaar,
  A.C., the Research Team of the PNRC (2011).

21
Multi-component community interventions

• Multi-component interventions implemented (and
  rigorously evaluated to ensure continual
  improvement) in high poverty neighborhoods
• PNRC is one example of a comprehensive community
  intervention

22
Major challenges to optimal evaluation

• Multiple levels of evaluation/research
• Complexity of packages of interventions
• Classes of evaluation/research questions

23
Possible designs

• RCT
• Regression-discontinuity designs
• Designs involving repeated measures
• Interrupted time-series designs
• Repeated-measures, non-equivalent control group
  designs
• Multiple-baseline designs
• Single-case studies

24
SP500 30yrs, 15yrs, 5yrs,
3mos
25
Time-series data visualization

• http//www.talkingpointsmemo.com/archives/2010/12/
  200_years_in_4_minutes.php/

26
Ways to improve designs

• Waves
• Settings
• DVs that are expected to change
• DVs not expected to change
• Randomization
• Multiple comparisons
• Adaptive intervention design

27
Standardized measurement system

• 1. Implementation of each intervention component
• 2. Reach of each component
• 3. Effects of each component on immediate
  outcomes
• 4. Effects of the comprehensive intervention
  package on outcomes