LAYING THE FOUNDATION: REAIM OVERVIEW

1
LAYING THE FOUNDATIONRE-AIM OVERVIEW

• Russell E. Glasgow, Ph.D.
• Kaiser Permanente Colorado
• http//research-practice.org
• See also www.re-aim.org

2
OVERVIEW

• Challenges to Translating Research into Practice
• Rationale for and Basics of RE-AIM
• Common Questions and Issues in Applying RE-AIM
• New Developments and Future Directions
• Q and A

3
WHATS UNIQUE ABOUT TRANSLATIONAL RESEARCH?

• Practical Questions About costs, implementation
  feasibility

• Contextual Want to know
• - If works in particular settings, groups,
  etc.
• - Generalizability how broadly applicable

• Issues of Adaptation vs. Fidelity

• Multiple perspectives

4
METHODOLOGICAL ISSUES

• Estimating Ones Denominator

• Usually Multi-level and Complex

• Mixed quantitative and qualitative measures
  helpful, especially around context

• Estimating Costs and Modeling

5
Simplified Systems Model for Translational
Research
Delivery Site(s)
Program or Policy
Fit
Appropriate For Question
Partnership
Broader Health Policy and Cultural Context
Estabrooks PA, Glasgow RE. Am J Prev Med
200631(4S)S45-S56
6
RECONCILING DIVERSE PERSPECTIVESFINDING COMMON
GROUND AMONG

• Researchers

Effect size, fidelity

• Practitioners and Educators

Feasibility, local applicability

• Policy Makers and Administrators

Costs, breadth of application

• Patients

Quality of life
7
The law of halves a story
8
(No Transcript)
9
MORAL OF THE STORY?
Focus on the Denominator

• (Each Step Provides Opportunities to Enhance
  Impact)

10
RE-AIM TO HELP PLAN, EVALUATE, AND REPORT STUDIES

• R Increase Reach
• E Increase Effectiveness
• A Increase Adoption
• I Increase Implementation
• M Increase Maintenance

Glasgow, et al. Ann Behav Med 200427(1)3-12
11
PURPOSES OF RE-AIM

• To broaden the criteria used to evaluate programs
  to include external validity and context.
• To evaluate setting issues relevant to program
  adoption, implementation, and sustainability.
• To help close the gap between research studies
  and practice by
• Informing design of interventions
• Providing guides for adoptees
• Suggesting standard reporting criteria to
  increase transparent reporting

12
RE-AIM ELEMENTS REACH

• Definition The number, percent of target
  audience, and representativeness of those who
  participate.
• Example 65 of chronic illness patients invited
  to group medical visit attended initial session
  those declining more likely to be Latino.

Key Issues Does program reach those at
highest risk? Are different promotional
approaches or visit options required?
13
RE-AIM ELEMENTS EFFECTIVENESS

• Definition Change in temporally appropriate
  outcomes, and impact on quality of life and any
  adverse outcomes.
• Example Mailed reminder/telephone outreach
  program increased colon cancer and mammography
  screening rates by 20 with no adverse effects
  on quality of life or cardiovascular screening
  rate.

Key Issues Logic model helps to clarify
anticipated effects quality of life provides
common metric across conditions and
interventions anticipate unintended
consequences.
14
RE-AIM ELEMENTS ADOPTION

• Definition Number, percent and
  representativeness of settings and clinicians who
  participate.
• Example Six months after in-service training,
  75 of case managers have used evidence-based
  programs varies across settings from 33-95.

Key Issues Need to focus on denominator and
barriers among non-users. Do initial adoptees
include peer opinion leaders?
15
RE-AIM ELEMENTS IMPLEMENTATION

• Definition Extent to which a program or policy
  is delivered consistently, and the time and costs
  of the program.
• Example Caregivers receive at least two
  follow-up phone calls 75 of the time takes an
  average of 5 minutes, but not all staff are
  calling consistently.

Key Issues Consistency across staff, program
components, and time. Balance between fidelity
and local customization.
16
RE-AIM ELEMENTS MAINTENANCE

• Definition
• Individual/member target Long-term effects and
  attrition.
• Setting/clinician Extent of discontinuation,
  modification, or sustainability of program.
• Example At one-year follow-up, was 58
  attrition from Internet weight loss program
  those present maintained weight loss. Only 40
  of clinicians initially referring continued to do
  so.

Key Issues Does attrition bias results
qualitative approaches to understanding program
adaptation.
17
RE-AIM BUILDING BLOCKS THAT TOGETHER PRODUCE
PUBLIC HEALTH IMPACT
Adoption
Building Programs and Policies with a Large
Public Health Impact
Efficacy Effectiveness
Implementation
Reach
Maintenance
18
RECOMMENDED PURPOSE OF TRANSLATION/EFFECTIVENESS
RESEARCH
To determine the characteristics of interventions
that can

• Reach large numbers of people, especially those
  who can most benefit
• Be widely adopted by different settings
• Be consistently implemented by staff members with
  moderate levels of training and expertise
• Produce replicable and long-lasting effects (and
  minimal negative impacts) at reasonable cost

19
CHALLENGES IN APPLYING RE-AIM
20
CHALLENGES IN APPLYING RE-AIM (cont.)
21
CHALLENGES IN APPLYING RE-AIM (cont.)
22
(No Transcript)
23
USING RE-AIM PROACTIVELY FOR PLANNING AND
PERIODIC SELF-EVALUATION

• Klesges, Estabrooks PA, et al. Ann Behav Med,
  2005, 2966-75
• Dzewaltowski, Glasgow, Klesges, et al. RE-AIM
  A web resource. Ann Behav Med 2004, 2875-80.
• Resources www.re-aim.org...
• especially, re-aim.org/database_quiz/intro.html

24
(No Transcript)
25
RATE (RE-AIM) YOUR PROGRAM

• REACH

26
A TALE OF TWO PROGRAMSWHICH IS BETTER?
27
(No Transcript)
28
(No Transcript)
29
REACH WHICH IS BETTER?
In-office attracted 50 (vs. 41) of eligible
participants
PERCENT
30
ADOPTION WHICH IS BETTER?
Linked, separate approach recruited 47 of PCPs
in-office PCP team delivery approach recruited
6 of PCPs
PERCENT
31
WHICH PROGRAM IS BETTER?
See www.re-aim.org for displays and evaluation
questions
Glasgow et al. AJPM 30(1)67-73
32
To every complex question, there is a simple
answer and it is wrong. H. L. Mencken
33
NEW RE-AIM SUMMARY METRICS THAT ADDRESS

• Health disparities e.g., who participates and
  who benefits
• Costs and cost-effectiveness
• Effects of different interventionists
• Combining different factors to produce composite
  outcomes

Glasgow et al. Using RE-AIM Metrics to Evaluate
DiabetesAJPM 200630(1)67-73 Glasgow et al.
Health Education Research 200621(3)688-694
34
RE-AIM AND RELATED GENERALIZATION ISSUES

• The 3 Rs of Integrating Research into Practice
• Representativeness (Reach, Adoption)
• Robustness (Effectiveness across
  subgroupsespecially re disparities)
• Cronbachs generalization across persons,
  time, measures
• Replicability (Implementation) in
  representative settings

Cronbach LH, et al. The dependability of
behavioral measurements Theory of
generalizability for scores and profiles. New
York, John Wiley Sons, 1972 Shadish WR, et al.
Experimental and quasi-experimental design for
generalized causal inference. Boston Houghton
Mifflin, 2002
35
The significant problems we face cannot be
solved by the same level of thinking that created
them. A. Einstein
All models (and designs) are wrong and
greater tolerance, respect, and creativity is
needed. Sterman JD. Syst Dynam Rev
200218501-531