Evidence-based Application of Evidence-based Treatments

1

• Evidence-based Application of Evidence-based
  Treatments
• Peter S. Jensen, M.D.
• President CEO
• The REACH Institute
• REsource for Advancing Childrens Health
• New York, NY

2
Effect Sizes of Psychotherapies
Children Adolescents
Adults
University
Mean Effect Sizes
Real World
Weisz et al., 1995
3

Barriers vs. Promoters to Delivery of
Effective Services (Jensen, 2000)

• Three Levels
• Child Family Factors
• e.g., Access Acceptance
• Provider/Organization Factors
• e.g., Skills, Use of EB
• Systemic and Societal Factors
• e.g., Organiz., Funding Policies

Efficacious Treatments
Effective Services
4

Teacher-Rated Inattention(CC Children Separated
By Med Use)
Key Differences, MedMgt vs. CC Initial
Titration Dose Dose Frequency Visits/year Len
gth of Visits Contact w/schools
5
Would You Recommend Treatment?
(parent)
Medmgt Comb Beh Not recommend 9 3 5 Neutral 9
1 2 Slightly Recommend 4 2 2 Recommend 35
15 24 Strongly recommend 43 79 67
6
Key Challenges

• Policy makers and practitioners hesitant to
  implement change
• Vested interests in the status quo
• Researchers often not interested in promoting
  findings beyond academic settings
• Manualized interventions perceived as difficult
  to implement or too costly
• Obstacles and disincentives actively interfere
  with implementation

7
Key Challenges

• Interventions implemented but titrate the dose,
  reducing effectiveness
• Clients too difficult, resources inadequate
  used to justify bad outcomes
• Research population not the same as youth being
  cared for at their clinical site
• Having data and being right neither necessary
  nor sufficient to influence policy makers

8
The Good and the Bad Effectiveness of
Interventions by Intervention Type

No. of Interventions demonstrating positive or
negative/inconclusive change
Davis, 2000
9
Little or No Effect (Provider
Organization-focused)

• Educational materials (e.g., distribution of
  recommendations for clinical care, including
  practice guidelines, AV materials, and electronic
  publications)
• Didactic educational meetings

Bero et al, 1998
10
Effective Provider Organizational Interventions

• Educational outreach visits
• Reminders (manual or computerized)
• Multifaceted interventions
• Sustained, interactive educational meetings
  (participation of providers in workshops that
  include discussion and practice)

Bero et al, 1998
11
Implications re Changing Provider Behaviors

• Changing professional performance is complex -
  internal, external, and enabling factors
• No magic bullets to change practice in all
  circumstances and settings (Oxman, 1995)
• Multifaceted interventions targeting different
  barriers more effective than single interventions
  (Davis, 1999)
• Little to no theory-based studies
• Consensus guidelines approach necessary, but not
  sufficient.
• Lack of fit w/HCPs mental models

12
Additional Perspectives

• Messenger of equal importance as the message
• Trusted
• Available
• Perceived as expert/competent
• Adult Learning Models
• Tailored to learners needs
• Learner-defined objectives
• Hands-on, with ample opportunities for practice
• Sustained over time
• Skill-oriented
• Feedback
• Attention to Maintenance and sustaining change

13
Dissemination and Adoption of New Interventions

• Sustained Interpersonal contact
• Organizational support
• Persistent championship of the intervention
• Adaptability of the intervention to local
  situations
• Availability of credible evidence of success
• Ongoing technical assistance, consultation

Source Backer, Liberman, Kuehnel (1986)
Dissemination and Adoption of Innovative
Psychosocial Interventions. Journal of
Consulting and Clinical Psychology, 54111-118
Jensen, Hoagwood, Trickett (1997) From Ivory
Towers to Earthen Trenches. J Appliied Developmen
tal Psychology
14
Science-based Plus Necessary -abilities

• Palatable
• Affordable
• Transportable
• Trainable
• Adaptable, Flexible
• Evaluable
• Feasible
• Sustainable

15
Models for Behavior Change (Jaccard et al, 2002)
The Theory of Reasoned Action (Fishbein Ajzen,
1975) Self-efficacy Theory (Bandura, 1977) The
Theory of Planned Behavior (Ajzen,
1981) Diffusion of Innovations (Rogers, 1995)
16
Influences on Provider Behavior

• Provider Factors
• Knowledge, training
• Self-efficacy
• Time pressures
• Fear of litigation
• Attitudes beliefs
• Social conformity
• Lack of information

• Patient Family Factors
• Stigma
• Adherence
• Negative attitudes
• Rapport, engagement

Prescribing Practices

• Systemic Societal Factors
• Organizational standards
• Staff support/resistance
• Staff Training
• Funding policy

• Economic Influences
• Compensation
• Reimbursement
• Incentives

17
First, Use an Atypical vs. Typical
Descriptives (n19) Min/Max
Mean(SD) Favor/Unfavor 0/5 3.73(1.61) Easy
/Hard -1/5 4.16(1.64) Improve/No 0/5 2.84(1.57)
Agree/Disagree 0/5 4.05(1.27)
18
First Use Atypical--Advantages

• Advantages Count Percent of
  Responses
• Avoids typicals' side effects
  13 59.1
• Better patient approval/compliance 5
  22.7
• Atypicals effective in treating aggression
  2 9.1
• Other (i.e. looks better politically)
  2 9.1
• Total responses 22 100.0

19
First Use Atypical Disadvantages

• Disadvantage Count Percent of
  Responses
• Typicals may work better for some patients
  6 23.1
• Avoids atypicals' side effects
  6 23.1
• If need to sedate patient, typicals may be better
  6 23.1
• More is known about typicals in kids
  4 15.4
• Can not be administered as IMs
  3 11.5
• Other
  1 3.8
• 
  
• Total responses 26
  100.0

20
First Use AtypicalObstacles

• Obstacle Count Percent of Responses
• Cost 5 23.8
• More data supporting typicals 5
  23.8
• Patient history of non-response to atypicals
  4 19.1
• Patient resistance 3
  14.3
• Less available 2
  9.5
• Other 2 9.5
• 
  
• Total responses 21 100.0

21
Limit the Use of Stats P.R.N.s
Descriptive Statistics (n19) Min/Max Mean
(SD) Favor/Unfavor -5/5 2.63(2.89) Easy/Hard -5
/5 -0.38(3.22) Improve/No -2/5 2.44(1.92) Agree
/Disagree -2/5 3.86(1.81)
22
Limit Stat P.R.N.s -- Advantages

• Advantage Count Percent of Responses
• Other (i.e avoids traumatizing patient,
  6 27.3
• Avoids unnecessary medication
  5 22.7
• Avoids unnecessary side effects
  4 18.2
• Allows doctor to better understand patients
  condition 4 18.2
• Patient learns techniques they can apply in real
  life 3 13.6
• 
  
• Total responses 22
  100.0

23
Limiting Stats P.R.N.'s Disadvantages

• Disadvantage Count Percent of
  Responses
• Possible safety risk to patient and others
  9 2.9
• Other (i.e. does not address biological factors
  6 28.6
• Difficult for staff, who may feel less in control
  4 19.0
• May need to rapidly sedate patient
  2 9.5
• Total responses 21
  100.0

24
Limiting Stats P.R.N.'s--Obstacles

• Obstacle Count Percent of Responses
• Safety 8 33.3
• Other (i.e.patient belief
• that p.r.n.s condone behavior
  5 20.8
• Staff resistance
  4 16.7
• Patient too aggressive 4
  16.7
• Staff availability and training
  3 12.5
• Total responses 24 100.0

25
Monitor Side Effects
Descriptives (n19) Min/Max Mean(SD) Favor/Unf
avor 3/5 4.57(.69) Easy/Hard -2/5 2.94(2.4) Im
prove/No 1/5 4.0(1.15) Agree/Disagree 3/5 4.68
(.58)
26
Use Standardized Scales for Side Effects --
Advantages

• Advantage Count Percent of Responses
• Helps captures side effects you might otherwise
  miss 8 27.6
• Other (i.e. increases patient compliance
  improves 6 20.7
• communication between doctors helps assess
• severity of side effects)
• Provides objective measure 4
  13.8
• Keeps doctors focus on side effects
  4 13.8
• Determines drug effectiveness for specific
  symptoms 4 13.8
• Enables doctor to track side effects over time
  3 10.3
• Total responses 29
  100.0

27
Use Standardized Scales for Side
Effects--Disadvantages

• Disadvantage Count Percent of
  Responses
• Doctor may ignore side effects not on scale
  3 27.3
• May minimize importance of clinical evaluations
  3 27.3
• Other (i.e. may make patient more aware of side
  effects) 3 27.3
• Methodological problems (i.e. inter-rater
  reliability) 2 18.2
• Total responses 1
  100.0

28
Scales for Side Effects--Obstacles

• Obstacle Count Percent of Responses
• Time 8 25.0
• Scales are complicated/require training
  6 18.7
• Instrument availability 5
  15.6
• Other (i.e. staff resistance instrument
  availability 5 15.6
• cost)
• Administrative barriers 3
  9.4
• Laziness
  3 9.4
• Clinician resistance 2
  6.3
• Total responses 32
  100.0

29
New Models for Behavior Change TMC, TII
(Gollwitzer, Oettingen, Jaccard, Jensen et al,
2002 Perkins et al., 2007)
30
Mental Contrasting/Implementation Intentions

• Use mental contrasting to strengthen behavioral
  intentions
• What are the advantages or positive consequences
  associated with the use of Guideline X
• Identify Obstacles
• What gets in the way of implementing guideline
  X
• Form Implementation Intentions to overcome
  obstacles
• If I encounter obstacle Y, then I will X.

31
Track Target Symptoms
32
Use A Conservative Dosing Strategy
33
Limit the Use of P.R.N.s
34
Intention to Use Guidelines in the Next Month
(n4)
Guideline Pre-Intervention Post-Intervention
Track Target Symptoms 4.6(2.89) 8.25(2.1)
Conservative Dosing Strategy 8.8(1.30) 10.00(.0)
Limit P.R.N. 5.6(3.64) 8.75(.96)
Track Side Effects 9.6(.89) 8.75(1.5)
35

Barriers vs. Promoters to Delivery of
Effective Services (Jensen, 2000)

• Three Levels
• Child Family Factors
• e.g., Access Acceptance
• Provider/Organization Factors
• e.g., Skills, Use of EB
• Systemic and Societal Factors
• e.g., Organiz., Funding Policies

Efficacious Treatments
Effective Services
36
CLINIC/COMMUNITY INTERVENTION DEVELOPMENT AND
DEPLOYMENT MODEL (CID) (Hoagwood, Burns Weisz,
2000)
Step 1 Theoretically and clinically-informed
construction, refinement, and manualizing of the
protocol within the context of the practice
setting where it is ultimately to be
delivered Step 2 Initial efficacy trial under
controlled conditions to establish potential for
benefit Step 3 Single-case applications in
practice setting with progressive adaptations to
the protocol Step 4 Initial effectiveness test,
modest in scope and cost Step 5 Full test of the
effectiveness under everyday practice conditions,
including cost effectiveness Step
6 Effectiveness of treatment variations,
effective ingredients, core potencies,
moderators, mediators, and costs Step
7 Assessment of goodness-of-fit within the host
organization, practice setting, or
community Step 8 Dissemination, quality, and
long-term sustainability within new
organizations, practice settings, or communities
37
Partnerships Collaborations in
Community-Based Research

• Why Partnerships?
• partnerships -- not with other scientists per se,
  but with experts of a different type -- experts
  from families, neighborhoods, schools, in
  communities.
• Only from these experts can we learn what is
  palatable, feasible, durable, affordable, and
  sustainable for children and adolescents at risk
  or in need of mental health services
• Partnership - changes in typical university
  investigator - research subject relationship
• Practice based Research Networks
• Bi-directional learning

38
Partnerships Collaborations in
Community-Based Research

• Traditional approach
• research question posed, building on theory and
  body of previous research
• logical next step in elegant chain of hypotheses,
  tests, proofs, and/or refutations
• isolation of variables from larger context limit
  potential confounds and alternative explanations
  of findings
• study designed, investigator then looks for
  subjects who will recipients of the bounty
• cannot answer questions about sustainability
• unidirectional
• blind to issues of ecological validity

39
Partnerships Collaborations in
Community-Based Research

• Alternative (collaborative) approach
• expert-lay distinction dissolved
• both partners bring critical expertise to
  research agenda
• research methods and technical expertise from the
  university investigator
• systems access and local-ecological expertise
  from the community collaborator
• so-called confounds can provide useful tests
  of the feasibility, durability, and
  generalizability of the intervention
• hence, importance of replication
• improved validity of knowledge obtained?

40
The REACH Institute.Putting Science to Work
Step I
Step II
- Problem area identification - Bring key
change agents and gatekeepers to the table
(federal or state partners, consumer and
professional organizations) - Identify
actionable knowledge among experts and
consumers
- Identify E-B QI procedures that are feasible,
sustainable, palatable, affordable,
transportable - Consumer and stakeholder
buy-in commitment to E-B practices -
Dissemination via partners across all 3 system
levels - with an edge (policy/legislative
strategy with relevant federal/state partners)

Step IV
- Training and TA/QI intervention all sites
eventually get intervention. -
Monitoring/fidelity - Report preparation -
Results fed back into Step II.
Step III
- Site recruitment and preparation within
natural replicate settings - Tool preparation,
fidelity/monitoring - Skimming the cream,
first taking those sites most ready
41
Design Considerations

• Begin with the end in mind CID model
• Enemy of the good is the perfect raise the
  floor, not the ceiling
• Randomized encouragement trials vs. randomized
  controlled trials
• Quality Improvement group vs. TAU
• How does one know the necessary ingredients of
  change?
• Attention Expectations Hawthorne effects?
  Measure them
• Attention dose, time in treatment? Measure them
• Measure change processes
• Assuring fidelity to model? Measure it
• Ensure therapeutic relationshipand measure it
• Ensure family buy-in and therapist buy-in.
  Measure it
• Need for two controls? TAU, attention control
  group

42
Overcoming Challenges A Motivational Approach

• Change implementation strategies based on
  motivational approaches - William Miller
• Practice what you preach
• Express empathy
• to challenges of policy makers and practitioners
  in implementing change with population
• Develop discrepancy between ideal and current
• Success of evidence-based treatment must be
  explainable, straightforward, simply stated,
  meaningful

43
Overcoming Challenges A Motivational Approach

• Avoid argumentation
• Clinician scientists credible to policy makers
  and community-based practitioners
• Avoid overstating the case and poisoning the
  well
• Roll with resistance
• Develop strategies for engagement, prepare for
  possible resistance

44
Foundation of Collaborative Efforts
Goals
Researcher driven
Shared equal investment
Power
Research retains
Fairly distributed
Skills
Research skills designated as primary
Recognition of contribution by community member
researchers
Communication
One-way Unbalanced
Open opportunities to discuss resolve conflict
Trust
Belief in the good faith of partners room for
mistakes
Continual suspicion
45
Degrees of collaboration
Focus groups
Community Advisors or Advisory Board
Community partners as paid staff
Collaboration
() identification of pressing community/family n
eeds () definition of acceptable
research projects or service innovations
() provides ongoing input regarding various
stages of research process
() collaboration regarding implementation of
project () access to researchers to provide
guidance as obstacles encountered
() co-creation co-implementation co-evaluation co
-dissemination

46
Points of Collaboration in the Research Process
Study Aims
Research design sampling
Measurement Outcomes
Procedures (recruit, retain, data collection
Implementation
Evaluation
Dissemination
Members of partnership define dissemination outle
ts OR Members of community fulfill
co-author co-presenter roles OR Researchers
present at conferences publish
Plans for analysis co-created to ensure questions
of both community researchers answered
OR Community members assist in
interpretation of results OR Researchers
analyze data
Shared responsibility (e.g. community to
recruit, research staff to collect
data) OR Designed with input OR Designed
by researchers
Decision made jointly OR Researcher educates on
methods advice sought OR Methods
pre- determined
Projects are co-directed OR Researchers train
community members as co-facilitators OR Resear
ch staff hired for project
Defined collaboratively OR Advice
sought OR Researcher defined
Defined within partnership OR Advice sought OR
Researcher defined
Collaboration
47
The REACH Institute.Putting Science to Work
Step I
Step II
- Problem area identification - Bring key
change agents and gatekeepers to the table
(federal or state partners, consumer and
professional organizations) - Identify
actionable knowledge among experts and
consumers
- Identify E-B QI procedures that are feasible,
sustainable, palatable, affordable,
transportable - Consumer and stakeholder
buy-in commitment to E-B practices -
Dissemination via partners across all 3 system
levels - with an edge (policy/legislative
strategy with relevant federal/state partners)

Step IV
- Training and TA/QI intervention all sites
eventually get intervention. -
Monitoring/fidelity - Report preparation -
Results fed back into Step II.
Step III
- Site recruitment and preparation within
natural replicate settings - Tool preparation,
fidelity/monitoring - Skimming the cream,
first taking those sites most ready