Washington States

1

• Washington States
• Evaluation of Evidence-Based
• Juvenile Justice Programs
• University of Lund
• 12 September, 2005

2
Background

• 1996 State and local juvenile court
  administrators discussed research-based
  programs.
• 1997 Community Juvenile Accountability Act
  (CJAA)
• Legislature said it would provide funding if
  research-based programs could be identified.
• State, juvenile court administrators, and the
  Institute met regularly to identify
  research-based programs.
• Six programs were identified, four were
  recommended.
• 1998 Legislature funded the programs 3.3
  million/year.
• 1999 Programs started.
• 2001 Evaluation groups filled.
• 2003 Evaluation completed.

3
Definition for Research Based Programs

• Measurable behavioral outcome preferably related
  to avoiding future public costs.
• At least a 3 on Institute's evaluation design
  scale.
• Sufficient documentation and an infrastructure to
  permit replication.
• Replicated results.
• (For example
• University of Colorado Blueprint Programs)

4
Institutes Rating of Program Evaluation Designs

• 5 Random assignment of subjects to a treatment
  and a control group that does not receive the
  treatment/program.
• 4 Quasi-experimental design with a program and
  matched comparison group, and statistical methods
  controlling for self-selection bias that might
  otherwise influence outcomes.
• Program and comparison groups matched for
  pre-existing differences in key variables with
  statistical techniques to control for
  pre-existing differences.
• 2 Program and matched comparison group where
  the two groups lack comparability on pre-existing
  variables and no attempt to control for these
  differences.
• 1 No comparison group, before and after the
  program comparison.

5
Programs Found (Meet Research Based Criteria)

• Functional Family Therapy (FFT)
• Multi-Systemic Therapy (MST)
• Aggression Replacement Training (ART)
• Tolans Coordination of Services
• Davidsons Adolescent Diversion
• Multidimensional Therapeutic Foster Care

6
Functional Family Therapy(2,100 per youth)

• Blueprint Program University of Colorados
  Center for the Study and Prevention of Violence.
• Structured family-based intervention to enhance
  protective and reduce risk factors in the family.
  
• FFT is a three-phase program
• Motivate family toward change.
• Teach family how to change a specific critical
  problem.
• Help family generalize their problem-solving
  skills.
• Trained therapists have caseloads of 10 to 12
  families.
• Involves about 12 visits during a 90-day period.

7
Multi-Systemic Therapy(5,000 per youth)
Juvenile Accountability Incentive Block Grant

• Blueprint Program.
• Ecological intervention for high-risk youth.
• Focuses on improving the familys capacity to
  overcome the known causes of delinquency. The
  goals
• Promote parents ability to monitor and
  discipline.
• Replace deviant peer relationships with
  pro-social.
• Trained MST therapists, working in teams
  consisting of one Ph.D. clinician and three or
  four clinicians with masters degrees.
• Caseload of four to six families for three and
  six months.

8
Aggression Replacement Training(700 per youth)

• Group training of 10 youth by two instructors.
• Classes meet for one hour three times per week
  for 10 weeks
• Aggression cycle
• Skill development
• Moral reasoning
• Training, not therapy, that uses guided
  discussion, modeling, role play, and home work.

9
Coordination of Services(400 per youth)

• Educational program for low-risk juvenile
  offenders and their parents. The goals
• Describe the consequences of delinquent behavior.
  
• Stimulate goal setting.
• Review the strengths of youth and family.
• Explain what resources are available.
• Participants are given a vehicle to open lines of
  communication and make shifts in thinking.
• Community groups present participants with
  information concerning the services they provide.

10
What Is Different About CJAA Programs?

• Limited to well-evaluated programs shown to
  reduce recidivism.
• Each local court chooses which program on the
  state-funded list works best for them.
• Courts implement off-the-shelf programs, not
  principles.
• Quality assurance efforts to maintain program
  fidelity.
• Formal assessment used to determine which program
  is best for which youth.

11
Evaluation Design

• A rigorous test
• Not enough funding for everyone, so Waiting
  List youth assigned to a control group by court
  staff.
• Includes completers and non-completers.
• Youths eligibility for program is based on risk
  assessment
• Level of risk for re-offending.
• Profile of risk factors.
• Measurable outcome of behavior recidivism

12
Evaluation Results
13
FFT Results

18-Month Unadjusted Felony Recidivism Percentage




















14
ART Results
18-Month Recidivism Percentage
15
Coordination of Services Results
Control (N 171)
WayOut (N 171)

21
19
18
17
12-Month Recidivism Percentage
5
1
3
2
Misdemeanor and Felony
Misdemeanor and Felony
Felony
Felony
Actual 12-Month Recidivism
Adjusted 12-Month Recidivism
16
MST Results
No findings because of problems implementing the
Institutes evaluation design.
18-Month Recidivism Percentage
17
Findings

• FFT delivered competently
• Reduces 18-month felony recidivism by 38 percent.
• Benefit to cost ratio of 10.69.
• (Regardless of competence 2.77 savings)
• ART delivered competently
• Reduces 18-month felony recidivism by 24 percent.
• Benefit to cost ratio of 11.66.
• (Regardless of competence 6.71 savings)
• Coordination of Services
• Reduces 12-month felony recidivism by 57 percent.
• Benefit to cost ratio of 7.89.
• MST
• No findings because of problems implementing the
  Institutes evaluation design.

18
Summary of Findings for Washington State
Evaluations
Institutional Setting
19
What Have We Learned to Date?

• The key to reducing recidivism is delivering the
  research-based program competently.

20
How Do We Guarantee Competent Program Delivery?

• Pick the best people
• Provide required training
• Assess and enhance skills
• Retain only competent providers

21
Annual Statistics on Quality Assurance to
Legislature

• Program providers training and competency
  ratings.
• Youth eligible, assigned, and completing programs
  (75).
• Changes in targeted risk and protective factors.
• Meeting expected outcomes - recidivism rates.

22
Monitoring Expected Results Illustration Of What
We Hope To Do
50
Expected Without Program
Expected With Program
Actual
40
18-Month Felony Recidivism Rate
30
20
10
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
FFT Youth Cohorts