National Prevention Network 21st Annual Prevention Research Conference Prevention Research: Driving

1
How CSAPs Service to Science Initiative
ImprovesThe Evaluation Capacity of Promising
Interventions A View from Provider and
Participant Perspectives
National Prevention Network21st Annual
Prevention Research ConferencePrevention
Research Driving Successful Outcomes August 25,
2008, Indianapolis, Indiana
2
Multiple perspectives about STS

• Federal Funder / Planner
• Carol McHale, SAMHSA/CSAP
• CAPT Evaluator
• Wayne Harding, NECAPT
• STS Clients Their CAPT TA Provider
• Zero Adult Provider Coalition
• Robyn Weisman, Director/Developer
• Rebecca Sechrist, Evaluator
• Mary Ellen Murphy, CAPT TA Provider
• Youth Action for Prevention Program
• Marlene Berg, Director/Developer
• Emil Cohen, Evaluator
• Kim Dash, CAPT TA Provider

3
Agenda

• Overview of Service to Science (Carol McHale)
• Characteristics of Participant Programs and
  Program Outcomes (Wayne Harding)
• Brief Descriptions of the Programs and their
  Experience with STS (Zero Adult Provider
  Coalition, Youth Action for Prevention Program)
• Discussion with You (Moderated by Wayne Harding)

4
Overview of Service to ScienceInitiative
5
SAMHSAs Strategic Prevention Framework Steps
Assessment
Profile population needs, resources, and
readiness to address needs and gaps
Evaluation
Capacity
Sustainability Cultural Competence
Monitor, evaluate, sustain, and improve or
replace those that fail
Mobilize and/or build capacity to address needs
Implement evidence-based prevention programs and
activities
Develop a Comprehensive Strategic Plan
Implementation
Planning
6
Service-to-Science Goals

• To help innovative, field-grown prevention
  interventions that address substance abuse (or
  related issues) develop and document evidence of
  their effectiveness.
• To increase the pool of evidence-based
• interventions.

7
Guiding Principles for Service-to-Science

• Meet programs where they are along a continuum
  of evidence of effectiveness.
• Participating programs have a common direction
  but different final destinations.

8
Service-to-Science Activities

• CAPTs solicit prevention program nominations from
  State prevention officials.
• CAPTs assess the readiness of candidate programs.
• CAPTs conduct regional Service to Science
  Academies.
• CAPTs provide follow-up technical assistance.

9
Service-to-Science Activities (Continued)

• Service to Science Academy participants
  eligible to compete for mini-subcontracts to
  further enhance evaluation capacity through the
  CRP, Inc. Service to Science Contract.
• CAPTs provide limited technical assistance post
  mini-subcontract award.

10
Service-to-Science Activities (Continued)

• Beginning in 2008, Service to Science has
  enhanced its outreach activities to
• 1. Pacific Jurisdictions
• 2. Native Alaskans
• 3. American Indians, with CSAPs Native
  American Center for Excellence (NACE)

11
Number of Programs Served Through Service to
Science

• As of June 2008, the CAPTs had provided 239
  prevention programs with technical assistance.
• Since 2006, 79 programs have received
  subcontracts of up to 30,000.

12
Defining Features of Service to Science TA

• Customized
• Long-term
• Relationship-based
• Flexible

13
SAMHSA/CSAPs Service-to-Science Initiative
Evidence-Based Review/Rating Processes
State Prevention Officials (National Prevention
Network) Nomination and Recruitment
SAMHSAs NREPP
Other Registries Federal Lists
CAPTs Regional S-S Academies and Follow-up Technic
al Assistance
CAPTs Technical Assistance Post Award Follow-up
Centers for the Application of
Prevention Technologies (CAPTs) Pre-assessment
CAPTs Support Mini Subcontracts Evaluation Capac
ity (1 year)
Published in Peer Reviewed Journal
Applicant Pool
Documented Effectiveness
14
Selected Evaluation Findings
15
Methods

• Pilot study (2006) interviews and/or online
  survey with program Directors. Target sample of
  79 programs 58 (73) responded.
• Six case studies of 2005 STS participants and
  extended follow-up case studies on 5 that
  received mini-subcontracts. Six case studies of
  2006 participants.
• 2007 online survey targeted 142 programs that had
  been participants for at least 15 months. 93
  responded (a 77 rate after removing 21 programs
  for which respondents were no longer available).
  

16
Participant ProgramCharacteristics
17
Geographic Distribution (N230)
18
Program Type (IOM classification)
(a) Multiple responses allowed. N93.
19
Program Duration and FTEs
(a) N79, (b) N89
20
Primary Behaviors Targeted for Change
(a) Multiple responses allowed. N93.
21
Ages of Population(s) Served
(a) Multiple responses allowed. N93.
22
Race/Ethnicity of Population(s) Served
(a) Multiple responses allowed. N93.
23
Strategies Used
(a) Multiple responses allowed. N93.
24
Evaluation Design Prior to STS
(a) Multiple responses allowed. N93.
25
Service To ScienceOutcomes
26
Value of STS Services
(a) N93.
27
Factors That Facilitated Progress

• When asked to discuss factors that facilitated
  progress on their evaluation the largest
  percentage of clients (79) identified TA from
  the CAPTs.

(a) From the 2006 pilot study. N58.
28
5 Most Common TA Issues
(a) Multiple responses allowed. N93.
29
TA Contributed to Program Capacity(a) in 5 Most
Common Issues
(a) Contributed A Little, Somewhat, or A
Great Deal vs. Did Not Contribute at All. (b)
N93.
30
Plan To Seek Recognition from NREPP or NASADAD
(a) N91, 2 programs that sought recognition
prior to STS assigned to missing