SAMHSA

1
SAMHSAs National Registry of Evidence-based
Programs and Practices (NREPP)
2
SAMHSAs Vision for NREPP NREPP becomes a
leading national resource for contemporary and
reliable information on the scientific basis
and practicality of interventions to prevent
and/or treat mental and substance use disorders.


3

• NREPP represents
• A major agency activity within SAMHSAs Science
  to Service initiative
• A decision support tool and valuable resource for
  state and community-based organizations seeking
  to identify and select interventions to meet
  their needs
• No longer a list of approved programs and
  practices

4
Decision-Support Dimensions of NREPP

• Descriptive Intervention summary, behavioral
  outcomes and key findings, implementation
  history, replications, estimated costs,
  populations and settings, adverse effects, and
  others
• Strength of evidence Single score derived from
  criterion-based, quantitative ratings provided by
  expert reviewers
• Readiness for dissemination Single score derived
  from criterion-based, quantitative ratings
  provided by expert reviewers

5
Descriptive Dimensions

• Intervention Summary and Contact Information
• Proprietary/Public Domain
• Key Findings
• Evaluation Design
• Populations and Settings, Culturally Specific
  Adaptations
• Costs
• Adverse Effects
• Adaptations/Replications
• Implementation History

6
Strength of Evidence Ratings Outcome-Specific
Criteria

• Reliability- Outcome measures should have
  acceptable reliability to be interpretable.
  Acceptable here means reliability at a level
  that is conventionally accepted by experts in the
  field.
• Validity - Outcome measures should have
  acceptable validity to be interpretable.
  Acceptable here means validity at a level that
  is conventionally accepted by experts in the
  field.
• Intervention Fidelity - The experimental
  intervention implemented in a study should have
  fidelity to the intervention proposed by the
  applicant. Instruments that have tested
  acceptable psychometric properties (e.g.,
  inter-rater reliability, validity as shown by
  positive association with outcomes) provide the
  highest level of evidence.

7
Strength of Evidence Ratings Outcome-Specific
Criteria

• 4. Missing Data and Attrition - Study results
  can be biased by participant attrition and other
  forms of missing data. Statistical methods as
  supported by theory and research can be employed
  to control for missing data and attrition that
  would bias results, but studies with no attrition
  or missing data needing adjustment provide the
  strongest evidence that results are not biased.
• 5. Potential Confounding Variables - Often
  variables other than the intervention may account
  for the reported outcomes. The degree to which
  confounds are accounted for affects the strength
  of causal inference.
• 6. Appropriateness of Analyses - Appropriate
  analysis is necessary to make an inference that
  an intervention caused reported outcomes.

8
Readiness for Dissemination Ratings

• Availability of implementation materials
• Availability of training and support resources
• Availability of quality assurance procedures

9
Three Minimum Requirements for NREPP Submission
1. Demonstrate one or more positive outcomes in
mental health and/or substance abuse
behavior among individuals, communities, or
populations.2. Demonstrate intervention results
have been published in a peer- reviewed
publication or documented in a comprehensive
evaluation report.3. Demonstrate
materials such as manuals, process guides, and
training guides that are available to the
public to implement the intervention.

10
Types of Materials Submitted

• Strength of Evidence
• Research articles and published/unpublished
  evaluation reports
• Grant final reports
• Replication reports
• Implementation manuals
• Data collection manuals

11
Types of Materials Submitted Continued

• Readiness for Dissemination
• Implementation guides, training manuals,
  presentations and curricula
• Quality assurance and monitoring protocols and
  procedures
• Data collection protocols for process and outcome
  measures
• Products and materials adapted for different
  age/cultural groups

12
Types of Materials Submitted Continued

• Additional Documents / Narrative
• To Describe
• Costs of implementation Start-up and maintenance
  
• Proprietary components of intervention
• Implementation history of intervention
• Unintended or adverse effects of the intervention
  
• Contact information

13
NREPP Supports Evidence-Based Decisions

• SAMHSA recognizes the descriptive information
  and ratings provided through the NREPP system are
  only useful within a broader context that
  incorporates multiple perspectives.
• These perspectivesincluding clinical,
  consumer, administrative, fiscal, organizational,
  and policy should influence stakeholder
  decisions about the identification, selection,
  and successful implementation of evidence-based
  services.

14
NREPP Priority Point System
1 priority point -- Research design
Experimental (e.g., randomized control study)
or quasi-experimental (e.g., pre/post with
comparison group, or interrupted time series)
research design 1 priority point -- Priority
topic area Primary outcome(s) of the
submitted intervention is (are) in one or more of
SAMHSAs Center-specific priority areas.

15
Center for Substance Abuse Prevention (CSAP)
Priority Areas
Priority areas focus on comprehensive community
strategies, actions, and interventions that1.
Prevent and/or reduce substance abuse and its
related problems (e.g., underage drinking,
inhalant use, cannabis use and abuse,
drug-related suicide, alcohol and drug abuse
among young adults, misuse of alcohol and
prescription drugs among elderly, HIV/substance
abuse problems)

16
CSAP Priority Areas Continued
2. Change policies and practices at community
level to reduce risk factors and/or increase
protective factors among multiple domains
(e.g., workplace, schools,
neighborhoods)3. Address emerging substance
abuse problems (e.g., methamphetamine,
over-the-counter drugs, fentanyl, other
synthetic drugs)

17
Availability of materials to support initial and
ongoing implementation

• (e.g., treatment manuals,
  brochures, information for administrators)
• 0 No materials or resources are available that
  could directly or indirectly
• assist with program implementation.
• 1 Limited materials and resources are available
  that could facilitate but not
• directly assist with program
  implementation.
• 2 Limited materials and resources are available
  that directly assist with
• program implementation.
• 3 Adequate materials and resources are
  available that directly assist with
• program implementation, and they are
  of adequate quality and appropriate
• for the intended audience(s).
• 4 Adequate materials and resources are
  available that directly assist with
• program implementation, and they are
  of high quality and appropriate for
• the intended audience(s).

18
Availability of training and/or resources to
support initial and ongoing implementation

• (e.g. tested training curricula, mechanisms for
  ongoing supervision and consultation)
• 0 No materials, resources, and/or technical
  assistance are available that directly or
• indirectly support training or
  coaching.
• 1 Limited materials, resources, and/or
  technical assistance are available that could
• facilitate but not directly assist with
  program implementation.
• 2 Limited materials, resources, and/or
  technical assistance are available to directly
• support training and coaching.
• 3 Adequate materials, resources, and/or
  technical assistance are available that
• directly support training and
  coaching, and they are of adequate quality and
• appropriate for the intended
  audience(s).
• 4 Adequate materials, resources, and technical
  assistance is available that directly
• support training and coaching, and
  they are of high quality and appropriate for the
• intended audience(s).

19
Availability of quality assurance procedures to
support initial and ongoing implementation

• (e.g. protocols for gathering process and/or
  outcome data, ongoing monitoring of
• intervention fidelity,
  supervision/training feedback)
• 0 No materials, resources, and/or procedures
  are available that could directly or
• indirectly help ensure quality
  assurance.
• 1 Limited materials, resources, and/or
  procedures are available that could facilitate
• but not directly assist in ensuring
  quality assurance.
• 2 Limited materials, resources, and/or
  procedures are available to directly support
• quality assurance.
• 3 Adequate materials, resources, and/or
  procedures are available that directly
• support quality assurance, and they
  are of adequate quality and appropriate for
• the intended audience(s).
• 4 Adequate materials, resources and/or
  procedures are available that directly
• support quality assurance, and they
  are of high quality and appropriate for the
• intended audience(s).

20
NREPP Supports Evidence-Based Decisions

• SAMHSA recognizes the descriptive information
  and ratings provided through the NREPP system are
  only useful within a broader context that
  incorporates multiple perspectives.
• These perspectivesincluding clinical,
  consumer, administrative, fiscal, organizational,
  and policy should influence stakeholder
  decisions about the identification, selection,
  and successful implementation of evidence-based
  services.

21
Contact for Submission Guidance
Individuals and/or organizations interested in
submitting an intervention for potential
inclusion in NREPP should contact the NREPP
contractor MANILA Consulting Group Telephone
571.633.9797, extension 406E-mail
nrepp_at_samhsa.hhs.gov or go on line at
www.nrepp.samhsa.gov