DDCAT State Collaborative

1
DDCAT State Collaborative Research Opportunities

• Jessica Brown, Ph.D.
• Joseph E. Comaty, Ph.D., M.P., A.B.M.P.
• DHH / OMH Central Office
• Mark P. McGovern, Ph.D.
• Dartmouth Medical School
• LITS Leadership Summit
• May 6, 2008

2
Introduction

• All available evidence indicates
• MH with SA
• SA with MH
• Many with either disorder do not receive
  treatment
• Those with COD are not treated in an integrated
  program
• Not using EBP

3
Substance Use among Youths Aged 12 to 17, by
Major Depressive Episode in the Past Year 2006
4
2006 Statistics - NSDUH
5
2006 Statistics - NSDUH
6
Substance Use among Adults Aged 18 or Older, by
Major Depressive Episode in the Past Year 2006
7
2006 Statistics - NSDUH
8
Past Year Treatment among Adults Aged 18 or Older
with Both Serious Psychological Distress and a
Substance Use Disorder 2006
9
Program Evaluation

• Need for identification of resources to treat COD
• If resources do not exist need to build
  capacity
• First identify current capacity for treatment
  services and gaps
• Standardized evaluation of system capacity using
  valid and reliable methodology

10
DDCAT INDEX DEVELOPMENT

• Practical program level policy, practice and
  workforce benchmarks Based on scientific
  literature and expert consensus
• Observational methodology Interviews Document
  review Social, environmental cultural
  ethnography (vs. self-report)
• Iterative process of measure refinement Field
  testing and psychometric analyses
• Materials Index, manual, toolkit Excel
  workbook for scoring and graphic profiles

11
DDCAT 7 DIMENSIONS CONTENT OF 35 ITEMS
12
DDCAT INDEX RATINGS

• 1 - Addiction Only Services (AOS) or Mental
  Health Only Services (MHOS)
• 2 -
• 3 - Dual Diagnosis Capable (DDC)
• 4 -
• 5 - Dual Diagnosis Enhanced (DDE)

13
ADDICTION ONLY SERVICES (AOS)

• Programs that either by choice or for lack of
  resources, cannot accommodate clients who have
  psychiatric illnesses that require ongoing
  treatment, however stable the illness and however
  well-functioning the client.

14
DUAL DIAGNOSIS CAPABLE (DDC)

• Programs that have a primary focus on the
  treatment of substance-related disorders, but are
  also capable of treating clients who have
  relatively stable diagnostic or sub-diagnostic
  co-occurring mental health problems related to an
  emotional, behavioral or cognitive disorder.

15
DUAL DIAGNOSIS ENHANCED (DDE)

• Programs that are designed to treat clients who
  have more unstable or disabling co-occurring
  mental disorders in addition to their
  substance-related disorders.

16
CAN WE USE FIDELITY SCALE METHODOLOGY FOR
OBJECTIVE RATING OF DUAL DIAGNOSIS CAPABILITY?

• Site visit (yields data beyond self-report)
• Multiple sources Chart, brochure program
  manual review Observation of clinical process,
  team meeting, supervision session Interview
  with agency director, clinicians clients.
• Objective ratings on operational definitions
  using a 5-point scale

17
DDCAT INDEX SCORING AND INTERPRETATION

• 7 dimension scores Average (Sum of ratings
  divided by number of items)
• Overall DDCAT score Sum of dimension scores
  divided by 7)
• Categorization of program by Overall DDCAT score
  AOS, AOS/DDC, DDC, DDC/DDE, DDE
• Categorization of program by category based upon
  of criteria met Cutoff 80 or greater
• Qualitative interpretation and feedback

18
DDCAT PROFILE PRACTICAL GUIDANCE FOR PROVIDERS
DDE
DDC
AOS
19
DDCAT PROFILES OVER TIME
DEPICTING PROGRAM CHANGE
DDE
DDC
AOS
20
Testing the Process in LA

• Reliability Among Raters at Selected Sites

21
The Structure for DDCAT Administration

• COSIG Statewide

• People Products

• Evaluation Team
• Field Staff as co-evaluators
• Training by McGovern
• Manual Development

22
The DDCAT Assessment Sites

• 4 Sites per each of the 10 regions
• 1 Large Addictive Disorders Center
• 1Large Mental Health Center
• 1 Small Addictive Disorders Clinic (rural)
• 1 Small Mental Health Clinic (rural)
• Assessments Completed on Site by Combination team
  (Evaluation Field Staff)
• DDCAT Scoring Procedure
• Each rater independently scores
• Copies of these scores are collected
• Consensus Scores - the Official Score

23
Characteristics of the Raters
24
Characteristics of the Raters (Cont)

• Credentials

• Primary Work Role

25
Reliability Analysis

• Item-Level Reliability

• Comparisons

• By item
• By rater type
• By region
• By type of site
• Size, agency

26
Relative Reliability across DDCAT dimensions
27
Overall Concordance among Raters across all DDCAT
Items
28
Other Comparisons of Inter Rater Reliability

• No differences between Addiction Mental Health
  Settings
• No Difference between Small versus Large Clinics
• Reliability of some Items helping to refine the
  Index
• Some trends noted across the geographic regions
• Measure of inter-rater reliability within a few
  regions showed a lower reliability
• Effect of Practice

29
Region X 1st SiteDDCAT Profiles by Rater
30
Region X 2nd SiteDDCAT Profiles by Rater
31
Region X 3rd SiteDDCAT Profiles by Rater
32
Region X 4th SiteDDCAT Profiles by Rater
33
Summary/ Suggestions

• Training to criterion possible
• Practice makes perfect
• Development of manuals and additional behavioral
  anchors enhance consistency
• Reliability can be achieved, even with new raters
• Importance of partnership between local and state
  raters
• Use of external state level raters across all
  programs important for minimizing bias

34
The Cross-State Process

• Building a Large Database for Testing the
  Instrument and Clinical Research

35
STAGE III

• Broader use of DDCAT (norms, cost data, training)
• Agencies ongoing use DDCAT for self-assessment,
  planning of services, strategic staff training
  and as measure of change
• State leadership Baseline system information,
  change measure, rational service system design,
  standards resource allocation
• Link DDCAT with process and outcome measures
• a. Existing data sources (e.g. MIS, treatments
  received)
• b. Prospective research designs

36
DDCAT CURRENT SAPRP RESEARCH

• Learning collaborative among 6 (8) states Data
  sharing, quality improvement exchange, research
  studies
• (Five 2007 AHSR reports from state Co-PIs)
• Combined DDCAT database Further refinement of
  the index (Larger program n)
• Baseline and follow-up DDCAT assessments
  Qualitative quantitative analyses of effective
  change strategies

37
ACKNOWLEDGEMENTS

• The Robert Wood Johnson Substance Abuse Policy
  Research Program
• Dartmouth Medical School Karen Becker, Chantal
  Lambert Harris, Stephanie Acquilano, Robert
  Drake, Aurora Matzkin, Greg McHugo Will Torrey
• IUPUI Gary Bond
• State of CT Julienne Giard, Rhonda Kincaid,
  Thomas A. Kirk, Kenneth Marcus, Lauren Siembab,
  Minakshi Tikoo, Sam Segal Arthur Evans
• State of IL Trina Diedrich, Randi Tolliver
  Phil Welches
• State of IN John Viernes David Garner
• State of LA Jessica Brown, Joseph Comaty Tanya
  McGee
• State of MO Ron Claus, Heather Gotham Kimberly
  Selig
• State of NH Joseph Harding, Lindy Keller Jim
  Shanelaris
• State of TX Laurel Mangrum Michelle
  Steinley-Bumgarner
• State of VT Kathleen Browne Paul Dragon
• NDRI/COCE John Challis JoAnn Sacks
• Oregon Health Sciences University Dennis
  McCarty

38
DDCAT PROGRAM CATEGORIESBASELINE ASSESSMENTS
ACROSS 6 STATES
39
DDCAT DDCMHT PROGRAM CATEGORIESBASELINE
ASSESSMENTS ACROSS STATES
40
DDCAT PROFILES BASELINE ASSESSMENTS ACROSS 6
STATES
DDE
DDC
AOS
41
DDCAT NEXT STAGE RESEARCH

• Demand outpacing development (22 states)
• Expanding learning collaborative (13 states)
• Linking baseline and follow-up program measure
  (DDCAT) with patient level performance measures
  (NIATx indicators) 40 programs/10 states
• Examining effective change strategies (program
  and patient level) using Implementation Index
• (Mangrum et al) and Implementation Component
  Measure (Fixsen et al) at follow-up assessment

42
RWJ Grant 2008

• To examine the relationship between baseline
  program dual diagnosis capability and patient
  outcomes.
• To examine the relationship between follow-up
  program dual diagnosis capability and patient
  outcomes.
• To examine the implementation strategies of
  programs that enhanced dual diagnosis capability
  and/or improved patient outcomes.

43
Design for Louisiana
44
RWJ 2008 Basic Sequence of Measures
45
Why participate in Research?

• Exposure to Best Practice Implementation
  Science
• Advancing the Field
• Incentive Money
• Training on the Use of Different Instruments
  Diagnosis, Outcomes, DDCATs

46
Here in Louisiana

• DDCAT and DDCMHT Profiles
• Across the State

47
Baseline Profile
48
Size Comparison -or- Is Bigger Really Better?
49
Clinic Type Natural Strengths
50
The DDCAT Items

• Baseline Averages Across the State

51
DDCAT Reminders

• Best Practice Standards High Standards
• Emphasis on a Formal Process
• Written Organizational Plan
• Established Protocols
• Routine Practice
• Strong Documentation

52
Averages Item by Item
53
Degree of CoordinationModel for an Integrated
Systems

• One stop shop
• Key One Tx Plan
• Providers working together
• Bigger Systems Change

Person
54
Degree of CoordinationModel for Parallel Systems

• Establishing the collaborative relationship
• KEY is Regular Communications between providers
• Planned and documented

55
Collaboration

• A more formal process of sharing responsibility
  for treating a person with co-occurring
  conditions, involving regular and planned
  communication, sharing of progress reports, or
  memoranda of agreement. In a collaborative
  relationship, different disorders are treated by
  different providers, the roles and
  responsibilities of the providers are clear, and
  the responsibilities of all providers include
  formal and planned communication with other
  providers. The threshold for collaboration
  relative to consultation is the existence of
  formal agreements and/or expectations for
  continuing contact between providers.

56
Literature

• Information and Resources on Both Disorders
• Specialized information for Co-occurring Disorder
  and the interaction between disorders
• Relatively easy way to advocate for co-occurring
  disorders
• Impact, but conserves clinical resources
• Often more of a logistic issue

57
DDCAT Item by Item
58
Assessment Guidance

• Establish (rule-out) Co-occurring Disorder. The
  assessment must establish justification for
  services and yield sufficient information to
  determine or rule-out the existence of
  co-occurring mental health and substance use
  disorders. A specific diagnosis is NOT
  required.
• Results used in treatment plan. The assessment
  results must routinely be included in the
  development of a treatment plan.

59
Stage-wise Assessment the Standard

• Routine component of Assessment
• Consistent across providers
• Gets documented

Stage-wise Assessment the Reality

• Very informal
• Rarely is documented
• Variable across Clinicians

60
DDCAT Item by Item
61
Guidance on Primary Treatment Items

• Treatment plans for COD routinely (at least 80
  of the time) address both the substance related
  and mental health disorders, although the
  treatment planning for the substance related
  disorders tends to be more specific and targeted,
  mental health concerns are regularly addressed
  albeit in a somewhat non-specific fashion.
• Treatment monitoring for individuals with CODs
  regularly (at least an 80 of the time) reflect a
  clinical focus on changes in mental health
  problems BUT- This monitoring tends to be a
  basic, generic or qualitative description within
  the record.

62
Education

• The program routinely (at least 80 of the time)
  provides general education about the other
  disorder and the interaction on co-morbid
  condition. Examples include a general
  orientation to CODs, and educational lectures
  about the connections between mental health
  symptoms and substance use, as well as the
  appropriate use of psychotropic medications
  (medications are not drugs). These are lectures
  designed to inform and are not designed to treat.
  

63
DDCAT Item by Item
64
How much do we formally discharge? (This is a
tough one.)

• Discharge Criteria
• Inclusive of co-occurring issues
• Means of planning and continuing COD treatment
  needs
• Often related to what was available during
  Treatment

65
Recovery Concept - Application to COD

• Complementary to the 12-step Approach
• Given that the notion of recovery derives from
  the self-help and self-advocacy communities in
  both addictions and mental health, the first
  definition of recovery refers to what people who
  have these conditions do to manage their mental
  illness and/or addiction and to claim or reclaim
  their lives in the community. In addition to
  managing the condition, this sense of recovery
  therefore also involves what people do to
  overcome the effects of being perceived as an
  addict or a mental patientincluding rejection
  from society, alienation from ones loved ones,
  poverty, substandard housing or homelessness,
  social isolation, unemployment, loss of valued
  social roles and identity, and loss of sense of
  self and purpose in lifein order to regain some
  degree of control over their own lives.

66
DDCAT Item by Item
67
Staffing Prescriber

• Prescriber Goals
• On site
• Provider of pharmacotherapy services
• Clinical competency to treat the other disorder

• Prescriber side effects ( and -)
• Significant Resource issues
• Physician leadership
• Ad hoc clinical supervision

68
Staffing Alumni

• Goals
• Use of Graduates as Peers
• Make it a General Part of Programming

• Benefits
• Concrete Manifestation of Recovery
• Under-Utilized Resource

69
Training Basic

• The programs strategic training plan requires
  basic training in COD issues for all staff -AND-
  The majority of program staff are trained in
  these basic COD issues including the prevalence
  of CODs, screening assessment of CODs, the
  signs symptoms of CODs, and triage and
  treatment decision-making for CODs.

• No more COSIG state-wide training
• Turnover
• Transition to Essential Learning

70
More DATA

• Co-occurring Measures

71
SAMHSA Data Collection Process

• Entirely based on Documentation in Charts
• Spans April to September 2007
• Evidence of screening for COD
• Evidence for assessment of COD
• Identification of the COD Population
• Types of Treatment for COD
• Degree of Coordination in the COD Treatment

72
screening
73
assessment N 2182
74
populations N 2182
75
types of treatment N781
76
treatment sophistication N497