Linking Multiple Administrative Data Sets to Measure Outcomes for Behavioral Health Services

1
Linking Multiple Administrative Data Sets to
Measure Outcomes for Behavioral Health Services
Tracy Leeper, Grant Projects Manager
Oklahoma Department of Mental Health and
Substance Abuse Services
2
Outcomes Substance Abuse Treatment Outcomes
Monitoring System

• Funded by CSAT
• Focused on Administrative Data
• All Adults Who Receive SA
  Treatment Services in a Given Year
• Two Year Pre Post Follow-Up Periods
• Determine Costs Associated with Outcomes

3
Administrative Data

• Ongoing Sustainable
• Empirically-Based Objective
• Less Resource Intensive
• Longitudinal
• Flexible for Populations Time Frames
• Unobtrusive

4
Administrative Data

• No Direct Measures for Certain Indicators
• Little Drug Use Information
• Level of Functioning or Symptomotology
• Indirect Measures
• Look at Contribution to Society Versus Cost to
  Society
• Employed, Paying Taxes
• Decrease in Jail and Inpatient Days

5
Sources of Data

• DMHSAS - State-funded AOD Mental Health
• DPS DUI Convictions
• DOC Incarcerations, Probation Parole
• OTC Household Income
• OESC Wage Unemployment Benefits

6
Sources of Data

• OSDH - Mortality Data
• OHCA Medicaid Claims
• OSBI Arrests/Dispositions
• DHS TANF, Food Stamps, Child Welfare

7
Getting Buy-In

• Legislation
• Interagency Pilot Projects
• Quid Pro Quo
• Good Ol Boy Approach

8
Getting Buy-In

• Leave Plenty of Time for
• Executing Agreements
• After Administrative OK, Still Need MIS Staff
  Buy-In
• Discussions
• Working Out Details
• Must Have a Working Relationship with Other
  Agencies MIS Staff

9
Confidentiality Concerns

• 42 CFR
• HIPAA
• Interagency Agreement
• IRB Approval

10
42 Codified Federal Regulation (CFR)
Subchapter A, Part 2 Confidentiality of Alcohol
and Drug Abuse Patient Records Subpart D
Disclosures without Patient Consent

• Medical Emergencies
• Research Activities
• Audit and Evaluation Activities

11
Health Insurance Portability and Accountability
Act of 1996 (HIPAA)
C. General Risk. ( 164.506) Covered entities
would be able to use or disclose an individuals
protected health information without
authorization for treatment, payment, and health
care operations.
B. Definitions. (160.103 and 164.504) 16.
Health Care Operations.
- Conducting quality assessment and improving
activities, including evaluating outcomes, and
developing clinical guidelines.
12
Interagency Agreement

• I. Purpose
• II. Effective Dates
• III. Confidentiality
• IV. Transfer of Data
• V. Specify Persons Working with Data
• VI. Modification or Amendment
• VII. Signatures

13
Data SecurityHandling and Storage

• Receive Data In A Variety Of Ways
• Tapes, CDs, Disks, FTP, E-Mail
• Dedicated File Server
• Limited Users
• Secure Location
• Separate Identifiers and Outcome Data

14
Cleaning Preparing the Data(Good News, Bad
News)

• Data Sets Are Not Well Documented
• Many Variables Are Defined Differently
• Data Formats Change Periodically (i.e.,
  Constantly)

15
Cleaning Preparing the Data(Good News, Bad
News)

• Recode the Variables
• 01/01/1999 vs. 01-01-99 vs. 990101
• Unduplicate the Records
• Allow for Aliases
• Format Names

16
Linking The Data
Methods

• Exact (All-Or-Nothing)
• Probabilistic
• Combination of the Two
• Overlap Procedure (PPE)

17
Linking The Data

• Block by Sex and DOB Components
• Sex Month Year
• Sex Day Year
• Sex Month Day
• Assign Weights to All Identifying Variables

18
Weighting The Data

• Positive Negative Weights
• Jaro, Matthew. Statistics in Medicine, Vol. 14,
  491-498 (1995).
• Determine Threshold of Likelihood
• Gray Area Manual Inspection
• Matching Algorithm Changes Depending on Available
  Variables

19
Weighting The Data

• Perfect Match 69 pts.
• SSN (all digits) 22.55
• SSN (8 digits) 14.51 pts.
• SSN (7 digits) 6.07 pts.

or -2.38 pts.
20
Weighting The Data

• Last or Maiden Name 9.58 or 3.62 pts.
• First Name 6.69 or 3.27 pts.
• Middle Initial 3.65 pts.
• Date of Birth 6.22 pts.

21
Data SecurityHandling and Storage

• Determine What Type of Results You
  Want Before You Start
• What Type of Analysis Do the Data Allow?
• What Do the Stakeholders Want?
• What Type of Data Structure Do You Need?
• Large Data Sets
• Need Lots of Hard Drive Space Memory
• Need Appropriate Software

22
Treatment Groups
Treatment Completers Planned Discharge more
than 5 units of Service Treatment
Non-completers No Planned Discharge more
than 5 units of Service Minimally Treated 5 or
fewer units of Service Detox Only
Detoxification Services Only
23
Disseminating The Results

• Work Closely with Participating Agencies
  Disclose Results To Them
• Think of Political Ramifications
  for All Agencies Involved
• Explicitly State What You Want the Audience to
  Get from the Results

24
Lessons Learned Along The Way

• Allow Much More Time Than You Think At Each Step
• Be Flexible Willing to Start Over
  Or Live With It
• Meet With Other MIS Staff
• Be Willing to Evolve With Technology and
  Experience

25
Special Interest Groups

• Managed Care vs. Traditional System
• Mental Health Court Involvement
• PACT/ACT Clients
• Persons with Co-Occurring

26
Additional Uses

• Case-Mix Analysis
• Cost Analysis
• Interstate Cooperative Study
• Parity Race, Gender, Geography

27
EVALUATING OUTCOMES OF CLIENTS RECEIVING BOTH
SUBSTANCE ABUSE AND MENTAL HEALTH TREATMENT

• Oklahoma Department of Mental Health
• and Substance Abuse Services
• Becki Moore and Tracy Leeper

28
Purpose

• To determine whether outcomes for clients with
  dual treatment differ from those of clients
  receiving mental health or substance abuse
  treatment only
• To demonstrate the feasibility and utility of
  using administrative data for different
  populations and outcomes

29
Methods

• Study Group
• Three Cohorts
• mental health treatment
• substance abuse treatment
• dual treatment
• Adults, 18 years of age or older, admitted to
  treatment during FY97 (N28,104)

30
Methods

• Administrative Data
• Two-year Pre- and Post-Treatment Study Periods

31
Sources of Data

• DMHSAS - State-funded SA MH
• OSDH - Mortality
• DOC - Incarcerations, Probation Parole
• OSBI - Arrests
• OESC - Wages

32
Clients
N14,920
N11,207
N1,977
Cohort
33
Clients
34
Mortality
Percent of Clients Who Died During the Two Years
Following FY97 Admissions Age-Adjusted to the
State Rate
Crude two-year state rate (1997, 1998)
35
Mortality
36
Mortality
37
Mortality
38
Employment
39
Employment
40
Employment