Management Practices in Addiction Treatment Programs

1
Management Practices in Addiction Treatment
Programs

• K. John McConnell, PhD
• Oregon Health Science University

Supported by a grant from NIDA 1R01DA020832
2
Two research threads

• Substance Abuse
• A call for research on the business of
  addiction treatment (Kimberly McLellan, JSAT
  2006)
• Economics
• Typically treat management quality as a fixed
  effect
• New research to measure management practice
  (Bloom Van Reenen, QJE 2007)
• Innovative telephone survey to score management
  practice
• Strongly correlated with firm performance (gt700
  firms in 4 countries)

3
Study Objective

• Adapt management practice survey tool to
  addiction treatment programs
• Are better management practices associated with
  better client treatment?

4
Study Sample

• Programs from the NIATx 200 project
• NIDA 1R01DA020832
• Approximately 200 agencies in 5 States (MA, MI,
  NY, OR, WA) randomized to different levels of
  support (coaching, interest circles, learning
  sessions, and combinations of all 3)
• We use data on 147 surveys by 3 interviewers
• 20 programs not interviewed because they were
  part of a multi-site agency
• Excluded 5 programs with different interviewers
• Excluded 5 with interview scored as not
  reliable
• Excluded 20 with missing data

5
Steps to measure management practices

• Developing management practice scoring
• Adapted scorecard for 14 questions on 4 areas
• Client intake and retention
• Quality improvement data monitoring
• Program targets
• Incentives employee management
• 60 minute phone interview of program executive
  sponsors
• Obtaining unbiased responses
• Interviews run by 3 MPH level research assistants
  
• Double-blind
• Interviewers do not know program performance
• Interviewees are not informed (in advance) they
  are scored
• Open-ended question format

6
(13) Promoting high performers Tell me about your promotion system. What about poor performers do they get promoted more slowly? Are there any examples you can think of? How would you identify and develop (i.e. train) your star performers? If two people both joined the agency 5 years ago and one was much better than the other would he/she be promoted faster? (13) Promoting high performers Tell me about your promotion system. What about poor performers do they get promoted more slowly? Are there any examples you can think of? How would you identify and develop (i.e. train) your star performers? If two people both joined the agency 5 years ago and one was much better than the other would he/she be promoted faster? (13) Promoting high performers Tell me about your promotion system. What about poor performers do they get promoted more slowly? Are there any examples you can think of? How would you identify and develop (i.e. train) your star performers? If two people both joined the agency 5 years ago and one was much better than the other would he/she be promoted faster? (13) Promoting high performers Tell me about your promotion system. What about poor performers do they get promoted more slowly? Are there any examples you can think of? How would you identify and develop (i.e. train) your star performers? If two people both joined the agency 5 years ago and one was much better than the other would he/she be promoted faster?
Scoring grid Score 1 People are promoted primarily upon the basis of tenure Score 3 People are promoted upon the basis of performance Score 5 We actively identify, develop and promote our top performers
Examples An agency based on an individuals commitment to the company measured by experience. An agency has no formal training program. People learn on the job and are promoted based on their performance on the job. At one agency each employee is given a red light (not performing), amber light (doing well and meeting targets) a green light (consistently meeting targets very high performer)
7
Analysis

• Negative binomial, clustered std. err.
• Outcome Time to treatment
• Days from first phone call to first appointment
• Pseudo-patient phone calls by NIATx researcher
• Caller identified herself as NIATx researcher
• Average of 7 calls made to each program
• Independent variables
• Average z-score of management practices across 14
  questions
• z-score removes scaling issues between questions
• Employee FTE
• State interviewer fixed effects
• Day of week for pseudo-patient phone call
• Month of survey

8
Results
9
Distribution of management practice scores for
147 addiction treatment programs
10
Double-scored interviews(N 14)
11
Model results

• Main analysis
• Time-to-treatment
• 1081 observations on 147 programs
• Management score significantly associated with
  time-to-treatment (P lt0.01)
• Sub-analyses
• 7 of 14 management practices were correlated with
  better time to treatment
• Of the 4 areas, 3 were associated with better
  time-to-treatment
• Client intake and retention
• Quality improvement data monitoring
• Program targets
• One was not
• Incentives employee management
• ?

12
Limitations

• Endogeneity
• Double scoring vs. test-retest
• Programs selected from NIATx 200 project

13
Concluding thoughts

• Management practices can be measured and are
  associated with client treatment
• Are they associated with financial outcomes?
• Do employee incentives matter or are substance
  abuse treatment programs different?
• NIATx aims for process improvement can we design
  interventions to improve management practices?

14
Thank you

• University of Wisconsin
• David Gustafson, PI
• Andy Quanbeck
• Alice Pulvermacher
• Jay Ford
• Anna Wheelock
• Renee Hill

• Oregon Health Science University
• Dennis McCarty
• Kim Hoffman
• Marie Shea
• Gretchen Luhr
• Susan Rosenkranz
• Traci Rieckmann
• Stanford University
• Nick Bloom

Research supported by NIDA R01DA020832