Just the facts, maam: In search of FACT fidelity

1
Just the facts, maam In search of FACT
fidelity

• Joe Morrissey
• University of North Carolina at Chapel Hill
• Festschrift for Gary Bond
• IUPUI, Indianapolis, IN
• September 23, 2009

Funding from the NIMH and MacArthur Foundations
Mental Health Policy Research Network is
gratefully acknowledged.
2
Just the facts, maam

• Joe Fridays signature directive in 1950-60s TV
  docu-drama about LAPD
• Searching for FACT fidelity has been a lot like
  detective work . . . cajoling findings from
  unruly data

3
Some facts about FACT

• Forensic adaptations of ACT (FACT) are one of the
  latest efforts to keep persons with severe mental
  illness out of jail
• Despite rapid dissemination, current evidence
  about FACTs public safety and mental health
  effects is weak
• Today, Id like to add to that evidence base and
  discuss future prospects for FACT fidelity
• Throughout this work Gary Bond has been an
  inspiration . . .

4
Gary Bonds First Principles of Mental Health
Services Research

• 1. Any untested service intervention should not
  be demonstrated because it will fail.
• 2. A social experiment is a contrivance that
  when applied to a group of people leads to a
  scientific publication.

5
Mentally ill jails

• People with mental illness in the criminal
  justice system have become the new frontier for
  community mental health interventions
• Now, more than 1 million jail bookings of people
  with SMI each year SMI prevalence 14.5 male
  31.0 female detainees (Steadman et al., 2009)
• Relative risk of persons with SMI being jailed is
  150 greater than being hospitalized (Morrissey
  et al., 2007)

6
Current ACT evidence

• Bond colleagues (2001) 8 of 10 trials, usual
  care equal-to-or-better-than ACT on arrests
  jail use
• Calsyn colleagues (2005) ACT no-better-than
  usual care on range of CJ outcomes
• Chandler Spicer (2007) IDDT no-better-than
  usual care on range of CJ outcomes

7
Significant ACT Outcomes in 25 RCTs pre-2000 n
of trials,
Source Bond, Drake, Mueser Lattimer, 2001
8
ACT as an intervention platform

• Bond colleagues (2001) review also showed weak
  effects of ACT on substance abuse vocational
  functioning outcomes
• But when ACT teams were retrofitted to address
  these issues (via IDDT supported employment)
  subsequent RCTs showed positive effects
• Would same thing happen if ACT was retrofitted to
  prevent arrests and incarcerations?

9
ACT to FACT Adaptations

• DACT Core Items
• Mobile/comprehen-sive services/team-based
• Psychiatrist on team
• 1-10 S-to-C ratio
• 24/7 crisis response
• Time unlimited

• FACT
• Same
• Same
• Same
• Same
• Same
• Prior arrests
• CJ referrals
• CJ partners
• Court sanctions

10
Other Differences

• ACT
• Target SMI _at_ risk of hospitalization
• Goal Prevent hospitalization sustain community
  living
• Vocational, AOD staffing on team
• DACT fidelity standards

• FACT
• Target SMI _at_ risk of arrest/ jail detention
• Goal Prevent re-incarceration
• Less vocational, linked AOD services
• Probation officers as team members
• No clinical model or fidelity standards

11
Current FACT Evidence

• Separate pre-post studies (no control groups)
  with small samples, FACT associated with fewer
  jail days, arrests, hospital days, and
  hospitalizations (Lamberti et al., 2001 Weissman
  et al., 2004 McCoy et al., 2004)
• Still no published reports based on rigorous
  comparison group data clearly showing FACT can
  improve both mental health public safety
  outcomes

12
Our efforts. . .

• Birmingham study (2004-05)
  abortive effort to retrofit the
  first RCT (so we thought) on a SAMHSA jail
  diversion site
• FACT survey and site visits (2005-06)
  surveyed 30 ACT CJ programs, visited 12
  FACT programs to document operating
  characteristics and sustainability
• Mentally Ill Offender Crime Reduction Grant
  (MIOCRG) program (2006-09)
  discovered 20 RCTs and opportunity to
  re- analyze data from several counties in
  California

13
MIOCRG initiative

• California Board of Corrections 30 county, 80.5
  million program 2000-04, Sheriffs Assn. and MH
  Association bill sponsors
• Goals identify what works most effectively in
  reducing recidivism among mentally ill offenders
• Local evaluations with random assignment
  individual data on 8,000 enrollees reported to
  BOC for cross-site evaluation
• BOC Report ACT-like programs improved CJ and MH
  outcomes, but major sampling and statistical
  problems not addressed

14
MIOCRG re-analysis saga

• Find Calif. counties w true FACT models
• Get local evaluators to share data
• Get CA-DMH to agree to create linked,
  de-identified services data files Get IRB
  approvals at UNC-CH CA-DMH
• Get county MHAs to approve re-analysis plan
  send study IDs to DMH for record linkage
• Obtain link data across counties with common
  prospective cohort format with common variables
• Run individual site pooled analyses

15
Found six MIOCRG sites that allowed for FACT v.
FICM comparisons
Police or probation officer on the team
16
Best laid plans go awry . . .

• CA-BOC failed to implement a true experimental
  study we had access to the MIOCRG data for all
  30 sites but we couldnt make sense of it
• Ended up working with three sites with same
  evaluator, but even then, the CA-BOC design led
  to incomplete data and we were unable to fix that
  for 2 of 3 FACT sites
• So, we resorted to administrative data to assess
  impact of FACT at 1 site

17
Setting and design

• Mid-size city
• FACT program (2000-03) enrolled consumers from
  county jail probation officers on team
• DACT scores of 4.5 and 4.6
• Retained MIOCRG randomized groups for our
  analyses FACT v. treatment as usual (TAU)
• Followed both groups in administrative data 12
  mos. pre and 12 24 months post

18
Administrative data elements

• Mental Health
• Service utilization
• psych hosp days
• crisis contacts
• outpatient visits
• Costs

• Public Safety
• Jail use arrests
• bookings
• felony/misdemeanor charges convictions
• jail days
• Costs

19
Study sample

• Participants had histories of frequent jail use
• But they also had a lot of mental health services
  use in baseline period
• Random assignment to FACT and TAU worked to
  produce two equivalent groups (age was only
  significant difference but it didnt matter in
  multivariable analyses)

20
Sample randomization results
plt.05
21
Results

• Compared to TAU participants
• FACT participants had fewer bookings (plt.01) and
  jail days (plt.05) in each year.
• FACT participants had more outpatient visits
  (plt.001) but fewer days of hospitalization
  (plt.05) and incurred lower overall costs for the
  county jail and the county mental health service
  system.

22
Conclusion

• A forensically-oriented, high-fidelity ACT (FACT)
  team can alter the criminal justice involvement
  of offenders with serious mental illness, reduce
  their time spent in inpatient psychiatric
  settings, while providing more appropriate and
  less costly outpatient services.

23
Implications for FACT fidelity

• Single RCT is never definitive, but it helps to
  elevate the evidence base
• FACT works, but it is expensive and it should be
  carefully targeted to those most in need, not
  everyone who ends up in jail
• More needs to be done to specify and test a
  clinical model for FACT then, fidelity issues
  become meaningful
• Some feel criminogenic needs should be targeted
  via a CBT add-on to FACT further research needed
  here

24
An interplanetary travelers advice

• In doing meaningful services research, try to .
  . . Fill whats empty. Empty whats full. And
  scratch where it itches.
• The really important thing is . . . not to stop
  questioning.
• The best is yet to come.

25
Acknowledgements . . .

• All personal attributions herein are apocryphal .
  . .they have been gleaned, stolen, modified,
  invented, and filched from various sources to fit
  the occasion!