Service-related research: Therapy outcomes audit

1
Service-related research Therapy outcomes audit

• Sarah Howley
• Trainee Clinical Psychologist
• UCL

2
Background (1)

• Big changes in NHS current emphasis on NICE
  guidance evidence-based practice
• NICE currently does not consider practice-based
  evidence as equivalent to RCT evidence
• CBT is NICE-recommended therapy for depression
  and anxiety problems based on RCT evidence

3
Background (2)

• BUT does evidence of efficacy from RCTs prove
  the effectiveness of therapies in real-life
  clinical settings? Many say no!
• AND does absence of efficacy evidence mean
  therapies are not effective?
• There has been very little research generally
  comparing CBT (gold standard) to other
  therapies

4
Rationale for study in this Service

• No recent audit of therapy outcomes in PTS
• Little explicit information about how clients are
  allocated to CBT or Exploratory waiting lists
• Little info on comparison of therapies (PCP,
  Existential and Cognitive Behavioural therapies)

5
Aims of current study

• To investigate
• Potential factors influencing allocation to
  different therapy modalities demographic
  variables (age, gender, ethnicity), problem
  characteristics (type, severity, duration)
• Are therapies equivalent at (a) reducing CORE-OM
  scores from pre- to post-therapy and (b)
  producing reliable and clinically significant
  change across therapy?

6
Methods

• CORE-OM forms used routinely in service across
  all therapeutic modalities
• Data taken from CORE database from 2002 to 2010
• CBT sample 386, PCP 15, ExT 45,
  Exploratory therapy 5
• Therefore PCP, ExT Exploratory amalgamated
  (Exploratory Therapy - EPT N 65) and random
  sample taken from CBT dataset (N 65
  representative of full CBT dataset as determined
  by statistical analyses)

7
Methods

• Variables
• Demographics Age, Sex, Ethnicity
• Referral source (GP, PCMHT etc)
• Problem characteristics type, severity
  duration
• CORE-OM mean scores at Assessment (IAS),
  Pre-therapy (1st treatment session) and
  Post-therapy (last treatment session)

8
Sample characteristics
GROUP N (M, F) Age (SD) Age Range Primary referral source Primary ethnic group ()
CBT 65 (21, 44) 41.66 (10.87) 18 - 64 GP (49) White British (96)
EPT 65 (18, 47) 41.70 (10.90) 18 - 64 GP (52) White British (94)
9
Outcomes analyses

• CORE-OM scores within therapy groups
  (Effectiveness)
• Assessment vs Pre-therapy (change on W/L)
• Pre-therapy vs Post-therapy (therapy-related
  change)
• Reliable Change Index calculation to determine
  whether clients achieved reliable and clinically
  significant change (Jacobson and Truax, 1991)

10
Outcomes analyses

• CORE-OM scores between groups (comparison of
  therapies)
• Assessment scores do clients vary in severity
  at assessment?
• Pre-therapy scores (baseline)
• Post-therapy scores are therapies equivalent in
  terms of outcome i.e. reduction in CORE scores?
• Reliable Change Index are therapies equivalent
  in terms of achieving reliable and clinically
  significant change?

11
Results

• Allocation to therapeutic modality
• Only difference between therapy groups was in
  primary problem type
• Anxiety disorders significantly more likely to be
  found in CBT group as primary presenting problem
• Chi Square test ?2 6.65, p lt 0.01

GROUP Primary problem Secondary problem
CBT Depression (68.8) Anxiety (46) Anxiety (25) Depression (17)
EPT Depression (75.8) Anxiety (6) Anxiety (33) Inter/p problems (14)
12
Results Within groups

• Within groups
• CBT EPT significant reductions in CORE domain
  scores from IAS to pre-therapy
• Effectiveness?
• CBT - sig reduction in all CORE domain scores
  pre-post
• EPT sig reduction in all CORE domain scores
  pre-post except Risk (p .059)

Sig differences in CORE-OM domain scores Sig differences in CORE-OM domain scores
Group Ax Pre-Therapy Pre-Post
CBT Functioning Risk ALL
EPT Subjective Wellbeing ALL except Risk
13
Results Between groups

• Comparing CBT and EPT on CORE-OM
• IAS No statistically significant differences
  between groups
• Pre-therapy No statistically significant
  differences between groups
• HOWEVER In CBT group Problems/Symptoms domain
  score was below clinical cut-off
• Post-therapy CBT group had significantly lower
  Problems/Symptoms and All Items scores compared
  to EPT

14
Comparing CORE-OM scores
STAGE Assessment Assessment Pre-therapy Pre-therapy Post-therapy Post-therapy
Subjective Wellbeing -0.39 p lt .05 -0.32 n.s. -0.34 n.s.
Problems/ Symptoms -0.12 n.s. -0.24 n.s. -0.39 p lt .05
Life Functioning -0.17 n.s. -0.20 n.s. -0.22 n.s.
Risk -0.14 n.s. -0.14 n.s. -0.17 n.s.
ALL Items -0.20 n.s. -0.17 n.s. -0.27 p lt .05
15
Reliable Change

• Jacobson Truax (1991) Method used to identify
  whether individual clients achieve reliable and
  clinically significant change (improvement OR
  deterioration)
• This indicator of clinical significance is
  distinct from statistical significance
• There was no statistically significant difference
  between the groups in terms of no. of clients
  achieving reliable improvement

GROUP Improvement No change Deterioration
CBT (n 53) 45.3 54.7 0
EPT (n 59) 50.8 44.1 5.1
16
Recap of results

• No differences observed in age, sex, ethnicity,
  referral source, problem severity or chronicity
  between therapy groups
• Anxiety disorders more likely to be main problem
  in CBT group
• Both therapies showed statistically significant
  improvement in CORE scores across domains (apart
  from EPT group in Risk but score below clinical
  cut off)
• CBT group showed significantly more improvement
  on Problems/Symptoms domain (but were below
  clinical cut off at pre-therapy)
• No statistically significant difference in number
  of clients achieving reliable clinical
  improvement between groups

17
Interpretation of results

• Clients presenting with anxiety as main
  difficulty tend to be referred more often to CBT
  Service is in line with NICE guidance on
  treating anxiety
• Other factors in allocation to W/L client
  preference, length of waiting lists, assessors
  preferred model?
• No significant deterioration in Risk scores in
  CBT or EPT groups effects of waiting list times
  on client risk?
• CBT targets specific symptoms therefore
  unsurprising that CBT clients show more reduction
  in this domain? (assuming result is valid small
  effect size)
• Both CBT and EPT are effective in reducing
  CORE-OM scores from pre- to post-therapy in this
  Service and clients in both achieve comparable
  levels of reliable improvement

18
Limitations

• Relatively small sample sizes (n 65 and smaller
  for most analyses due to missing data)
• Use of data from up to 10 years ago does this
  reflect current service?
• Amalgamation of PCP, ExT exploratory
  difficult to draw conclusions (but unavoidable!)
• Use of non-parametric stats (increased risk of
  not finding significant differences i.e. type 1
  error)

19
Recommendations

• Encourage all clinicians to record CORE-OM scores
  at each stage of therapy vital to
  practice-based evidence for therapies offered
  here
• Carry out similar audits at regular intervals in
  order to establish a bedrock of practice-based
  evidence