The University of Manchester

1
Methods of explanatory analysis for psychological
treatment trials workshop

• The University of Manchester
• 17th September 2009
• Presented by
• Graham Dunn, Richard Emsley, Andrew Pickles Ian
  White

Funded by MRC Methodology Grant G0600555 MHRN
Methodology Research Group
2
Funding
MRC Methodology Research Programme Design and
methods of explanatory (causal) analysis for
randomised trials of complex interventions in
mental health (G0600555 ) G Dunn (PI), R Emsley,
L Davies, J Green, A Pickles, C Roberts, I White
F Windmeijer (with collaborators C
Barrowclough, R Bentall, S Birch P Garety).
3
Explanatory versus Pragmatic questions

• Pragmatic
• Effectiveness what is the effect of offering
  treatment?
• Analyse as randomised Intention-to-treat (ITT)
• Explanatory
• Efficacy what is the effect of receiving
  treatment?
• Mechanisms how do the treatments work?
• What are the potential mediators?
• How might process variables induce
  treatment- effect heterogeneity?
• We are dealing with an observational study
  embedded within a randomised trial.

4
The central problem

• Treatment receipt, mediators and other putative
  process
• variables are not under the direct control of the
  
• investigator. They are not randomised.
• The effects of these intermediate variables are
  likely to be
• subject to confounding (i.e. there are other
  variables
• frequently not measured that influence both the
  
• intermediate variable and the final outcome).
• If we cannot measure all of the confounders then
  we have
• hidden confounding or hidden selection effects
• (selection on unobservables).

5
Distinguishing treatment-free prognosis from
treatment effects

• A typical example from the literature
• An RCT demonstrates that CBT is effective.
• Investigators have measured the strength of
  alliance in the
• CBT arm.
• Outcome and alliance are correlated in the CBT
  arm.
• So what? This tells us nothing about treatment
  effects.
• Those capable of forming a strong alliance might
  have had
• the best outcomes in the absence of therapy.
• The correct question is What is the moderating
  effect of the alliance on the effect of
  treatment? Not a question that is easy to
  answer.

6
Missing data and other complications

• Missing data closely associated with
  non-compliance and
• other process variables.
• Participants who do not comply with their
  treatment
• allocation are less likely to provide outcome
  data.
• Participants with a poor working alliance with
  their
• therapist are less likely to attend the full
  course of therapy
• and may also be less likely to provide outcome
  data.
• Intermediate variables will be measured with
  error.

7
Potential mediators

• What are the participants beliefs?
• Does psychotherapy change attributions
  (beliefs), which, in turn, lead to better
  outcome? How much of the treatment effect is
  explained by changes in attributions?
• What is the concomitant medication?
• Does psychotherapy improve compliance with
  medication which, in turn, leads to better
  outcome? What is the direct effect of
  psychotherapy?
• What is the concomitant substance abuse?
• Does psychotherapy reduce cannabis use, which in
  turn leads to improvements in psychotic symptoms?

8
Process measuresSources of treatment-effect
heterogeneity

• Compliance with allocated treatment
• Does the participant turn up for any therapy?
• How many sessions does she attend?
• Fidelity of therapy
• How close is the therapy to that described in
  the treatment manual? Is it a cognitive-behavioura
  l intervention, for example, or merely emotional
  support?
• Quality of the therapeutic relationship
• What is the strength of the therapeutic or
  working alliance?

9
The plan for the day

• A mixture of lectures and general discussion
• Ian will start by introducing causal inference
  and looking
• at the treatment effects in those who comply with
  their
• treatment allocation (Complier Average Causal
  Effects).
• Andrew will introduce concepts of mediation and
• moderation.
• Richard will describe the use of instrumental
  variable
• methods.
• Graham will describe the application of principal
  
• stratification.

10
Level of difficulty

• We realise that most of our audience are not
  statisticians.
• Our aim is to introduce key concepts and to
  describe the
• logic and philosophy behind the modern approaches
  to
• causal inference.
• There will some technical details that not
  everyone will
• easily follow, but we will try to keep
  mathematics to an
• absolute minimum.
• We will provide software scripts (or details of
  where to find
• them) for those who are interested in having a go
  themselves, but we will not require any prior
  knowledge of any particular software packages.