Modelling treatmenteffect heterogeneity in psychological treatment trials

1
Modelling treatment-effect heterogeneity in
psychological treatment trials

• Graham Dunn
• Biostatistics Group, University of Manchester
• in collaboration with Richard Bentall
  (Psychology)
• March 2007

2
Modelling the effects of complex interventions
3
Motivating Example - SoCRATES

• MRC SoCRATES Trial psychological interventions
  for schizophrenia.
• 3 groups treatment as usual (TAU), supportive
  counselling (SC), cognitive behaviour therapy
  (CBT). Here consider Control (C) vs. Treatment (T
  SC/CBT).
• Three centres (Liverpool(1), Manchester(2)
  Nottingham(3)).
• Prognostic variables Baseline PANSS, logDUP and
  years of education.
• Record of number of sessions attended (S).
• Record of level of therapeutic alliance at 4
  weeks (A).
• Outcome (Y) PANNS total score at 18 months.

4
SoCRATES Means

• Complete cases (apart from Contol Group CALPAS)
• Centre 1 Centre 2 Centre 3
• N68 N84 N49
• Control Treated Control Treated Control Treated
  
• N39 N29 N35 N49 N26 N23
• Baseline
• LogDUP 1.1 1.3 1.4 1.4 0.8 0.8
• Years education 11.3 11.4 12.7 11.7 11.7 10.8
• PANSS 80.1 77.7 98.0 100.5 84.9 83.4
• Post-randomisation
• CALPAS (A) - 5.7 - 5.1 - 5.2
• Sessions (S) 0 18.1 0 16.2 0 13.9
• Outcome 18m
• PANSS (M) 69.5 50.2 73.2 74.4 54.5 49.1

5
Mediators

• Mediators are intermediate outcomes on the causal
  pathway between allocation to or receipt of
  treatment and final outcome.
• By definition, in an RCT, they are measured after
  randomisation.
• If part of the explanation of efficacy then only
  relevant if the patient receives treatment.
• Treatment effect may be fully or partially
  explained by a given mediator.
• Possibility of multiple mediators (multiple
  pathways) and interactions between mediators.

6
Incomplete Mediation (with hidden confounding)
Receipt of Therapy
Thoughts
Mood
dX
dY
U
If receipt of treatment randomised then
assumption of no confounding of treatment
received with other variables is justified.
7
Incomplete Mediation (the traditional model)
Receipt of Therapy
Thoughts
Mood
dX
dY
e.g. Baron and Kenny (1986).
8
Mediation Direct and Indirect Effects

• Although there is an enormous methodological
  literature on the
• estimation of the effects of mediators, most of
  it completely ignores the
• technical challenges raised by potential hidden
  confounding of mediator(s) and
• outcome.
• In particular, the traditional approaches to the
  investigation of mediation, and
• the accompanying estimation of direct and
  indirect effects of treatment
• (typically using multiple regression or linear
  structural equation modelling)
• depend on the implicitly-assumed absence of
  hidden confounding.
• The assumptions concerning the lack of hidden
  confounding and measurement
• errors are very rarely stated, let alone their
  validity discussed.
• One suspects that the majority of investigators
  are oblivious of these two
• requirements.
• One is left with the unsettling thought that the
  thousands of investigations of
• mediational mechanisms in the psychological and
  other literatures are of
• unknown and questionable value.

9
Complete Mediation
Offer of Treatment
Sessions Attended
Symptoms
dY
dZ
U
If offer of treatment made at random then there
is no confounding with sessions or symptoms.
Here randomisation is an instrumental variable.
10
Treatment-effect modifiers 1. Moderators

• Moderators are pre-randomisation
  characteristics that influence the effect of
  treatment.
• They are baseline effect-modifiers.
• Possible to get moderated mediation.
• Possible examples sex, age, previous history
  of mental illness, treatment centre, therapist,
  etc.

11
Effect Modification Moderated Mediation
Offer of Treatment
Therapist or Centre
Sessions Attended
Symptoms
dX
dY
U
12
Treatment-effect modifiers 2. Process variables

• These are post-randomisation variables that
  influence the effect of treatment.
• They are post-randomisation effect- modifiers.
• Possible examples therapeutic alliance,
  adherence to therapeutic model.

13
Challenges!
14
Effect Modifier- Mediator interactions
da
Offer of Treatment
Alliance
U2
Sessions Attended
Symptoms
dS
dY
U1
Hidden confounders, U1 and U2, are likely to be
correlated If randomised, offer of treament is,
again, an instrumental variable.
15
The identification problem

• Our model is not identified. There are too many
  parameters to be estimated given the naure of the
  data.
• We need to be able to find additional variables
  which influence sessions and alliance but have no
  direct effect on outcome (more instrumental
  variables). We need multiple instruments.

16
Effect Modifier- Mediator interactions
da
IVs
Offer of Treatment
Alliance
U2
Sessions Attended
Symptoms
dS
dY
U1
Hidden confounders, U1 and U2, are likely to be
correlated
17
Multiple IVsWhere do we get them from?

• Several options
• Randomisation involving more than one active
  treatment i.e. to interventions specifically
  targeted at particular mediators.
• Randomisation-by-baseline variable interactions.
  Randomisation-by-Centre, for example.
• Randomisation-by-trial (multiple trials).
• Not really relevant here but an interesting
  possibility, for other types of trial, is
  genotyping (so-called Mendelian Randomisation).

18
Counterfactuals
19
Individual treatment effects
20
A simple structural (causal) model

• First consider a simple linear dose-response
• model (ignoring possible effects of A) for
• the ith subject, the effect of attending the
• number of sessions, s, is given by
• E(?i Sis) ßss
• No sessions no treatment effect
• (an exclusion restriction)

21
The influence of Alliance

• With a quantitative effect modifier, A, the
  linear
• structural model we deal with here has the form
• E(?i Sis Aia) ßss ßsasa
• Again, no sessions no treatment effect
• (an exclusion restriction)
• The effect of alliance is multiplicative

22
Effect Modifier- Mediator interactions
da
IVs
Offer of Treatment
Alliance
U2
Sessions Attended
Symptoms
dS
dY
U1
Hidden confounders, U1 and U2, are likely to be
correlated
23
Estimation

• Two-Stage Least Squares
• (2SLS) (ivreg in Stata, for example)
• - needs a slightly different approach to cope
  with missing outcome data
• Structural Mean Models (SMM)
• - G-estimation (Goetghebeur et al.).

24
Stata ivreg commands(complete case analysis)

• y is outcome (18-month PANSS total)
• g is treatment group
• (control0 treated1)
• c1 and c2 are centre dummies
• bp baseline PANSS total
• ld logDUP
• ed years of education
• s is sessions attended
• sa is product of sessions and alliance (N.B. zero
  if g0) (Alliance is observed alliance score
  minus max observed value)
• ivreg y c1 c2 bp ld ed (s sag c1g c2g bpg ldg
  edg)
• c1, c2, bp, ld and ed influence s, sa and y
• the instruments are g (group), c1g, c2g (group by
  centre interactions), bpg (baseline PANSS by
  group interaction), ldg (logdup by group
  interaction) and edg (years of education by group
  interaction they influence s and sa, but have
  no direct effect on y

25
Stata ivreg results(complete case analysis)

• ivreg y c1 c2 bp ld ed (s sag c1g c2g bpg ldg
  edg)
• ßS -2.40 (se 0.70) ßSA -1.28 (se 0.48)
• In contrast (assuming no hidden confounding)
• regress y c1 c2 bp ld ed s sa
• ßS -0.95 (se 0.22) ßSA -0.39 (se 0.11)
• Here we see attenuation when using the incorrect
• model, but when we ignore hidden confounding it
  is also
• possible to see a sign reversal!

26
Stata ivreg results, contd.

• ßS -2.40 (se 0.70) ßSA -1.28 (se 0.48)
• When A0 (the maximum alliance score)
• slope for effect of Sessions is -2.40
• When A-7 (the minimum alliance score)
• the slope is -2.40 71.28 6.56
• This suggests that when alliance is very poor
  attending
• more sessions makes the outcome worse!

27
Comments

• SMM (G-estimation) gives identical results.
• All IV methods allow for measurement errors in
  the mediators.
• If look at marginal effects of alliance then we
  add a constant term to the normal treatment
  effect model (i.e. drop exclusion restriction).
• ivreg y c1 c2 bp ld ed g (ac1g c2g bpg ldg edg)

28
Simulated data set (N1000)

• ? -0.5sessions (i.e. no effect of alliance)
• Strong hidden selection effect
• Baseline predictors of sessions, alliance, etc
• x1, x2, x3
• Correlations
• Outcome Sessions Alliance
• Outcome 1.00
• Sessions -0.10 1.00
• Alliance -0.52 0.43 1.00

29
Results from simulated data

• regress outcome alliance x1 x2 x3 if group1
• Alliance effect estimate -0.94 (se 0.09)
• regress outcome alliance group x1 x2 x3
• Alliance effect estimate -0.22 (se 0.04)
• ivreg outcome x1 x2 x3 group (alliancex1g x2g
  x3g)
• Alliance effect estimate -0.09 (se 0.04)

30
Where now?

• We need to extend to longitudinal data.
• We need to deal with missing data (in both
  process measures and outcomes. (If weeks in
  treatment less than 4, there is no alliance
  measure). Extensions of latent ignorability?
• We need to think more carefully about designs.

31
References

• Baron, R.M. Kenny, D.A. (1986). The
  moderator-mediator variable distinction in social
  psychological research conceptual, strategic,
  and statistical considerations. Journal of
  Personality and Social Psychology 51, 1173-1182.
  Classic in the field but ignores hidden
  confounding/selection
• Dunn, G. Bentall, R. (2007). Modelling
  treatment-effect heterogeneity in randomised
  controlled trials of complex interventions
  (psychological treatments). Statistics in
  Medicine 2007, in press.
• Fischer-Lapp, K., Goetghebeur, E. Practical
  properties of some structural mean analyses of
  the effect of compliance in randomized trials.
  Controlled Clinical Trials 1999 20 531-546.
• Gennetian, L.A., Morris, P.A., Bos, J.M. Bloom,
  H.S. (2005). In H.S. Bloom (Ed.), Learning More
  From Social Experiments (pp75-114). New York
  Russell Sage Foundation. design issues
  multiple IVs by design
• Kraemer, H.C., Fairburn, C.G. Agras, W.S.
  (2002). Mediators and moderators of treatment
  effects in randomized clinical trials. Archives
  of General Psychiatry 59, 877-883. recent
  general discussion
• TenHave, T., Joffe, M. Lynch, K. (2005). Causal
  mediation analysis with structural mean models.
  University of Pennsylvania Working Paper on
  Biostatistics (see http//www.biostatsresearch.com
  /upennbiostat/papers/art1) partial mediation in
  presence of hidden confounding/selection