Randomisationbased efficacy estimators in randomised trials: when are they useful

1
Randomisation-based efficacy estimators in
randomised trials when are they useful?

• Ian White
• MRC Biostatistics Unit, Cambridge
• RSS Manchester local group, 20th April 2005

2
Setting for todays talk

• Randomised controlled trial to evaluate an
  intervention
• Departures from randomised intervention
• non-compliance
• changes in prescribed treatment
• Want to infer causal effect of treatment
• Ill discuss medical applications, but the
  methods apply in any (quasi-)experiment

3
Types of departure from randomised intervention

• Switches to other trial treatment
• Or changes to non-trial treatment
• regard nothing as a treatment!
• Departures may be
• yes/no or quantitative
• constant or time-dependent

? FOCUS ON SWITCHES
4
Plan of talk

• Methods
• Intention-to-treat analysis
• Per-protocol analysis
• Randomisation-based efficacy estimators (RBEEs)
• Examples where RBEEs are useful
• Patient information
• Treatment-time interactions
• Treatment-covariate interactions
• Cost-effectiveness analysis
• Obstacles to wider use of RBEEs

5
Intention-to-treat analysis(ITT)
6
Intention-to-treat analysis

• Compare groups as randomised, ignoring any
  departures
• Now the standard analysis and rightly so
• Advantages
• respects randomisation
• avoids selection bias
• answers an important pragmatic question e.g. what
  is public health impact of prescribing X?
• Disadvantages
• may answer the wrong question

7
Disadvantage of ITT

• Doctor doctor, how much will taking this tablet
  reduce my risk of heart disease?
• I dont know, but me prescribing it will reduce
  your disease risk by 10
• on average
• thats on average over whether you take it or not

8
Alternatives to ITT

• Per-protocol analysisexclude any data collected
  after a departure from randomised treatment
• Randomisation-based efficacy estimators (RBEEs)

9
Simple example

• Experimental vs. Standard treatment
• Interested in mean outcome
• E arm some patients get E, some immediately
  switch to S
• all-or-nothing switches
• S arm no switches

10
Observed data
if rand to E
Define compliers as those who would get E if
randomised to E.
11
Model
12
Three contrasts
13
Is per-protocol analysis reasonable?

• Not reasonable to assume random non-compliance
• if it were, then we wouldnt do RCTs at all!
• Less unreasonable
• if we condition on covariates
• in double-blind trial

14
Randomisation-based efficacy estimators (RBEEs)
15
Randomisation-based efficacy estimators

• Estimate causal effect of treatment in trials
  with treatment switches
• Based entirely on comparisons of groups as
  randomised
• No assumptions of comparability between groups as
  treated

16
A simple RBEE (binary outcome)
nNE
if nE nS
17
General techniques Potential outcomes framework

• Model outcome given the pair of potential
  treatments
• actual treatment if randomised to control
• actual treatment if randomised to treatment
• Maximum likelihood or Bayesian estimation
• Mostly for all-or-nothing switches
• Causal model relating actual outcome to potential
  outcome if untreated
• fit e.g. by G-estimation
• works for complex switch patterns

18
Likelihood techniques

• Complier, non-complier with prob. wC, wN
• Density
• fCE(yq) for compliers in E
• fCS(yq) for compliers in S
• fN(yq) for noncompliers
• Likelihood (e.g. Imbens and Rubin, 1997)
• wC fCE (yq) for compliers in E
• wN fN(yq) for noncompliers in E
• wC fCS (yq) wN fN(yq) for all in S

19
G-estimation techniques

• Define a causal model relating
• outcome Y(z) if treatment z
• to potential outcome Y(0) if no treatment
• e.g. Y(z) Y(0) bz
• Given b, deduce Y(0) for each individual
• Estimate b using Y(0) - R
• e.g. Robins, 1994
• Basic analysis consistent with ITT P-value
• but covariates can increase power

20
Four examples of situations where RBEEs are useful
21
Example 1.Patient information
22
MASS trial

• Abdominal aortic aneurysms are often fatal if
  they rupture
• May be repaired if detected before rupture
• Reliably detected by ultrasound screening
• MASS trial 67,800 men were randomised to
  invitation to screening or control
• ITT analysis invitation to screening reduced
  aneurysm-related death (Lancet, 2002)
• hazard ratio 0.58 (95 CI, 0.42 to 0.78), P0.002

23
Non-attendance in MASS

• 20 of invited group didnt attend for screening
• ITT measures the average benefit of screening in
  invitees
• What is the benefit of screening in attenders?

24
MASS model

• Method of Loeys Goetghebeur (2003)
• Stata implementation by Kim White (2004)
• Survivor functions
• SCE(t) in attenders randomised to E,
• SNE(t) in non- attenders randomised to E
• SS(t) in all randomised to S
• P(non-attender) a
• Hazard ratio in attenders y
• Assume exclusion restriction
• So (1-a)SCE(t)1/y a SNE(t) SS(t)

25
MASS estimation

• Kaplan-Meier estimates of SCE(t), SNE(t), SS(t)
• Estimate P(non- attender) by a
• Find y where (1-a)SCE(t)1/y a SNE(t) balances
  SS(t)
• a form of G-estimation

26
MASS results

• Effect of screening in attenders (CACE) HR
  0.47 (95 CI 0.36 to 0.70), P0.002
• Effect of invitation to screening (ITT) HR
  0.58 (95 CI, 0.42 to 0.78), P0.002

27
Comment

• In the MASS trial, the P-value is unchanged
  because causal and ITT null hypotheses are the
  same (under the exclusion restriction)
• causal NH screening has no effect in those
  screened
• ITT NH screening has no effect overall
• The CACE is simply an interpretation of the ITT
  result
• In the next 2 examples, the causal and ITT null
  hypotheses differ

28
Example 2.Treatment-time interactions
29
Treatment-time interactions

• Switches accumulating over time affect the
  treatment-time interaction
• Constant causal effect ? declining ITT effect
• Transient causal effect may ? reversed ITT effect

30
Simple case

• Causal effect of E is
• b1 at time 1 after receiving E
• b2 at time 2 after receiving E
• Switches occur just after time 1
• a fraction a of the S arm get E
• but all the E arm get E at the start
• Then the ITT difference at time 2 is b2 - a b1
• e.g. b110, b2 2, a 0.3 ? ITT2 -1
• ITT can get wrong sign with treatment-time
  interaction

31
Estimation

• Solve equations like ITT2 b2 - a b1
• This is a special case of a structural nested
  mean model (Robins, 1994)
• More generally
• add covariates that predict switch or outcome
• model covariance
• allow for missing data

32
TARGET trial

• Treatment of glue ear in children
• 376 children randomised to 3 arms
• insertion of ventilation tubes (VT) (grommets)
• insertion of ventilation tubes adenoidectomy
  (AD)
• medical management (MM)
• Outcome hearing loss measured at 5 visits
• Approx 50 of MM arm eventually got VT
• Some of VT arm had re-insertion

33
Sorry!

• The talk as presented contained numerical results
  from TARGET
• These cant yet be presented on the WWW
• Broadly, RBEEs demonstrated a longer-lasting
  benefit of VTs

34
Example 3.Treatment-covariate interactions
35
Treatment-covariate interactions

• Suppose the switch rate depends on covariates
• Then ITT effect will be more attenuated in
  subgroups with more switches
• Common causal effect ? different ITT effects

36
Example

• Suppose in subgroup j
• causal effect of E is bj
• fraction aj of S arm get E
• then ITT effect of E is bj(1-aj)
• e.g. by baseline severity
• less severe b18, a10.1 ? ITT1 7.2
• more severe b2 10, a20.5 ? ITT2 5
• Treatment works better in more severe subgroup,
  but appears to work worse

37
TARGET again

• Covariate baseline hearing loss (dichotomised
  at median)
• Clinically plausible that VT works better when
  hearing loss is greater not confirmed by ITT
  analyses
• Modify RBEE to incorporate covariate and
  interaction

38
Sorry again!

• Again, the numerical results from TARGET cant
  yet be presented on the WWW
• Broadly, RBEEs demonstrated a pattern of
  interactions that was more in line with clinical
  plausibility

39
Example 4.Cost-effectiveness analysis
40
Cost-effectiveness analysis

• In a RCT, explore difference in costs as well as
  difference in effects
• A pragmatic analysis cost-effectiveness among
  compliers is not of interest
• But compliance in practice is probably less than
  in an RCT
• Need to allow for this in cost-effectiveness
  analysis

41
Two cases

• If its reasonable to assume
• effect of intervention ? compliance
• cost of intervention ? compliance
• then cost-effectiveness is independent of
  compliance
• But cost of intervention is more likely to
  comprise
• a constant part (e.g. cost of prescribing drug)
• a part ? compliance (e.g. cost of repeat
  prescriptions)
• Here, cost-effectiveness does depend on
  compliance.

42
Obstacles to wider use of RBEEs
43
What happens in practice?

• Most trials report an ITT analysis
• or at least claim to do so
• Many trials additionally report a per-protocol
  analysis
• RBEEs are rarely used
• Why?

44
Changes to non-trial treatment

• E.g. in a trial of A vs. placebo, some patients
  get B
• Awkward to adjust for receipt of B, because we
  need to know/estimate effect of B
• estimate it observationally within trial?
• sensitivity analysis?
• use knowledge from other trials?
• Similar problem arises with changes to no
  treatment in an equivalence trial

45
How well developed are RBEE methods?

• Plenty of examples for all-or-nothing compliance
• pitfalls are known
• Few examples for more complex compliance
• Methods are complex not very general
• Not much software available
• Stata strbee, snmm, stcomply (BSU web site)
• Can implement in Mplus, gllamm, WinBUGS

46
How useable are RBEE methods?

• Methods are unfamiliar to most statisticians
• often wrongly seen as a variant of per-protocol
  analysis
• Need ways to explain them to non-statisticians
• may be usefully described as interpreting ITT
  analysis

47
Conclusions

• ITT analysis is still essential
• Per-protocol analysis should be avoided
• Statisticians should learn to recognise
  situations in which RBEEs can give clinically
  useful information
• those in which ITT is sufficient
• At present this is largely restricted to
  situations with switches

48

• Thanks to
• Mark Haggard and the TARGET investigators
• Lois Kim

49
References

• Imbens GW, Rubin DB. Bayesian inference for
  causal effects in randomized experiments with
  noncompliance. Annals of Statistics 1997 25
  305327.
• Robins JM. Correcting for non-compliance in
  randomized trials using structural nested mean
  models. Communications in Statistics Theory
  and Methods 1994 23 23792412.
• The Multicentre Aneurysm Screening Study Group.
  The multicentre aneurysm screening study (MASS)
  into the effect of abdominal aortic aneurysm
  screening on mortality in men a randomised
  controlled trial. Lancet 2002 360 15311539.
• Loeys T, Goetghebeur E. A causal proportional
  hazards estimator for the effect of treatment
  actually received in a randomized trial with
  all-or-nothing compliance. Biometrics 2003 59
  100105.
• Kim LG, White IR. Compliance-adjusted treatment
  effects in survival data. STATA journal 2004 4
  257264.
• White IR. Uses and limitations of
  randomisation-based efficacy estimators.
  Statistical Methods in Medical Research, to
  appear.
• ian.white_at_mrc-bsu.cam.ac.uk
• http//www.mrc-bsu.cam.ac.uk/pub/software/stata/