Alternative inference for adaptive designs

1
Alternative inference for adaptive designs

• Carl-Fredrik Burman, PhD
• AstraZeneca RD Sweden

2
Disposition

• Adaptive designs
• Broad definition and rationale
• Flexible designs
• Weighted test (Bauer Köhne 94)
• The fundamental idea
• Drawbacks
• The dual test

3
What are adaptive designs?

• An adaptive design uses accumulating data to
  decide on how to modify aspects of the study
• PhRMA working group definition
• Dragalin (DIJ 2006)

Typical application clinical trials. Many other
potential applications.
4
A very wide definition

• Adaptation may be based on
• Interim effect estimates
• Interim estimates of nuisance parameter (e.g.
  variance)
• using unblinded data or
• using blinded data
• Baseline covariates
• (External information)

5
A very wide definition

• Adaptation may be based on
• Interim effect estimates
• Interim estimates of nuisance parameter (e.g.
  variance)
• using unblinded data or
• using blinded data
• Baseline covariates
• (External information)
• Adaptations may include changes of e.g.

• Sample size
• Treatments
• Study population

• Response variable
• Treatment duration
• Null hypothesis

6
Adaptive designs good and bad

• Intuitively appealling
• The scientific process has always been adaptive.
  We learn from experience / experiments and design
  new experiments based on that.
• Why not look at incoming data during a trial and
  adjust the design?
• Adaptive designs could be more efficient and
  ethical
• E.g. stop treatment arms with adverse effect /
  lack of positive effect
• However, some proposed designs are inefficient
  and / or nonconvincing for a medical audience

7
Different statistical approaches

• Bayesian statistics
• Adaptive designs fit well into a Bayesian
  framework
• Prior distribution can be updated with more and
  more data
• No focus on type I error
• Classical frequentist statistics
• Need to prespecify sample space / stopping rules
• Can handle designs when adaptation rule is
  completely predetermined
• But cannot allow flexible redesigns which are not
  prespecified

8
Different statistical approaches

• Bayesian statistics
• Adaptive designs fit well into a Bayesian
  framework
• Prior distribution can be updated with more and
  more data
• No focus on type I error
• Classical frequentist statistics
• Need to prespecify sample space / stopping rules
• Can handle designs when adaptation rule is
  completely predetermined
• But cannot allow flexible redesigns which are not
  prespecified Yes, it can!

9
The Bauer-Köhne breakthrough

• Z1 is test statistic based on data from stage 1
• Assume that Z1 is N(0,1) under the null
  hypothesis
• Based on stage 1 data, we may choose any design
  for stage 2
• Choose any test variable Z2 based on stage 2 data
  so that Z2 is N(0,1) under the null hypothesis
  (given the design of the stage, which depends on
  Z1)
• Then the weighted statistic Zw Z1 ?w Z2
  ?(1-w) is N(0,1) under the null hypothesis
  provided that the weight w is choosen
  independently of the data

10
Full flexibility

• All you have to do is prespecify a weight for the
  first stage
• Then you can view the two stages as separate
  studies with different designs / sample sizes /
  primary variables, etc.
• May look at stage 1 and design stage 2 as you
  like
• The type I error is protected
• Note Bauer Köhne used slightly different
  weighting
• May combine p-values (e.g. via Z-score)
• Easy to generalise to gt 2 stages
• May be combined with early stopping (cf.
  group-sequential)

11
Criticism 1 Inefficient

• Weighted test violates sufficiency principle
• (Somewhat) less efficient
• but flexibility may be worth it?

Jennison Turnbull (2003) Tsiatis Mehta (2003)
12
Criticism 2 Not sound

• Weighted test gives different weight to
  exchangible observations
• Violates One patient one vote
• Violates inference principles
• May lead to absurd consequences
• Average of all observations may be negative,
  while the weighted test concludes that the mean
  is positive

Burman Sonesson (2006)
13
The dual test, preliminaries

• The most discussed flexible design is sample size
  reestimation based on unblinded interim data
• Assume Xs and Ys are i.i.d. N(m,s)
• Null hypothesis m0
• Let Z1 (X1Xn) / ?n,
• Z2 (Y1Ym) / ?m
• where m is determined after observing Z1
• Weighted test statistic Zw Z1 ?w Z2 ?(1-w) is
  generally not equal to plain statistic Z
  (X1Xn Y1Ym ) / ?(nm)

14
The dual test

• Weighted statistic Zw Z1 ?w Z2 ?(1-w)
• Protects type I error
• May lead to absurd conclusions
• Plain statistic Z (X1Xn Y1Ym ) / ?(nm)
• Unknown type I error
• Z based on simple average
• Cf. Bayes with noninformative prior
• Dual test is significant iff min(Z,Zw)gtF-1(1-a)
• Protects type I error
• Avoids problems of weighted test

Denne (2001)
15
The dual test ? eating the cake and keeping it

• Obviously,
• Dual test significant ? Weighted test
  significant
• How much power do we loose?
• For a large class of sample size reestimations,
  the dual test has no loss of power
• After stage 1, we can always see which sample
  sizes m for stage 2 would make ZwgtF-1(1-a) ?
  ZgtF-1(1-a).
• Note, however, although
• ?a ?m no power loss
• It is not true that
• ?m ?a no power loss

16
Conclusions

• Adaptive designs
• Are potentially useful
• But should be handled with care
• Flexible designs weighted test
• Loose some power
• Violates good statistical practice
• Dual test is worth further investigations