THE USE OF

1
THE USE OF HISTORICAL CONTROLS IN DEVICE
STUDIES Vic Hasselblad Duke Clinical Research
Institute
2
HISTORICAL CONTROLSTHE SETTING

• New trial will have a single experimental arm
• The endpoint is dichotomous
• Comparison will be to either
• summary rates from other studies
• another single arm using patient level analyses

3
HISTORICAL CONTROLSNO PATIENT LEVEL DATA

• Each historical arm has to be treated as a
  sample
• Results are usually calculated from a random
  effects model
• The distribution for the next sample is
  estimated
• This requires that the between study variance
  be estimated specifically

4
AN EXAMPLE FROM DISTAL PROTECTION DEVICES

• A new distal protection device proposed using
  data from existing distal protection devices as
  historical controls
• The endpoint was major adverse cardiac events
  (MACE)
• The results from these three arms appeared to
  be very consistent
• The estimation proved difficult (as we shall
  see)

5
DISTAL PROTECTION DEVICES
6
DISTAL PROTECTION DEVICES
FilterWire (FIRE Trial)
GuardWire (FIRE Trial)
GuardWire (SAFER Trial)
0.0
0.1
0.2
0.3
0.4
MACE Rate at 30 Days
7
CONSTRUCTING AHIERARCHICAL BAYESRANDOM EFFECTS
MODEL

• The prior for the mean rate was a
  non- informative (Jeffries) prior
• The prior for the study-to-study variation (t2)
  was assumed to be non- informative (1/t2)
• The expected distribution of the log-odds of
  the event rate was assumed to be normal

8
POSTERIOR FOR VARIANCE (t2)
9
POSTERIOR FOR MEAN RATE
10
CONCLUSIONS FORHISTORICAL CONTROLSNO PATIENT
LEVEL DATA
In order to use the results from a small number
of arms, one has to assume that the variation
between arms is quite small. In other words,
one has to add subjective information to the
prior.
11
HISTORICAL CONTROLSWITH PATIENT LEVEL DATA

• Propensity scores are used to correct for the
  fact that the two populations are not
  guaranteed to be similar
• Patients are stratified by their propensity to
  get a particular treatment
• Patients within a given propensity score are
  compared and the results are pooled across
  propensity categories

12
AN EXAMPLE WITH STENTS

• The object was to compare two different stent
  methodologies, one of which was a historical
  one
• The safety endpoint was MACE at 30 days
• The comparison was based on non-inferiority
• Propensity scores were used to make the
  comparison two different models were used as a
  sensitivity analysis

13
FIRST PROPENSITY SCORE
Used vessel diameter, lesion length, and presence
of diabetes as predictors.
SECOND PROPENSITY SCORE
Used vessel diameter, lesion length, presence of
diabetes plus several others factors including
smoking and EF as predictors.
14
AN EXAMPLE WITH STENTS
Delta
First propensity score
Second propensity score
15
STRATIFIED PROPENSITY SCORES CAN HAVE DIFFICULTIES
Percent Bias in the Estimated Treatment
Effect Based on a Stratified Propensity
Score (from Lunceford and Davidian, 2004)
16
STRATIFIED PROPENSITY SCORES CAN HAVE DIFFICULTIES
Ratio of Means Squared Errors Stratified
Propensity Score Versus Doubly Robust
Estimator (from Lunceford and Davidian, 2004)
17
HISTORICAL CONTROLSWITH PATIENT LEVEL DATA

• Even with the use of propensity scores, the
  results of a historical control analysis may not
  be definitive
• The use of stratified propensity scores is not
  always the solution
• In certain situations doubly robust estimators
  are better as long as they have the correct model
  (for propensity or risk)
• If the models are wrong, all bets are off

18
SUMMARY

• Historical control analyses are fraught with
  difficulties
• In many cases you dont know if problems exist
  until after the data have been collected