An Analytic Road Map for Incomplete Longitudinal Clinical Trial Data

1
An Analytic Road Map for Incomplete Longitudinal
Clinical Trial Data

• Craig Mallinckrodt
• Graybill Conference
• June 12, 2008Fort Collins, CO

2
Acknowledgements

• PhRMA Expert Team on Missing Data Peter Lane
  GSK Craig Mallinckrodt Lilly James Mancuso
  Pfizer Yahong Peng Merck Dan Schnell
  PG
• Geert Molenberghs
• Ray Carroll
• Many Lilly colleagues

3
Outline

• Why do we care
• What do we know
• Theory
• Application
• What we should do

4
Medical Needs

• Every hour we expect

195 deaths due to cancer 1950 new diagnoses of
anxiety disorders 15 new diagnoses of
schizophrenia 30 osteoporosis related hip
fractures 1500 surgeries requiring pain
treatment 70 deaths due to cardiovascular
disease Alan Breier Nov 2006
5
Need for More Effective Medicines

• Therapeutic Area Efficacy
  rate()
• Alzheimers 30
• Analgesics (Cox-2) 80
• Asthma 60
• Cardiac Arrhythmias 60
• Depression (SSRI) 62
• Diabetes 57
• HCV 47
• Incontinence 40
• Migraine (acute) 52
• Migraine (prophylaxis) 50
• Oncology 25
• Osteoporosis 48
• Rheumatoid arthritis 50
• Schizophrenia 60

There is an efficacy gap in terms of customer
expectations and the drugs we prescribe
Trends in Molecular Medicine 7(5)201-204, 2001
6
RD Productivity Decreasing
Source PhRMA, FDA, Lehman Brothers Dr.
Robert Ruffolo
7
Outline

• Why do we care
• What do we know
• Theory
• Application
• What we should do

8
Starting Point

• No universally best method for analyzing
  longitudinal data
• Analysis must be tailored to the specific
  situation at hand
• Consider the hypothesis to be tested, desired
  attributes of the analysis, and the
  characteristics of the data

9
Missing Data Mechanisms

• MCAR - missing completely at random
• Conditional on the independent variables in the
  model, neither observed or unobserved outcomes of
  the dependent variable explain dropout
• MAR - missing at random
• Conditional on the independent variables in the
  model, observed outcomes of the dependent
  variable explain dropout, but unobserved outcomes
  do not

10
Missing Data Mechanisms

• MNAR - missing not at random
• Conditional on the independent variables in the
  model and the observed outcomes of the dependent
  variable, the unobserved outcomes of the
  dependent variable explain dropout

11
Consequences

• Missing data mechanism is a characteristic of
  the data AND the model
• Differential dropout by treatment indicates
  covariate dependence, not mechanism
• Mechanism can vary from one outcome to another
  in the same dataset

12
Missing Data in Clinical Trials

• Efficacy data in clinical trials are seldom MCAR
  because the observed outcomes typically influence
  dropout (DC for lack of efficacy)
• Trials are designed to observe all the relevant
  information, which minimizes MNAR data
• Hence in the highly controlled scenario of
  clinical trials missing data may be mostly MAR
• MNAR can never be ruled out

13
Implications

• All analyses rely on missing data assumptions
• Any options in the trial design to minimize
  dropout should be strongly considered

14
Assumptions

• ANOVA with BOCF / LOCF assumes
• MCAR constant profile
• MAR always more plausible than MCAR
• MAR methods will be valid in every case where
  BOCF/ LOCF is valid
• BOCF / LOCF will not be valid in every
  scenario where MAR methods are valid

15
Research Showing MAR Is Useful And / Or Better
Than LOCF

• Arch. Gen. Psych. 50 739-750.
• Arch. Gen. Psych. 61 310-317.
• Biol. Psychiatry. 53 754-760.
• Biol. Psychiatry. 59 1001-1005.
• Biometrics. 52 1324-1333.
• Biometrics. 57 43-50.
• Biostatistics. 5445-464.
• BMC Psychiatry. 4 26-31.
• Clinical Trials. 1 477489.
• Computational Statistics and Data Analysis. 37
  93-113.
• Drug Information J. 35 1215-1225.
• J. Biopharm. Stat. 8 545-563.
• J. BioPharm. Stat. 11 9-21.

16
Research Showing MAR Is Useful And / Or Better
Than LOCF

• J. Biopharm. Stat. 12 207-212.
• J. Biopharm. Stat. 13179-190.
• J. Biopharm. Stat. 16 365-384.
• Neuropsychopharmacol. 6 39-48.
• Obesity Reviews. 4175-184.
• Pharmaceutical Statistics. 3161-170.
• Pharmaceutical Statistics. 3171-186.
• Pharmaceutical Statistics. 4267-285.
• Pharmaceutical Statistics (2007 early view) DOI
  10.1002/pst.267
• Statist. Med. 11 2043-2061.
• Statist. Med. 14 1913-1925.
• Statist. Med. 22 2429-2441.

17
Why Is LOCF Still Popular

• LOCF perceived to be conservative
• Concern over how MAR methods perform under MNAR
• More explicit modeling choices needed in MAR
  methods
• LOCF thought to measure something more valuable

18
Conservatism Of LOCF

• Bias in LOCF has been shown analytically and
  empirically to be influenced by many factors
• Direction and magnitude of bias highly situation
  dependent and difficult to anticipate
• Summary of recent NDA showed LOCF yielded lower p
  value than MMRM in 34 of analyses

Biostatistics. 5445-464. BMC Psychiatry. 4
26-31.
19
Performance Of MAR With MNAR Data

• Studies showing MAR methods provide better
  control of Type I and Type II error than LOCF
• Arch. Gen. Psych. 61 310-317.
• Clinical Trials. 1 477489.
• Drug Information J. 35 1215-1225.
• J. BioPharm. Stat. 11 9-21.
• J. Biopharm. Stat. 12 207-212.
• Pharmaceutical Statistics (2007 early view) DOI
  10.1002/pst.267
• JSM Proceedings. 2006. pp. 668-676. 2006.

20
More Explicit Modeling Choices Needed

• MMRM 6 lines of code, LOCF 5 lines of code
• Convergence and choice of correlation not
  difficult in MMRM

Clinical Trials. 1 477489.
21
LOCF Thought To Measure Something More Valuable

• LOCF is effectiveness, MAR is efficacy
• LOCF is what is actually observed
• MAR is what is estimated to happen if patients
  stayed on study
• Non longitudinal interpretation of LOCF
• LO, LAV
• Dropout is an outcome

22
Non-longitudinal Interpretation Of LOCF

• An LOCF result can be interpreted as an index of
  rate of change times duration on study drug - a
  composite of efficacy, safety, tolerability
• An index with unknown weightings
• The same estimate of mean change via LOCF can
  imply different clinical profiles
• The LOCF penalty is not necessarily proportional
  to the risk
• Result can be manipulated by design

23
Completion Rates in Depression Trials
Drug
Placebo
24
Placebo Dropout Rates Influenced by Design In a
Recent MDD NDA
Trial DC-AE
Dropout 1 4.3 34.3 2
6.7 41.3 3 3.3 31 4
9.0 42 5 3.2 19
6 1.0 9 7 2.5 29.5
8 4.3 35.3
Trials 5 and 6 had titration dosing and extension
phases
Lillytrials.com
25
Outline

• Why do we care
• What do we know
• Theory
• Application
• What we should do

26
Modeling Philosophies

• Restrictive modeling
• Simple models with few independent variables
• Often include only the design factors of the
  experiment

Psychological Methods, 6, 330-351.
27
Modeling Philosophies

• Inclusive modeling
• Auxiliary variables included to improve
  performance of the missing data procedure
  expand the scope of MAR
• Baseline covariates
• Time varying post-baseline covariates Must
  be careful to not dilute treatment effect. Can
  be dangerous to include time varying
  postbaseline covariates in analysis model, may
  be better to use via imputation (or propensity
  scoring or weighted analyses)

Psychological Methods, 6, 330-351.
28
Rationale For Inclusive Modeling

• MAR conditional on the dependent and independent
  variables in the analysis, unobserved values of
  the dependent variable are independent of dropout
• Hence adding more variables that explain dropout
  can make missingness MAR that would otherwise be
  MNAR

29
Analytic Road Map

• MAR with restrictive modeling as primary
• Use MAR with inclusive modeling and MNAR methods
  as sensitivity analyses
• Use local influence to investigate impact
  ofinfluential patients

Pharmaceutical Statistics. 4 267285.J.
Biopharm. Stat. 16 365-384.
30
Why Not MNAR As Primary

• Can do better than MAR only via assumptions
• Assumptions untestable
• Sensitivity to violations of assumptions and
  model misspecification more severe in MNAR
• MNAR methods lack some desired attributes of a
  primary analysis in a confirmatory trial
• No standard software
• Complex

31
Implementing The Road Map Example From A
Depression Trial

• 259 patients, randomized 11 ratio to drug and
  placebo
• Response Change of HAMD17 score from baseline
• 6 post-baseline visits (Weeks 1,2,3,5,7,9)
• Primary objective test the difference of mean
  change in HAMD17 total score between drug and
  placebo at the endpoint
• Primary analysis LB-MEM

32
Patient Disposition

• Drug Placebo
• Protocol complete 60.9 64.7
• Adverse event 12.5 4.3
• Lack of efficacy 5.5 13.7
• Differential rates, timing, and/or reasons for
  dropout do not necessarily distinguish between
  MCAR, MAR, MNAR

33
Primary Analysis LB-MEM
proc mixed class subject treatment time
site model Y baseline treatment time
site treatmenttime repeated time / sub
subject type un lsmeans treatmenttime /
cl diff run This is a full multivariate
model, with unstructured modeling of time and
correlation. More parsimonious approaches may be
useful in other scenarios Treatment contrast
2.17, p .024
34
Inclusive Modeling in MI Including Auxiliary AE
Data

• Imputation Models
• Yih µ ?1 Yi1 ?h-1 Yi(h-1) ?ih
• Yih µ ?1 Yi1 ?h-1 Yi(h-1) ?1 AEi1
  ?h-1 AEi(h-1) ?ih
• Yih µ ?1 Yi1 ?h-1 Yi(h-1) ?1 AEi1
  ?h-1 AEi(h-1)
• ?11 (Yi1 AEi1 ) ?i(h-1) (Yi(h-1)
  AEi(h-1) ) ?ih
• Analysis Model
• MMRM as previously described

35
Result

• MI results were not sensitive to the different
  imputation models Endpoint contrastMMRM 2.2
  MI YAE 2.3MI YAEYAE 2.1
• Including AE data might be important in other
  scenarios. Many ways to define AE

36
MNAR Modeling

• Implement a selection model
• Had to simplify model modeled time as linear
  quadratic, and used ar(1) correlation
• Compare results from assuming MAR, MNAR
• Also obtain local influence to assess impact of
  influential patients on treatment contrasts and
  non-random dropout

37
Selection Model Results
MAR MNAR
Contrast (p-value) 2.20 (0.0179) 2.18 (0.0177)

Missingness Parameters Estimate SE Missingness Parameters Estimate SE Missingness Parameters Estimate SE Missingness Parameters Estimate SE Missingness Parameters Estimate SE
?0 -2.46 0.27
?1 0.11 0.05
?2 -0.08 0.06

38
Local Influence Influential Patients
39
Individual Profiles with Influential Patients
Highlighted
40
Investigating The Influential Patients

• The most influential patient was 30, a
  drug-treated patient that had the unusual profile
  of a big improvement but dropped out at week 1
• This patient was in his/her first MDD episode
  when s/he was enrolled
• This patient dropped out based on his/her own
  decision claiming that the MDD was caused by high
  carbon monoxide level in his/her house
• This patient was of dubious value for assessing
  the efficacy of the drug

41
Selection Model Influential Patients Removed
Removed Subjects ( 30, 191) ( 30, 191) (6, 30, 50, 154, 179, 191) (6, 30, 50, 154, 179, 191)
MAR MNAR MAR MNAR
Diff. at endpoint(p-value) 2.07 (0.0241) 2.07 (0.0237) 2.40 (0.0082) 2.40 (0.0083)
Missingness Parameters Missingness Parameters Missingness Parameters Missingness Parameters Missingness Parameters
?0 -2.22 (0.14) -2.44 (0.27) -2.23 (0.15) -2.47 (0.28)
?1 0.05 (0.02) 0.11 (0.05) -0.05 (0.02) 0.11 (0.06)
?2 -0.07 (0.06) -0.08 (0.06)
42
Implications

• Comforting that no subjects had a huge influence
  on results. Impact bigger if it were a smaller
  trial
• Similar to other depression trials we have
  investigated, results not influenced by MNAR data
• We can be confident in the primary result

43
Discussion

• MAR with restrictive modeling was a reasonable
  choice for the primary analysis
• MAR with inclusive modeling and MNAR was useful
  in assessing sensitivity
• Sensitivity analyses promote the appropriate
  level of confidence in the primary result and
  lead us to an alternative analysis in which we
  can have the greatest possible confidence

44
Opinions

• Inclusive modeling has been under utilized
• More research to understand dropout would be
  useful
• Did not discuss pros and cons of various ways to
  implement inclusive modeling. Use the one you
  know? Be careful to not dilute treatment
• The road map for analyses used in the example
  data is specific to that scenario

45
Conclusions

• No universally best method for analyzing
  longitudinal data
• Analysis must be tailored to the specific
  situation at hand
• Considering the missingness mechanism and the
  modeling philosophy provides the framework in
  which to choose an appropriate primary analysis
  and appropriate sensitivity analyses

46
Conclusion

• LOCF and BOCF are not acceptable choices for the
  primary analysis
• MAR is a reasonable choice for the primary
  analysis in the highly controlled situation of
  confirmatory clinical trials
• MNAR can never be ruled out
• Sensitivity analyses and efforts to understand
  and lower rates of dropout are essential