On the Road to Genomic Predictive Medicine An Interim Analysis

1
On the Road to Genomic Predictive Medicine An
Interim Analysis

• Richard Simon
• Chief, Biometric Research Branch
• National Cancer Institute

2
How I got involved in genomics

• In the late 1990s genomic data was for me the
  most exciting scientific data of our generation
• Analysis of that data shouldnt be left to
  amateurs
• We had a great cadre of statisticians involved in
  clinical trials and we know how to do reliable
  clinical trials, but the drugs are often
  disappointing
• Statisticians should be involved in basic
  research, pre-clinical target discovery and
  policy

3

• Biomedical leaders were looking to computer
  scientists and physicists for help, not to
  statisticians
• Statisticians were viewed as useful for testing
  hypotheses and computing p values, not for
  discovery

4

• Many statisticians tend to see themselves as
  methods developers not as scientists focused on
  subject matter area

5
(No Transcript)
6
(No Transcript)
7
Imatinib chronology

• 1960 - Philadelphia chromosome described (P
  Nowell)
• 1973 Ph characterized as translocation of BCR
  on chromosome 9 with ABL on chromosome 22 (J
  Rowley)
• 1986 BCR-ABL fusion gene characterized as
  constituatively activated kinase (D Baltimore)

8
Imatinib chronology

• 1988 -1995 CIBA-GEIGY develops kinase inhibitors
  (A Matter, N Lydon, J Zimmermann, E Buchdunger)
• 1996 B Drucker (Dana Farber -gt Oregon) screens in
  ex-vivo tumors and normal lymphocytes against
  compounds provided by Novartis and convinces
  company to sponsor clinical trials in CML in
  spite of only 5000 cases/yr in US

9

• Success depended on collaboration between
  industry and academia
• Delayed development resulted from reluctance of
  field to accept hypothesis that kinases can be
  selectively inhibited or that inhibiting a single
  gene could be very effective
• Industry involvement dependent on vision of a
  small leadership group in one company
• Clinical translation dependent on vision of one
  oncologist

10

• Success depends on serendiptiy
• Academic medicine (NIH) is a bottom-up system not
  optimized for risk taking or exploiting
  scientific leads for translating basic research
  to clinical products or for mounting large
  cooperative programs for overcoming bottlenecks
  in translation
• Academic medicine is very dependent on industry
  but industry has its own constraints

11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
Predictive Medicine

• Germline genetics
• GWAS
• 23andMe
• Tumor genomics
• Tumor Cell Genome Atlas

22
Ioannidis et al.JNCI 102846(2010)

• 56 GWAS
• 92 statistically significant associations between
  cancer phenotype and genetic variant
• Median OR 1.22
• IQR OR 1.15 1.36

23
(No Transcript)
24
Relative Risk of Disease PrDisease PrDisease test
1.3 0.01 0.013
1.3 0.10 0.13
25

• Cancers of a given histologic diagnosis are
  genomically heterogeneous
• Cancers are mostly caused by somatic mutations
  not genetic polymorphisms
• Most of the information about the disease is in
  the tumor genome, not the germ-line genome

26
(No Transcript)
27
(No Transcript)
28
Biomarkers for Early Detection

• Because of the long time between first mutation
  and clinical diagnosis of human solid tumors,
  there would seem to be great opportunity for
  early detection

29

• Phase II trials of early detection have used
  samples from patients at diagnosis
• Effective detection must have long lead time and
  high specificity for tumors which will evolve to
  be life threatening

30
(No Transcript)
31
Biomarkers for Informing Treatment Selection

• Prognostic biomarkers
• Measured before treatment to indicate long-term
  outcome for patients untreated or receiving
  standard treatment
• To identify which patients have excellent
  prognosis on conservative treatment
• Predictive biomarkers
• Measured before treatment to identify who is
  likely or unlikely to benefit from a particular
  treatment

32
Prognostic Markers

• Vast literature on prognostic markers
• Very few used in practice
• Most studies motivated by desire to learn about
  disease biology
• Broad selection of cases
• Little focus on intended use
• Little focus on analytical validation of assay

33
Validation of Biomarkers

• Analytical validity
• Measures what it supposed to
• Reproducible
• Clinical validity
• Correlates with something clinically
• Clinical utility
• Is actionable
• Measuring marker leads to action that benefits
  patient
• Requires clarity on intended use

34
If you dont know where you are going, you might
not get thereYogi Berra
35
Prognostic Markers

• OncotypeDx Which patients with node negative ER
  positive breast cancer who are receiving
  tamoxifin will have such good prognosis that they
  do not need cytotoxic chemotherapy?
• Analysis focused on whether marker identifies
  such a subset, not on statistical significance

36
B-14 ResultsRelapse-Free Survival
Paik et al, SABCS 2003
37
(No Transcript)
38
Major problems with prognostic studies of gene
expression signatures

• Inadequate focus on intended use
• Cases selected based on availability of specimens
  rather than for relevance to intended use
• Heterogeneous sample of patients with mixed
  stages and treatments. Attempt to disentangle
  effects using regression modeling
• Overemphasis on statistical significance and
  hazard ratios.
• Over-fitting data

39
For pgtn problems

• Fit of a model to the same data used to develop
  it is no evidence of prediction accuracy for
  independent data

40
(No Transcript)
41
Validation of Prognostic Model

• Completely independent validation dataset
• Splitting dataset into training and testing sets
• Cross-validation

42
(No Transcript)
43

• Partition data set D into K equal parts
  D1,D2,...,DK
• First training set T1D-D1
• Develop completely specified prognostic model M1
  using only data T1
• Compute prognostic score for cases in D1
• Develop model M2 using only T2 and then score
  cases in D2

44

• Repeat for ... TK -gt MK -gt DK
• Group patients into risk groups (e.g. 2 or more)
  based on their cross-validated scores
• Calculate Kaplan-Meier survival curve for each
  risk-group

45
(No Transcript)
46
Complete cross Validation

• Cross-validation simulates the process of
  separately developing a model on one set of data
  and predicting for a test set of data not used in
  developing the model
• All aspects of the model development process must
  be repeated for each loop of the cross-validation
• Feature selection
• Tuning parameter optimization

47
Prediction on Simulated Null DataSimon et al. J
Nat Cancer Inst 9514, 2003

• Generation of Gene Expression Profiles
• 20 specimens (Pi is the expression profile for
  specimen i)
• Log-ratio measurements on 6000 genes
• Pi MVN(0, I6000)
• Can we distinguish between the first 10
  specimens (Class 1) and the last 10 (Class 2)?
• Prediction Method
• Compound covariate predictor built from the
  log-ratios of the 10 most differentially
  expressed genes.

48
(No Transcript)
49
Cross Validation

• The cross-validated estimate of misclassification
  error is an estimate of the prediction error for
  the model fit applying the specified algorithm to
  full dataset

50

• Statistical significance of the difference in
  survival among risk groups is usually not the
  point
• But to evaluate significance, the log-rank test
  cannot be used for cross-validated Kaplan-Meier
  curves because the survival times are not
  independent

51

• Statistical significance can be properly
  evaluated by approximating the null distribution
  of the cross-validated log-rank statistic
• Permute the survival times and repeat the entire
  cross-validation procedure to generate new
  cross-validated K-M curves for low risk and high
  risk groups
• Compute log-rank statistic for the curves
• Repeat for many sets of permutations

52
(No Transcript)
53
Does measuring new biomarkers have medical
utility over using a standard staging system?

• It is usually more useful to compute K-M curves
  for the high and low risk groups of the new model
  within each level of risk for the model based on
  standard staging variables.
• That representation will enable clinicians to
  determine whether the refinement in risk within
  groups of patients that are more homogeneous with
  regard to therapeutic decision-making is of
  practical significance

54
Predictive Biomarkers

• Single gene or protein measurement
• ER protein expression
• HER2 amplification
• EGFR mutation
• KRAS mutation
• V600E mutation
• ALK translocation
• Index or classifier that summarizes expression
  levels of multiple genes

55
The standard approach to designing phase III
clinical trials is based on two assumptions

• Qualitative treatment by subset interactions are
  unlikely
• Costs of over-treatment are less than costs
  of under-treatment

56

• Cancers of a primary site often represent a
  heterogeneous group of diverse molecular diseases
  which vary fundamentally with regard to
• the oncogenic mutations that cause them
• their responsiveness to specific drugs
• Most new cancer drugs are very expensive
• the aspirin paradigm on which current clinical
  trial dogma is based can be a roadblock to
  progress

57
Standard Clinical Trial Approaches

• Have led to widespread over-treatment of patients
  with drugs to which few benefit
• Are not scientifically well founded nor
  economically sustainable for future cancer
  therapeutics

58

• Neither current practices of subset analysis nor
  current practices of ignoring subset analysis are
  effective for evaluating treatments and informing
  physicians in heterogeneous diseases

59

• How can we develop new drugs in a manner more
  consistent with modern tumor biology and obtain
  reliable information about what regimens work for
  what kinds of patients?

60
Develop Predictor of Response to New Drug
Using phase II data, develop predictor of
response to new drug
Patient Predicted Responsive
Patient Predicted Non-Responsive
Off Study
New Drug
Control
Targeted (Enrichment) Design
61
Evaluating the Efficiency of Targeted Design

• Simon R and Maitnourim A. Evaluating the
  efficiency of targeted designs for randomized
  clinical trials. Clinical Cancer Research
  106759-63, 2004 Correction and supplement
  123229, 2006
• Maitnourim A and Simon R. On the efficiency of
  targeted clinical trials. Statistics in Medicine
  24329-339, 2005.
• http//brb.nci.nih.gov

62

• Relative efficiency of targeted design depends on
  
• proportion of patients test positive
• effectiveness of new drug (compared to control)
  for test negative patients
• When less than half of patients are test positive
  and the drug has little or no benefit for test
  negative patients, the targeted design requires
  dramatically fewer randomized patients than the
  standard design in which the marker is not used

63

• Companion diagnostic test with intended use of
  identifying patients who have disease subtype for
  which the drug is proven effective

64
Stratification Design for New Drug Development
with Companion Diagnostic
65
Fallback Analysis Plan

• Compare the new drug to the control overall for
  all patients ignoring the classifier.
• If poverall 0.01 claim effectiveness for the
  eligible population as a whole
• Otherwise perform a single subset analysis
  evaluating the new drug in the classifier
  patients
• If psubset 0.04 claim effectiveness for the
  classifier patients.
• The test in the subset is not dependent on
  finding an overall significant finding or a
  significant interaction
• The trial is sized for powering both tests
• The validity of the analysis does not depend on
  stratifying the randomization by the test

66
(No Transcript)
67
(No Transcript)
68
Strong confidence in test Small r2 and large
r1 Weak confidence in test Small r2 and small
r1 p00 selected to control type I error rates
69
(No Transcript)
70
(No Transcript)
71
(No Transcript)
72
The Objectives of a Phase III Clinical Trial

• Test the strong null hypothesis that the new
  treatment E is uniformly ineffective relative to
  a control C while preserving the type I error of
  the study
• If the null hypothesis is rejected, develop an
  internally validated labeling indication for
  informing physicians in their decisions about
  which patients they treat with the drug.
• Not a hypothesis testing problem

73
(No Transcript)
74
The keys to developing effective drugs

• The target of the drug must be central to the
  progression of the disease
• Drug should be selective for the target so that
  it can be administered at a concentration that
  totally shuts down the de-regulated pathway
• Need a test that identifies the patients who have
  disease driven by de-regulation of the target

75
Tumors can contain large numbers of genetic
alterations

• Passenger mutations
• Occur at rate of non-synonymous mutations
• Driver mutations
• Occur more frequently than non-synonymous
  mutations and presumably have a functional role
  in oncogenesis and pathogenesis of the tumor
• Determined from sequencing of many tumors of a
  histological type
• Founder Mutations
• Originating mutation

76
(No Transcript)
77
Extend Previous Methods to Allow

• Background mutation rate to vary among tumors
• Background mutation rate to depend on sequence
  context of mutation

78
(No Transcript)
79
(No Transcript)
80
Founder mutations

• Mathematical modeling studies indicate that 2-4
  rate-limiting events occurring at normal
  mammalian mutation rates can account for
  age-incidence statistics for many types of human
  solid tumors
• These rate limiting events may correspond to the
  founder mutations of a tumor
• They may be rate-limiting because they occur when
  the tumor is restricted to small anatomic
  compartments prior to the occurrence of genome
  destabilizing mutations
• They may permit the tumor to grow to a size in
  which acquisition of subsequent mutations is not
  rate-limiting
• Additional driver mutations occur that aid tumor
  invasion and metastatic dissemination

81
Founder mutations may be of special importance

• They exist in all sub-clones of the tumor and so
  all tumor cells may be susceptible to founder
  mutation targeted treatment
• Subsequent mutations develop in the context of
  the founder mutations and be viable only in that
  context rendering the tumors addicted to the
  early mutations

82
(No Transcript)
83
(No Transcript)
84
(No Transcript)
85
(No Transcript)
86
(No Transcript)
87
(No Transcript)
88
(No Transcript)
89
Closing

• Germ-line genomics has so far had a limited
  impact on individual risk prediction in oncology
  and in understanding the nature of oncogenesis
• Tumor genomics is revolutionizing our
  understanding of cancer and is providing
  important opportunities to identify key molecular
  targets and for improving therapeutic decision
  making

90
Closing

• Treatment of broad populations with regimens that
  do not benefit most patients is increasingly less
  necessary nor economically sustainable
• The established molecular heterogeneity of cancer
  requires the use new approaches to clinical trial
  design and analysis
• Developments in high dimensional assays and NGS
  have stimulated many areas of biostatistics and
  placed greater emphasis on discovery and
  prediction

91
To Meet the Challenges and Opportunities
Available to Impact on Human Disease
Biostatistics Should

• Continue to broaden identity beyond probabilistic
  inference and methods development
• Reward statistical scientists, not just
  statistical mathematicians
• Embrace information technology
• Include the informatics end of bioinformatics
• Transcend hypothesis testing to include
  discovery, prediction and decision making

92
Joi Ito Director, MIT Media Lab

• In the old days, the world didnt change very
  much, so once you became a plumber, you didnt
  really need to learn that much more about
  plumbing. Today you have to keep learning and
  learning is somewhat of a childlike behavior. We
  want the Media Lab to me more like kindergarten
  and less like a lumber mill.

93

• Prediction is difficult particularly the
  future.
• Neils Bohr

94
Acknowledgements

• Boris Freidlin
• Wenyu Jiang
• Stella Karuri
• Aboubakar Maitournam
• Michael Radmacher
• Jyothi Subramanian
• Ahrim Youn
• Yingdong Zhao