Challenges and Opportunities for Statisticians in The Drug Development Process

1
Critical Path An Overview

• Challenges and Opportunities for Statisticians in
  The Drug Development Process
• Charles Anello, Sc.D.
• Deputy Director
• OB, OPaSS, CDER, FDA

2
Disclaimer

• The views expressed in this talk are those of the
  author and do not necessarily represent those of
  the Food and Drug Administration

3
Talk Outline

• Drug Development Process
• Definition of Critical Path
• Three Dimensions of Critical Path
• Statistical aspects of Critical Path
• Strategic Plan or Way forward
• Summary

4
Ten Year Investment Trends
Doubling over 5 years of NIH funding Pharmaceutic
al RD investment increasing at the same
rate Major investments in biotechnology
5
(No Transcript)
6
Ten Year Trend in Product Submissions

• Decline in original BLAs Submissions
• Decline in NMEs

7
(No Transcript)
8
Development Costs are Escalating

• Costs of bringing a successful drug to market
  estimated between 0.8 1.7B
• Higher failure rate of candidates in clinical
  development

9
Critical Path Initiative

• Federal Register April 22,2004
• Innovation Challenge and Opportunity on the
  Critical Path to New Medicinal Products
• Request for comments ( hurdles , priorities and
  possible solutions, FDA role etc.)
• Interested parties were given unit a chance to
  comment

10
The Critical Path for Medical Product Development
11
Critical Path

• Defined as those aspects of the drug
  developmental process that can impact the safety
  , efficacy and quality of medial products

12
Three Dimensions of the Critical Path

• Assessment of Safety how to predict if a
  potential product will be harmful?
• Proof of Efficacy -- how to determine if a
  potential product will have medical benefit?
• Industrialization how to manufacture a product
  at commercial scale with consistently high
  quality?

13
Working in Three Dimensions on the Critical Path
14
Office of Biostatistics Critical Path
Initiatives

• Conduct Research, Gain Consensus and Develop
  Guidance to Remove Obstacles to efficient Drug
  Development
• Improve the processes and Approaches to
  Quantitative Analysis of Safety Data
• Apply Modern Statistical approaches to Product
  Testing and Process Control

15
Critical Path Statistical Issues

• Missing Data
• Flexible and Adaptive designs
• Non-Inferiority
• Multiple endpoints
• Modeling and simulation
• Bayesian Methods
• Drug Quality Control Methodology

16
Clinical Trial Methodology

• Better use and analysis of safety data collected
  in clinical trials to facilitate risk estimation,
  risk management and risk communication
• Enhance product testing and characterization
  during the drug manufacturing process

17
Missing Data due to patient withdrawals and drop
out

• Patients leave a study for many reasons (lack of
  effect, side effects, removed by PI because
  patient stopped study medication, etc)
• Clinical Trials may have few (1) to many (say
  50) of drop outs.
• Since the goal is to analyze all the patients
  randomized missing data presents a real problem.
• Of special concern is informative censoring, how
  to identify it and how to adjust for it.

18
Missing Data due to patient withdrawals and drop
out

• For decades the Agency has relied on Last
  Observation Carried Forward (LOCF) and has been
  criticized for this.
• Numerous alternative imputation methods have been
  developed and proposed
• But consensus on the best approach has been hard
  to achieve

19
Missing Data due to patient withdrawals and drop
out

• This problem impacts both safety and efficacy and
  the ability to make sound benefit risk decisions
• FDA is in a unique situation because it sees many
  different types of missing data problems and many
  different types of approaches to the analysis of
  Clinical Trials with missing data
• Also, how the study is designed could impact the
  nature and extent of the missing data problem.

20
Adaptive/Flexible Clinical Trial Design

• Most Phase III clinical trials are based on a
  fixed sample size design
• The success of the trial will depend on the
  validity of the protocol planning assumptions
• Our experience shows that trials may fail because
  of inadequately chosen doses, patient
  populations, primary endpoints, or anticipated
  effect size

21
Adaptive/Flexible Clinical Trial Design

• Flexible / Adaptive designs may provide more
  structure to the learn and confirm paradigms
• Flexible designs require interim looks at the
  data ( maybe unblinded)
• The applicability to the regulatory setting needs
  to be further explored

22
Active Control Studies

• The ethics of clinical trial has led to the more
  extensive use of active control clinical trails
• ICH E10 has dealt extensively with ACTs
• The importance of assay sensitivity and defining
  the margin have emerged as critical design
  features.

23
Active Control Studies

• There is a need for clarity for a regulatory
  perspective. (How to define the margin, what is
  the goal of the non-inferiority trial)
• Several methods have emerged as approaches to NI
  trial ( the syntheses method and the confidence
  method)
• But there is a lack of agreement on the best
  approach.

24
Active Control Studies

• The use of ACTs depends on the availability and
  the quality of the historical evidence
• If there is little confidence in the historical
  data should ACTs be conducted?
• Is the concept of retention a viable approach?
• The CP goal is to try to reach a consensus

25
Multiple Endpoints

• All drug-disease areas try to define clinically
  meaningful treatment outcomes
• Often more than one endpoint is studied in
  clinical trials
• Currently there is little consensus on how to
  treat these multiple endpoints and what kinds of
  adjustments are needed to maintain a
  pre-specified alpha error.

26
Multiple Endpoints

• Multiple endpoints in a regulatory setting have
  approval implications
• Multiple endpoints in a regulatory setting have
  label implications
• The recent( Oct. 20-21) PhRMA workshop in
  Washington DC focused completely on this topic.

27
Multiple Endpoints

• The main problem is how to get a consensus on
  multiple primary and secondary endpoints.
• This is not because of a lack of statistical
  models or proposals, the literature has many
  suggestions.
• FDA is trying to take advantage of its
  regulatory experience and has engaged the
  clinical and statistical staff to try to specify
  how to define this problem and to recommend best
  practices

28
Multiple Endpoints (Huque PhRMA 2004)

• Why in some trials more than 1 endpoints needs
  to show effects?
• Multiple Endpoints involve both clinical and
  statistical concepts
• The choice of endpoints may depend on where the
  endpoints lie on the causal pathways of the
  disease process and the mechanisms of actions of
  the study intervention
• Clinical expectation of the desired clinical
  benefit

29
Multiple Endpoints (Huque PhRMA 2004)

• Acne trial example Clinical Win criterion
• Three primary endpoints X non-inflammatory
  lesion counts, Y inflammatory lesion counts, Z
  physician global
• Clinical Decision rule for effectiveness (1) Z
  must show statistical significance. (2) In
  addition, X or Y must show statistical
  significance (without showing worsening in any)
• Possible Rationale X and Y lie on different
  causal pathways, and Z intersect with both.

30
Multiple Endpoints (Huque PhRMA 2004)

• Rheumatology Example (ACR20)
• Required
• at least 20 improvement in tender joint count
• at least 20 improvement in swollen joint count
• Plus at least 20 improvement in 3 out of the 5
• patient pain assessment
• patient global assessment
• physician global assessment
• patient self-assessed disability
• acute phase reactant (ESR or CRP)

31
each endpoint at level 0.025 (1-sided)
Multiple Endpoints (Huque PhRMA 2004)
Y-axis Type I error probability
X-axis Correlation

32
Multiple Endpoints (Huque PhRMA 2004)Adjustments
in the Type I error rateCase of 2 Endpoints
? Adjstment by Sidaks method on accounting
for correlation
33
Multiple Endpoints (Huque PhRMA 2004)Power
Comparison Case of K2 endpoints
34
Multiple Endpoints (Huque PhRMA 2004)Inference
about individual endpoints in clinical trials

• Uses a procedure that controls the family-wise
  Type I error rate appropriate fore endpoint
  specific claims
• Examples
• closed testing procedure
• gate-keeper/fixed-sequence methods
• alpha-calculus approach
• other approaches
• Reference (SAS publication 99).Westfall PH,
  Tobias RD, Rom D. Wolfinger, R.D. Hochberg,Y.
  Multiple Comparisons and Multiple Tests

35
Multiple Endpoints (Huque PhRMA 2004)Extent of
multiplicity adjustments between endpoints

• correlation

high
Practically no adjustments
Small adjustments
Good case for combining endpoints
Large adjustments
low
high
low
Causal dependence (Homogeneity of treatment
effects across endpoints)
36
Multiple Endpoints (Huque PhRMA 2004)Triaging of
multiple endpoints into meaningful families by
trial objectives

• Hierarchical ordered families

1) Prospectively defined 2) FWE controlled
Primary endpoints
Secondary endpoints
Exploratory endpoints
(usually not prospectively defined)

• Primary endpoints are primary focus of the
  trial. Their results determine
• main benefits of he clinical trials
  intervention.

• Secondary endpoints by themselves generally not
  sufficient for characterizing
• treatment benefit. Generally, tested for
  statistical significance for extended
• indication and labeling after the primary
  objectives of the trial are met.

37
Modeling and Simulation

• Has had a minor role to date in the planning and
  analyses of clinical trials in a regulatory
  setting
• But with the advent of high speed computers, a
  better understanding of the mechanism of drug
  action and a desire to minimize the number of
  clinical trials that are failing FDA is taking a
  another look at this approach.

38
Modeling and Simulation

• Failures of clinical trials can be traced to ill
  chosen dose, inadequate numbers of patients,
  studies that where to short in length, not
  anticipating the number and nature of dropouts
  and the impact of combining different subgroups
  into a clinical trial
• M/S has a potential role in testing the
  implications of the underlying assumptions before
  the first patient is entered into a clinical trial

39
Modeling and Simulation

• The first problem will be to understand the scope
  and utility of M/S in the drug development
  process
• The second level will be to find or train the
  necessary expertise in the use of M/S
• The third issue is to find or develop the
  necessary computational tools that are designed
  for the clinical trial planners and analyzers and
  are user friendly

40
Defining the role of Bayesian Methods in a
regulatory Setting

• Traditionally the design and analyses of clinical
  trials has been centered on the frequentist
  approach of hypothesis testing, estimation,
  p-values and confidence intervals
• In recent years, with the advances in
  computational methods we are beginning to see
  Bayesian designed clinical trials and analyses in
  the regulatory setting.

41
Defining the role of Bayesian Methods in a
regulatory Setting

• Bayesian methods focus on how belief can be
  modified by the data
• Even though this is an active area of statistical
  research and methodological development its
  utility in a regulatory setting has yet to be
  established.

42
Defining the role of Bayesian Methods in a
regulatory Setting

• FDA has been proactive in recent years by
  providing training to the statistical staff on
  Bayesian methods and acquiring computational
  tools which will support the evaluation of
  submission which include Bayesian analyses
• In May of 2004 FDA and the Johns Hopkins
  University held a workshop on Bayesian Approach.
  (the proceeding of this workshop are being
  prepared for publication)

43
Defining the role of Bayesian Methods in a
regulatory Setting

• To date there is no guidance on what kinds of
  trials or clinical settings are candidates for
  the application of Bayesian methods
• There is concern about can these methods give the
  right answer to the right question
• Can the type I error rate be properly controlled

44
Defining the role of Bayesian Methods in a
regulatory Setting

• Will the Bayesian approach be able to deal with
  the other issues discussed, multiple endpoints,
  missing data, and non-inferiority
• Will priors be based on data or subjective
  opinions
• While CDRH has been adopting Bayesian methods for
  years CDER is just beginning to explore this
  approach.
• We need to develop an understanding of what are
  the regulatory situations where BM can be applied
  with confidence and how to document the results
  of these types of designs and analyses

45
Risk assessment, management and Communication

• In June 2002, Congress passed the Prescription
  Drug User Fee Act (PDUFA III)
• FDA had a performance goal in the area of drug
  safety
• In May of 2004 FDA issued three draft guidance to
  comply with this Act. FDA is currently reviewing
  to comments to these drafts

46
Risk assessment, management and Communication

• It is clear that over the past few decades less
  attention has been given to the design and
  analyses of safety compared to efficacy
• Safety data is not consistently collected,
  analyzed and reported
• While there is a requirement for a statistical
  analysis plan for efficacy none has been required
  for safety.

47
Risk assessment, management and Communication

• The assessment of early phase I and II safety
  data needs to happen before one proceeds to Phase
  III trial
• Certain safety areas require special attention (
  hepatotoxicity, QT prolongation, nephrotoxicity
  and the rigorous assessment of informative
  dropouts from clinical studies)

48
Risk assessment, management and Communication

• FDA draft Guidance for Industry Premarketing
  Risk Assessment( Docket Number 2002D-0187)
  defines the problem and has suggestions for
  improvement
• There has been significant FDA/PhRMA interaction
  on this topic (including workshops and working
  groups)
• Also, FDA has several CRADAs to develop efficient
  tools for reporting, looking at and analyzing
  safety data

49
Enhancing Product Quality

• Traditionally pre-marketing product quality has
  been tested by an end product multiple batch
  approach
• Often post-marketing product quality requires a
  modification of this approach
• The current designs for testing product quality
  my not be efficient

50
Enhancing Product Quality

• Modern in process testing raises the possibility
  that alternatives to product quality should be
  considered
• There have also been advances in Process
  Analytical Technology (PAT) which depends on in
  process assess of product quality all along the
  drug manufacturing process

51
Enhancing Product Quality

• To advance it this area FDA needs to expand its
  expertise in modern PAT technology
• FDA as recently announced an initiative referred
  to cGMP for the 21st century

52
Strategic Plan or Way forward

• Progress on these critical path issues will
  require
• Clarity in stating problems and the hurdles to be
  overcome
• Getting the regulators, industry and academia
  working on these problems with the goal of
  finding a consensus

53
Summary of Critical Path Issues Dr. Woocock, DIA
Arlington Va. 2004

• Rising public expectations (new technology ,
  genomics etc.)
• Ten year investment in R and D going up
• Review time for priority applications going down
• Accelerated Drug Development
• Number of new submissions remains flat
• Development costs are increasing

54
Summary of Critical Path Issues Dr. Woocock, DIA
Arlington Va. 2004

• Success rate of new submissions less than 20
  years age
• Negative impact on the public health (companies
  focus on blockbuster drugs rather than public
  health )
• FDAs Critical Path effort targets the science
  underlying the drug development process
• Need more science and emphasis on mechanistic
  models
• Need more collaboration between industry,
  academia and patient groups

55
Strategic Plan or Way forward

• Holding workshops to build consensus
• FDA develop and publish guidance and seek public
  input.
• Monitor the success of this effort.

56
Conclusions

• There are real challenges in the field of Drug or
  Product Development
• These challenges include statistical issues that
  have existed for decades but have not been fully
  addressed
• With these challenges comes the opportunity to
  make progress on these issues

57
Conclusions

• Some of the solutions will involve understanding
  the existing methodology and reaching consensus
• Some of the solutions will be breaking new ground
  (e.g., modeling and simulation and Bayesian
  methods) and for these we will need to build
  understanding and expertise

58
Conclusions

• The FDA statistical staff is committed to making
  progress in these areas and move from stagnation
  to progress