Thoughts on the Use of Decision Analysis in the Review of New Drug Applications

1
Thoughts on the Use of Decision Analysis in the
Review of New Drug Applications

• October 3, 2007
• Todd Durham

2
Outline

• NDAs and the nature of the decision
• Potential benefits and challenges of decision
  analysis
• An illustration
• Learning and opportunities

3
Mental Exercise

• Imagine that tomorrow you are diagnosed with a
  disease from which you will die in exactly 7
  days.
• If you could take a pill
• That would definitely cure you from this disease,
  how much would you pay for it?
• That would give you a 25 chance of a cure, how
  much would you pay for it?
• That would give you a 25 chance of a cure, how
  much risk (s) of a debilitating stroke would you
  accept?

4
New Drug Applications

• Marketing applications for new drugs
• FDA reviewed between 20-30 NDAs (for NMEs) per
  year between 2001-2003 (FDA, Critical Path, 2004)
• Data submitted with a NDA
• Human evidence of benefit
• Human evidence of risk
• Manufacturing controls
• Animal data on toxicology and carcinogenicity

5
Objective in Reviewing a NDA

• Decide if a drugs benefits outweigh its risks
• Evolved historically with various changes in the
  law to
• Avoid misleading doctors or consumers
• Keep dangerous drugs out of the system
• What does the law really say?

6
Substantial Evidence from FDC Act of 1962

• Substantial evidence was defined in section
  505(d) of the Act as evidence consisting of
  adequate and well-controlled investigations,
  including clinical investigations, by experts
  qualified by scientific training and experience
  to evaluate the effectiveness of the drug
  involved, on the basis of which it could fairly
  and responsibly be concluded by such experts that
  the drug will have the effect it purports or is
  represented to have under the conditions of use
  prescribed, recommended, or suggested in the
  labeling or proposed labeling thereof.
• (FDA, Clinical Evidence of Effectiveness, 1998)

7
Sufficient Criteria for Demonstration of Efficacy

• Choice of Primary Endpoint
• Reliably measures a clinically relevant
  characteristic
• Statistically sensitive to treatment
• Identified a priori (with corresponding analysis
  methods)
• Results for Primary Endpoint
• Treatment effect is statistically significant
  in at least two studies
• Magnitude of treatment effect (?) is clinically
  relevant
• Results for Secondary Endpoints
• Results from secondary endpoints further describe
  the relevance of ? (primary endpoint) if results
  from primary endpoint in the same study are
  statistically significant

8
The Case of Carvedilol

• the usual two-study FDA paradigm does not make
  sense under all situations. This much is clear.
  But I would also suggest, as stated above, that
  experience has shown the paradigm to be a very
  useful guideline exceptions should therefore be
  relatively unusual, and, when in doubt one
  should err on the side of conservatism.
  Nevertheless, it strikes me as absurd in extreme
  cases to insist that if one does not meet the
  original primary end point in a study, that
  conclusions can never be definitive but only
  hypothesis generating. (Fisher, 1999)

9
Criteria Used in Reviewing a NDA

• Benefit
• Quantity of evidence
• Quality of evidence
• Typically restricted to one or a few endpoints
• Leads to a labeled claim consistent with results
• Safety
• From any number of reported adverse events
• Cardiac safety studies (e.g., QTc)
• Potentially animal studies (e.g., risk to fetus)
• Manufacturing

10
Decision to be Made

• Approve the new drug
• Reject the new drug
• Ask the sponsor for more information
  (approvable)

11
Influences on the Decision

• Statistical robustness of the apparent benefit,
  with appropriate statistical control of the false
  positive rate
• Clinical relevance of the benefit
• Excess safety risks, with no control of the false
  positive rate
• Severity of the disease
• Availability of other treatments

12
When a Drug is Approved

• Can be legally marketed in the U.S.
• Doctors have a prescribing option
• Patients have a treatment option
• Pharmaceutical companies make revenue
• Need for education all around
• Safety will continue to be monitored
• Surveillance has less rigor than RCTs
• May be studied further
• Expand the label
• Clarify the role of the new drug or its effects

13
When a Drug is Approvable

• Can not be legally marketed in the U.S.
• Doctors can not prescribe it
• Patients can not take it
• May be studied further
• Pharmaceutical companies spend more money on
  research
• Time for further research and submission

14
When a Drug is Rejected

• Sponsor may withdraw application
• Can not be legally marketed in the U.S.
• Doctors can not prescribe it
• Patients can not take it

15
Easy Approval Decisions

• A lot of evidence of clear benefit
• Clinically relevant
• Statistically robust (very unlikely due to
  chance)
• At least moderately sized safety database
  reflects reasonable risks
• No evidence of toxic or carcinogenic effects
• No other available treatments or just a few
  treatments with some toxicities

16
Easy Rejection Decisions

• Obvious hazards with little benefit
• Poor manufacturing controls

17
Decisions are Much Harder When

• Mixed results for benefit
• Drug which has been shown to have a benefit in
  some populations but not others.
• A lot of studies, only a few of which were
  successful.
• Statistical criteria for success are not met.
• Some significant trade-offs must be reckoned with.

18
Made Even More Difficult

• Changing landscape
• Regulatory standards (e.g., emerging concerns)
• Medical advances
• Changing standard of care
• Ex-US medical care
• External pressures
• Congress
• Patient advocates
• Pharmaceutical industry

19
Benefits of a Decision Analysis

• Transparency of the decision
• Many objectives possible (identified, weighting)
• Influences for all stakeholders
• Role of uncertainties
• Which ones make the most difference?
• Model that can be applied to many products in the
  same therapeutic area, but evolve over time.
• Dissection of the problem ? greater understanding

20
Transparency

• Patients
• To pharmaceutical company
• Within the FDA
• Congress

21
Role of Uncertainties

• How much do the following uncertainties bear on
  the consequences?
• Quality or quantity of evidence of benefit
• Medical need, population affected
• Available therapies
• How many patients will actually use the
  treatment?
• Dont need to be accurate but having a grasp on
  the range of uncertainties can still be
  instructive (through tornado diagrams)

22
An Evolving Model

• Changes in medicine
• Changes in how medical expenses are reimbursed
• Changes in societal priorities or norms

23
Dissection of the Problem

• Factors which most influence the best decision
  can lead to new priorities
• Role of available therapy ? compare the new
  treatment to available therapy
• Quantity of evidence ? additional information
• The safety/benefit tradeoff ? patient involvement
• Insensitivity of the model to various
  uncertainties can make decisions easier

24
Challenges of DA for this Application

• How to define the safety risks?
• All of them?
• Control of false positive rate?
• How to assess the consequences
• By whom?
• Using what measure?

25
Consequences

• Time
• Money
• Human lives
• Unwanted events
• Quality-adjusted life years
• Credibility / trust (how to value?)
• Quality of information (what is its value?)

26
Basic Decision Tree
Approved
Approvable
NDA Decision
Rejected
27
Waiting for More Information
Success
Approved
Outcome
Yes
Failure
Approvable
NDA Decision
New Study?
No
Rejected
Presumably, success would lead to a greater
chance of regulatory approval, but what are the
consequences of having made this decision to wait
for more information?
28
Illustration Serious Diagnosis

• Advanced cancer that affects 50,000 individuals
  per year
• Current expected life-span (median) is 20 months
  from diagnosis.
• The one available treatment is not tolerated well
  such that most patients choose not to take it.
• Loosely adapted from story in New York Times,
  2007.

29
Results from Clinical Trials

• New treatment compared to placebo
• Efficacy
• Treatment effect is 4 months of survival
  (benefit) in two studies.
• In one study survival had a nominal p-value
  lt0.050, but it was a secondary endpoint.
• Primary endpoint was stopping progression of
  cancer (failed in both studies).
• Safety
• Most common side effect is flu-like symptoms
• 1-2 chance of a stroke from new treatment

30
Considerations

• DA could address the consequences of a world with
  (now or later) and without the new treatment
• Lives lost in a period of time
• New strokes in a period of time
• Bouts of flu-like symptoms
• Was survival a false positive finding?
• Zero survival benefit
• What to do with the conventional hypothesis
  testing interpretation?
• Wont the benefit depend on how many patients
  might use the new treatment?

31
Could this Ever Be Applied?

• Modest proposals
• FDA could conduct an exercise by writing out an
  influence diagram for approval decisions in one
  therapeutic area.
• Carry out research on how to best communicate
  risk to patients (both benefit and safety).
• Increased emphasis on risk communication to
  patients. Steiner, 1999 has tremendous insight
  on the topic.
• More difficult proposal
• Conduct focus groups with patients to examine
  ability to elicit their trade-offs. Howard has
  written on ways to value life and other outcomes.
• Fantasy-land proposal
• Make all drugs available for marketing and change
  the regulatory paradigm such that regulators
  verify accuracy of labeling and educate doctors
  and the public.

32
Learning from Experience

• Unexpected clarity, almost profound new
  understanding of the decision to be made.
• Ability to proceed without regret knowing the
  problem had been understood as best as humanly
  possible.
• Training is important. Even highly intelligent
  people do a poor job of estimating uncertain
  quantities.

33
Illustration What if

• The benefit was only 0-4 months of survival, with
  a great deal of skepticism that 4 months from the
  trials was real?
• Some patients might trade the chance of a stroke
  for a chance at an extra month or two of life.
• But they cant make this choice unless the drug
  is made available to them.
• We wont know unless we ask.

34
References

• US Department of Health and Human Services, Food
  and Drug Administration, 2004. Challenge and
  opportunity on the critical path to new medical
  products. Available from www.fda.gov.
• US Department of Health and Human Services, Food
  and Drug Administration, 1998, Providing clinical
  evidence of effectiveness for human drug and
  biological products. Available from www.fda.gov.
• Fisher L. Carvedilol and the Food and Drug
  Administration (FDA) Approval Process The FDA
  Paradigm and Reflections on Hypothesis Testing.
  Controlled Clinical Trials 1999201639.
• Steiner J. Talking About Treatment The Language
  of Populations and the Language of Individuals.
  Annals of Internal Medicine 1999 130,7 618-622.
• Howard RA. On Making Life and Death Decisions.
  Readings on the Principles and Applications of
  Decision Analysis. Howard RA and Matheson JA,
  editors. 1989. Strategic Decisions Group.
• Howard RA. On Fates Comparable to Death.
  Readings on the Principles and Applications of
  Decision Analysis. Howard RA and Matheson JA,
  editors. 1989. Strategic Decisions Group.
• Andrew Pollack, Panel Endorses New Anti-Tumor
  Treatment, The New York Times (March 30, 2007).