Approach to Outcome Measure Development or Selection: A Regulatory Perspective

1
Approach to Outcome Measure Development or
Selection A Regulatory Perspective

• Initiative on Methods, Measurement, and Pain
  Assessment in Clinical Trials (IMMPACT)
• April 17-18, 2014
• Ashley F. Slagle, MS, PhD
• Study Endpoints and Labeling Development (SEALD)
• Office of New Drugs (OND)
• Center for Drug Evaluation and Research (CDER)

2
Disclaimer

• The views expressed in this presentation are
  those of the speaker, and do not necessarily
  represent an official FDA position.

3
Treatment Benefit

• Treatment benefit is demonstrated by evidence
  that the treatment has a positive impact on a
  concept of interest
• How long a patient lives
• How a patient feels or functions in daily life
• Treatment benefit can be demonstrated as either
• A comparative advantage in how patients survive,
  feel or function
• A comparative reduction in treatment-related
  toxicity

4
Purpose of Outcome Assessment

• To determine whether or not a drug has been
  demonstrated to provide treatment benefit
• A conclusion of treatment benefit is described in
  labeling in terms of the concept of interest
  (COI), the thing measured by the outcome
  assessment

5
Types of Outcome Assessments

• Survival
• Biomarkers
• A physiologic, pathologic, or anatomic
  characteristic that is objectively measured and
  evaluated as an indicator of some normal or
  abnormal biologic function, process or response
  to a therapeutic intervention 
• Clinical outcome assessments (COAs)
• Performance outcomes (PerfOs)
• Clinician-reported outcomes (ClinROs)
• Observer-reported outcomes (ObsROs)
• Patient-reported outcomes (PROs)

6
Choice of COA Type

• Determine the most appropriate reporter for the
  COI in the COU
• If symptom intensity is the concept of interest
  in a patient population that can respond
  themselves, a PRO is most appropriate.
• If clinical judgment is required to interpret an
  observation, a ClinRO is chosen.
• If the COI can only be adequately captured by
  observation in daily life (outside of a
  healthcare setting), and the patient cannot
  report for him or herself, then an ObsRO is
  chosen.
• When it would be useful to observe an actual
  demonstration of defined tasks demonstrating
  functional performance in the clinical setting, a
  PerfO may be appropriate.

7
Evidence of Treatment Benefit

• Direct evidence of treatment benefit is derived
  from studies with endpoints that measure
  survival, or how patients feel and function in
  daily life.
• Indirect evidence of treatment benefit is derived
  from studies with endpoints that measure other
  things that are related to how patients survive,
  feel or function

8
Direct Verses Indirect Evidence of Treatment
Benefit
Survival Pain Breathlessness
Blood Pressure PSA
6MWT

Indirect Evidence
Direct Evidence
Evidence Continuum
9
Treatment Benefit What To Measure?All are
important, but interpretation of trial results
depends on knowing how treatment impacts the core
disease-defining concepts first.
Proximal disease Impact concepts
Distal disease Impact concepts
Disease impact on general life concepts
Disease-defining concepts
General psychologicalfunctioning
Productivity
Core signs,symptomsor decrements in functioning
Related functioning
Additional functioning
Health status
General physical functioning
Health-related quality of life
Additional S/Ss
Related S/Ss
Social functioning
Satisfaction withhealth
10
Adequate and well-controlled efficacy (AWC)
studies

• Studies that provide
• Evidence to support drug marketing authorization
• Substantial evidence of effectiveness
• Required by law to support a conclusion that a
  drug is effective
• See 21 CFR 314.126
• Deemed AWC based on multiple features of a
  clinical study design including
• Nature of the primary endpoint
• Well-defined and reliable
• Rigor of control of the Type I error rate
• Prospectively planned analyses designed with rigor

11
When is a COA adequate for use in adequate and
well-controlled studies?

• Regulatory standard measures are well-defined
  and reliable
• Empiric evidence demonstrates that the score
  quantifies the concept of interest in the
  targeted context of use
• What does this mean?
• This means measuring the right thing (concept of
  interest), in the right way in a defined
  population (targeted context of use), and the
  score that quantifies that thing does so
  accurately and reliably, so that the effects seen
  in the outcome assessment can be interpreted as a
  clear treatment benefit.

12
Good Measurement Principles

• Defines good measurement principles to consider
  for well-defined and reliable (21 CFR 314.126)
  PRO measures intended to provide evidence of
  treatment benefit
• All COAs can benefit from the good measurement
  principles described within the guidance

http//www.fda.gov/downloads/Drugs/GuidanceComplia
nceRegulatoryInformation/Guidances/UCM205269.pdf
12
13
Well-defined and Reliable

• The tool adequately measures the concept of
  interest in the context or clinical setting of
  interest
• To assess this, we review the tools measurement
  properties
• Content validity
• Construct validity
• Reliability
• Ability to detect change
• Information to support interpretation of change

13
14
Review of ClinRO, ObsRO, PRO, and PerfO Measures
Any Differences?

• SAME
• Instrument
• Targeted Claims
• Endpoint Model
• Conceptual Framework
• Content Validity
• Other Measurement Properties
• Interpretation of Scores
• Language Translation and Cultural Adaptation
• Data Collection Method
• Modifications
• Clinical Trial Design and Data Analysis Issues
• Key References

• DIFFERENT

Nothing?

15
Clinical Outcome Assessment Considerations

• Not all patient reported, clinical-reported,
  observer-reported, or performance outcome
  assessments are appropriate Clinical Outcome
  Assessments
• May be useful for other purposes
• Diagnostic
• Prognostic
• Trial eligibility and trial enrichment
• Epidemiologic or population studies
• Clinical practice decision-making
• Measures used successfully for these other
  purposes will not necessarily be appropriate
  outcomes assessments (i.e., they may not be able
  to reliably detect treatment benefit in clinical
  trials or support labeling claims in a
  non-misleading way)

16
Seeking Advice from FDA

• Discuss plans early!
• 2 pathways
• In the context of an Investigational New Drug
  (IND) program
• Drug Development Tool (DDT) Qualification

17
DDT Guidance (Final January 2014)

• Describe a process NOT evidentiary standards
• Qualification process described for Biomarkers,
  Animal Models, and Clinical Outcome Assessments
  (COA)

http//www.fda.gov/downloads/Drugs/GuidanceComplic
anceRegulatoryInformationi/Guidances/UCM230597.pdf
17
18
CDER Qualification of Clinical Outcome
Assessments

• DDT Qualification Guidance COA qualification is
  a conclusion that within the stated context of
  use (COU), the results of measurement can be
  relied upon to represent a specific concept (COI)
  with a specific interpretation when used in drug
  development and regulatory decision-making
• Plain language Within a specific clinical
  context, were measuring the right thing, in the
  right way, and we can rely upon the results of
  the qualified assessment across clinical trials
  within that clinical context
• CDER qualification is currently reserved for
  those COAs that are ultimately intended to
  support primary or secondary endpoints in
  clinical trials
• Qualified instruments shall be made available
  publically available

19
Roadmap to Patient-Focused Outcome Measurement in
Clinical Trials

• Intended to illustrate how one might embark upon
  a sound, orderly, instrument selection or
  development pathway, beginning with the clinical
  context in which the instrument is intended to be
  used.

20

Roadmap to PATIENT-FOCUSED OUTCOME MEASUREMENT in
Clinical Trials

• Identify the meaningful health aspect that is
  the intended benefit to patients in their daily
  lives
• Survives (e.g., length of survival)
• Feels (e.g., symptom severity)
• Functions (e.g., walking ability)

Natural history of the disease or condition
Onset/Duration/Resolution Diagnosis
Pathophysiology Range of manifestations

• Search for existing clinical outcome assessment
  measuring the concept(s) of interest in the
  context of use
• Measure exists
• Measure exists but needs to be modified
• No measure exists
• Measure under development

• B. Identify the measureable concept of interest
  that represents the meaningful health aspect,
  which can be
• Equivalent to the meaningful health aspect (e.g.,
  patients self-reported ambulatory activities in
  daily life) OR
• Distinct from, but related to the meaningful
  health aspect (e.g., 6-minute walk test)

Patient subpopulations By severity By
onset By comorbidities By phenotype
B. Begin clinical outcome assessment
development Document content validity
(qualitative or mixed methods research)
Evaluate cross-sectional measurement properties
(reliability and construct validity) Create
user manual Consider submitting to FDA for
qualification for use in exploratory studies
Health care environment Treatment
alternatives Clinical care standards Health
care system perspective

• C. Define context of use for clinical trials,
  e.g.
• Disease/Condition entry criteria
• Clinical trial design
• Endpoint positioning

C. Complete clinical outcome assessment
development Document longitudinal measurement
properties (construct validity, ability to
detect change) Document guidelines for
interpretation of treatment benefit and
relationship to claim Update user manual
Submit to FDA for qualification as
effectiveness endpoint to support claims
Patient/caregiver perspectives Definition of
treatment benefit Benefit-risk tradeoffs
Impact of disease

• D. Consider appropriate clinical outcome
  assessment type(s)
• Patient-Reported Outcome (PRO)
• Observer-Reported Outcome (ObsRO)
• Clinician-Reported Outcome (ClinRO)
• Performance Outcome (motor, sensory, cognition)

U.S. Food and Drug Administration Center for Drug
Evaluation and Research Office of New
Drugs http//www.fda.gov/Drugs
Updated on March 14, 2014
21
(No Transcript)
22
Understanding the Disease or Condition
23
Understanding the Disease or Condition
24
Understanding the Disease or Condition
25
Understanding the Disease or Condition
26
FDAs Patient Focused Drug Development Initiative

• Systematically gather patients perspectives on
  their condition and available therapies to treat
  their condition
• 20 public meetings over the course of PDUFA V,
  each focused on a specific disease area
• http//www.fda.gov/ForIndustry/UserFees/Prescripti
  onDrugUserFee/ucm368342.htm

27
(No Transcript)
28
Conceptualizing Treatment Benefit
29
Conceptualizing Treatment Benefit
30
Defining Context of Use

• Each of the following variables can impact the
  adequacy of a COA to support a claim
• Disease definition including, if appropriate
• Disease subtype
• Disease severity
• History of previous treatment
• Patient subpopulations
• Patient demographics
• Reporting ability
• Culture and language
• Clinical trial design and objectives
• Endpoint positioning
• Endpoint definitions
• Analysis plan
• Methods for interpretation of study results
• Targeted labeling claim
• Clinical practice and study setting
• Inpatient vs. outpatient
• Geographic location
• Clinical practice variation

30
31
Endpoint Definition and Positioning

• Create study objectives based on the COI in the
  COU
• Position the outcomes as trial endpoints that
  will be interpretable in comparison with a
  control group
• Define endpoints using COA scores
• Plan analysis
• Measurement of change over time in individual
  patients that are combined for a means of
  assessing a group score
• Analysis of means
• Analysis of proportions
• Hierarchy for testing multiple assessments

32
Conceptualizing Treatment Benefit
33
(No Transcript)
34
Selecting/Developing the Measure
35
Selecting/Developing the Measure
36
Selecting/Developing the Measure
37
COA Wheel and Spokes

• This diagram identifies the key components of the
  documentation submitted to CDER to support COA
  qualification
• The Wheel and Spokes diagram also represents the
  general iterative process used in developing a
  COA for qualification.

38
(No Transcript)
39
Spoke I
40
Spoke II
41
What Is Content Validity?

• Content validity is the extent to which the
  content of an instrument represents important
  aspects of a given concept for an intended use
  and for a defined target population
• Establishing content for a new instrument may
  involve both qualitative and quantitative
  research methods. Qualitative data are essential
  for establishing content validity of a COA
• Input from the target population is essential

42
Conceptual Framework

• An explicit description or diagram of the
  relationships between the questionnaire or items
  in an assessment and the concepts measured
• Describes how the individual items contribute to
  the total score that will be analyzed and
  ultimately described in labeling

43
Conceptual Framework
Score of Domain A
Item 1 Item 2 Item 3 Item 4 Item 5 Item 6
Domain Concept A
Total Score
Overall Concept
Score of Domain B
Domain Concept B
44
Spoke III
45
Qualification for Use in Exploratory Studies / as
Exploratory Endpoints

• At this point in time, submitters may consider
  the option of submitting evidence for COA
  qualification. Qualification at this point in
  development will be for use in exploratory
  analyses for purposes of testing other
  measurement properties.

46
Spoke IV
47
COA Qualification for Use as Primary or Secondary
Endpoint

• When all measurement properties are tested,
  evidence will be reviewed to support COA
  qualification for use in adequate and
  well-controlled studies as a primary or secondary
  endpoint measure of effectiveness.

48
Spoke V
49
Conclusion

• The roadmap to a well-defined and reliable
  outcome assessment begins with a full
  understanding of the disease or condition to be
  tested
• An assessment cannot be chosen or developed
  without a well-defined context of use,
  understanding of the meaningful health aspect,
  and targeted concept of interest to be assessed
• The science of measurement continues to evolve
  with new tools and methods for efficient
  development and modification of assessments
• There is no one size fits all approach to measure
  development, we all must endeavor to be flexible,
  while applying good measurement principles as
  appropriate for each unique situation