Infusion of Statistical Science Into Comparative Effectiveness Research: What, Why, How, Who? Sally C. Morton Statistics and Epidemiology, RTI International

1
Infusion of Statistical Science Into Comparative
Effectiveness ResearchWhat, Why, How,
Who?Sally C. MortonStatistics and
Epidemiology, RTI International
2
What is Comparative Effectiveness Research (CER)?

• The generation and synthesis of evidence that
  compares the benefits and harms of alternative
  methods to prevent, diagnose, treat, and monitor
  a clinical condition or to improve the delivery
  of care. The purpose of CER is to assist
  consumers, clinicians, purchasers, and policy
  makers to make informed decisions that will
  improve health care at both the individual and
  population levels.

Institute of Medicine Committee on Comparative
Effectiveness Research Prioritization (2009)
3
Other Names for CER Clinical Effectiveness
Research or Patient-Centered Research

• Which treatment works best, for whom, and under
  what circumstances? (Slutsky and Clancy. Am J
  Med Qual 20092467-70)
• Informs a clinical decision (screening,
  diagnosis, treatment)
• Focuses on effectiveness not efficacy
• Compares at least 2 alternatives, one may be
  usual care
• Measures outcomes that matter to patients,
  including harms and benefits, at both the
  population and subgroup levels
• Includes generation and analysis of new evidence
  secondary data analysis and synthesis

Institute of Medicine Committee on Comparative
Effectiveness Research Prioritization (2009)
4
Why CER Now?

• Focus on healthcare reform
• Clinical practice often not based on evidence
• Coverage decisions often unrelated to
  effectiveness
• Large and escalating volume of information
• Synthesis required
• Relevance to clinical practice unclear
• CER expected to increase quality, effectiveness,
  and efficiency by
• Summarizing existing knowledge
• Identifying knowledge gaps

5
Its Where the Money Is

• Dramatic increases in funding at AHRQ for CER
• 30 million in FY07
• 50 million in FY08, now 88 million
• ARRA (Stimulus Bill) targeted for CER
• 1.1 billion in FY09 and FY10
• 300 million to AHRQ
• 400 million to NIH
• 400 million to HHS Office of the Secretary
• Institute of Medicine (IOM) Committee to
  recommend priorities for CER

6
How Do Statistical Methods Contribute To
Comparative Effectiveness Research?
Evidence generation prospective and
restrospective analysis of secondary databases
such as medical registries and claims data
Evidence generation primary data collection via
clinical trials and observational studies
Evidence synthesis systematic reviews and
meta-analyses
Prioritization of gaps in evidence generation,
synthesis, communication, or translation
Evidence translation generation of information
for different decision-makers, e.g., clinical
practice guidelines
Evidence synthesis cost-effectiveness analysis
and decision modeling
Evidence communication dissemination of findings
Challenge Statistical methods that are
scientifically rigorous, relevant
(patient-centered), and timely
7
Factors That Enter Into Clinical Decisions
Mulrow et. al. Ann Intern Med 1997126389-391
8
Continuum From Research Studies to Clinical
Practice Guidelines
9
Focus on Evidence Generation Via Primary Data
Collection Studies

• RCTS were recommended for 49 of the top 100 CER
  research priorities identified by the IOM
• The paradox of the clinical trial is that it is
  the best way to assess whether an intervention
  works, but arguably the worst way to assess who
  will benefit from it. (Mant. Lancet
  1999353743-46)
• Knowing a treatment is efficacious is necessary
  but not sufficient for knowing it is effective

10
Distinguishing Efficacy and Effectiveness
Study Elements Efficacy Studies Effectiveness Studies
Study design Protocol driven visits conducted at regular intervals medication often masked Clinician visits driven by patient needs and physician practice treatment usually unmasked, allows modification of treatment
Patient population Homogeneous highly selected for patient characteristics, adherence, no comorbidities Heterogeneous more representative of patients in the real world
Study sites Research specialists, academic medical centers General practice physicians usual caregivers, facilities, settings
Sample size Sufficient only to detect trial outcome often not adequate for subgroup analyses Adequate to assessing minimally important difference from patient perspective, sometimes for subgroups as well
Study endpoints Often intermediate primary endpoint is often a physiologic endpoint (surrogate) May include patient- reported outcomes clinically relevant outcomes
11
Distinguishing Efficacy and Effectiveness(Cont)
Study Elements Efficacy Studies Effectiveness Studies
Duration Often short several weeks to months Often longer to parallel use in practice and to allow for nonadherence
Comparator Usually placebo (for pivotal efficacy trials for FDA approval) Relevant clinical comparator or usual care
Data Collection Stops with event or study discontinuation data collected as part of treatment visits Captures data even after event or study discontinuation data collected as part of routine care
Analysis Typically intention to treat Various, depending on study design
External validity Limited applicability Broader applicability
12
How Do We Expand CER Beyond Explanatory (Ideal
Setting) RCTs

• How can we improve clinical trial organizational
  efficiency, analytic efficiency, and usability
  for decision-makers? (Luce et al. Ann Intern Med
  2009151206-209)
• Pragmatic trials (real-world settings)
• Cluster randomized trials
• Bayesian adaptive designs
• How can we improve methods to control for
  confounding in observational studies?
• Propensity scores
• Instrumental variables

13
Focus on Evidence Synthesis Unadjusted Indirect
Comparison
How does one compare A vs. B when no direct
(head to head) evidence is available?
14
Adjusted Indirect Comparison
In this approach, the comparison is adjusted by a
common comparison group
Alternative approach network analysis
15
Overarching Principle Fit For Purpose

• Decisions about the use of therapeutic
  interventions, whether for individuals or entire
  healthcare systems, should be based on the
  totality of available evidence. The notion that
  evidence can be reliably or usefully placed in
  hierachies is illusory. Rather, decision makers
  need to exercise judgement about whether (and
  when) evidence gathered from experimental or
  observational sources is fit for purpose.

Rawlins. Clin Med 20088579-88.
16
Questions to Ponder

• How can RCTs be designed to most benefit CER?
• When and how can observational studies complement
  experimental studies to answer CER questions?

RCTs efficacy Observational
Studies effectiveness
17
Questions to Ponder

• What methods for observational studies adequately
  address confounding?
• What meta-analytic methods for indirect evidence
  will inform CER questions?
• Whose data are they?
• What are the statistical methods research
  priorities?
• Priority list A product of this conference

18
Comparative Effectiveness Research

• (Sort of) new wine
• Emphasis is driven by technology availability,
  payer interest, rising chronic disease burden
• New bottle
• Federal and payer interest high in the next few
  years
• Critical to be successful early
• I encourage the statistical community to have a
  glass!

Adapted from lecture by Tim Carey on CER