Comparative Effectiveness Research

1
Comparative Effectiveness Research

• Shalini Kulasingam, PhD
• University of Minnesota

2
Overview

• Learning objectives
• Background why do we need a special type of
  research agenda?
• Definition comparative effectiveness research
• What areas/conditions have been prioritized for
  study?
• Role of nursing?
• Methods for conducting comparative effectiveness
  research
• Examples
• RCTs
• Observational studies
• Simulation modeling
• Funding

3
Learning Objectives

• Why there is a need for CER?
• Priority CER topics
• Study designs for conducting CER
• Examples of CER studies

4
Background

• Health care expenditures were 2.4 trillion in
  2008 and are projected to grow by an average of
  6.2 percent per year for the next 10 years, more
  than triple the projected rate of overall gross
  domestic product (GDP) growth (Sisko et al.,
  2009)
• The Congressional Budget Office (CBO) projects
  that under current law, health care will consume
  more than 30 percent of GDP by 2035 (CBO, 2008).

IOM report, 2009
5
Background

• Regional variations in treatment patterns and
  cost growth provide deeper insight into the need
  for more informed medical decision making.
• Patients in the highest-spending regions of the
  country receive 60 percent more health services
  than those in the lowest-spending regions, yet
  this additional care is not associated with
  improved outcomes (Fisher et al., 2003).

6
Background

• Research suggests that physicians in
• higher-spending areas are more likely than
  physicians in
• other regions to recommend costly interventions
  that
• have not been definitively shown to be effective
  (Fisher et
• al., 2009).
• Nationwide, the Institute of Medicine (IOM) has
• estimated that less than half of all treatments
  delivered
• today are supported by evidence (IOM, 2007).

7
Background

• A recent review of practice guidelines developed
  by the American College of Cardiology and the
  American Heart Association found that relatively
  few recommendations were based on high-quality
  evidencerandomized controlled trials, for
  instanceand many were based solely on expert
  opinion, individual case studies, or standard of
  care (Tricoci et al., 2009).

8
What is comparative effectiveness research?

• Comparative effectiveness research (CER) is the
  generation and synthesis of evidence that
  compares the benefits and harms of alternative
  methods to prevent, diagnose, treat, and monitor
  a clinical condition or to improve the delivery
  of care. The purpose of CER is to assist
  consumers, clinicians, purchasers, and policy
  makers to make informed decisions that will
  improve
• health care at both the individual and
  population levels.

9
CER Summary and Investment in Research

• How good is the intervention/treatment/test?
• In what patients?
• Under what circumstances?
• American Recovery and Reinvestment Act of 2009
• 1.1 billion down payment to support CER
• 400 million given to the NIH
• 300 million given to the AHRQ
• 400 million to Health and Human Services

10
What are the priority areas for research?

• The American Recovery and Reinvestment Act of
  2009 called on the Institute of Medicine to
  recommend a list of priority topics to be the
  initial focus of a new national investment in
  comparative effectiveness research. The IOMs
  recommendations are contained in the report,
  Initial National Priorities for Comparative
  Effectiveness Research.

11
What are the priority areas for research?

• Treatment strategies for atrial fibrillation
• Treatment for hearing loss
• Primary prevention versus clinical treatments in
  preventing falls in older adults
• Biologics for inflammatory diseases
• Upper endoscopy for patients with
  gastroesophageal reflux disease
• Dissemination and translation of techniques for
  use of CER by clinicians

IOM, 2009
12
What are the priority areas for research?

• Comprehensive care programs for people with
  chronic disease
• Interventions for MRSA
• Strategies to reduce health care associated
  infections
• Management of prostate cancer
• Registry for lower back pain
• Detection and management of dementia in a
  community setting

IOM, 2009
13
What are the priority areas for research?

• Management of behavioral disorders associated
  with dementia
• School-based interventions for treating obesity
  in children
• Interventions to reduce hypertension, obesity
  etc. in urban poor and Native American
  populations
• Management strategies for ductal carcinoma
  in-situ
• Use of imaging technologies for cancer
• Genetic and biomarkers for cancer

IOM, 2009
14
What are the priority areas for research?

• Prevention of dental caries in children
• Treatment strategies for children with ADHD
• Management of serious emotional conditions in
  children and adults
• Interventions to reduce health disparities
• Literacy sensitive disease management
• Interventions to reduce adverse birth outcomes in
  women especially African American women
• Prevention of unintended pregnancies

IOM, 2009
15
Role of Nursing

• Statement by MaryJean Schumann, Chief Program
  Officer, ANA, 2009
• Perspective is based on two types of nurses
• the registered nurse providing direct care
• the advanced practice registered nurse
• Certified Registered Nurse-Anesthetists (CRNAs)
  who provide critical anesthesia services
• Clinical Nurse Specialists (CNSs) who provide
  acute care expertise for complex patients
• Certified Nurse-Midwives (CNMs) who provide
  health care to women across the lifespan
• Nurse Practitioners (NPs) who deliver a wide
  range of primary care services.

16
Role of Nursing

• Nursings holistic view attention to the whole
  person makes nurses particularly effective in
  advancing these priorities. Nurses, with their
  expertise in health promotion, disease
  prevention, and health literacy, can contribute
  to changing the current sickness care system into
  a true health care system.

MaryJean Schumann,, ANA,. 2009
17
Setting priorities based on ANA- related work and
data

• National Quality Forum National Priorities and
  Goals
• Six Priority areas
• What are the most effective tools and systems to
  engage patients in their care?
• What are the most effective models for care
  coordination?
• How do we reduce 30-day readmission rates?
• How is palliative care best provided?
• How do we eliminate unnecessary or risky care?
• Improve health by ensuring that patients receive
  the most effective preventive services
  recommended by the U.S. Preventive Services Task
  Force.

MaryJean Schumann,, ANA,. 2009
18
Setting priorities based on ANA related work and
data

• CER priorities based on quality indicators
  developed by ANA
• 1998, ANA established the National Database of
  Nursing Quality Indicators (NDNQI), the only
  national database that provides nursing data and
  patient outcomes at the unit level where care
  occurs.
• Data are collected on structure, process and
  outcome measures in approximately 1400 hospitals
  of all sizes, in all 50 states and the District
  of Columbia.
• Data is collected on 17 measures, 11 of which
  have been endorsed by the National Quality Forum.

MaryJean Schumann,, ANA,. 2009
19
Area NOT recommended for further research

• A Cochrane review concluded that appropriately
  trained nurses can produce as high quality care
  as primary care doctors and achieve as good
  health outcomes for patients. It was noted that
  the research available is limited and some may
  call for further comparative studies. There are,
  however, no other professionals who have been
  subjected to the depth of study that NPs and CNMs
  have, and we question the need to expend limited
  resources on additional studies comparing
  professional groups, though we stand ready to
  play a role in the design and conduct of such
  studies should they be deemed necessary.

MaryJean Schumann, ANA. 2009
20
Study Designs for CER

• Synthesis of existing data
• Analysis of observational data
• Randomized controlled trials

21
Study Design

• Synthesis of existing data
• Systematic review
• Meta-analysis
• Decision modeling

22
Study Design

• Observational data
• Administrative claims
• Electronic medical records
• Registries
• Case control or cohort studies

23
Study Design

• Randomized controlled trial
• Luce et al. Annals Internal Medicine, 2009
• How to change RCTs for comparative effectiveness
  research
• Analytic and operational efficiency
• Reduce costs of running a trial, and be able to
  up date trials on an ongoing basis, dropping
  tests/drugs/interventions that are not promising
• Accomplish this using Bayesian approaches
• Pragmatic clinical trial

24
Study Design

• Pragmatic RCT
• CER objective is to provide information to help
  patients, consumers, clinicians, and payers make
  informed decisions.
• Trials tend to exclude relevant patient
  populations, commonly used comparators, long term
  outcomes, and non-expert providers
• Clinically effective comparators
• Study patients with common co-morbid conditions
• Diverse study patients
• Providers from community settings
• Provider and patient chosen outcomes

25
Potential Sequence for Identifying and Proposing
CER

• IOM list of priority topics lists those that are
  most likely to get funded
• IOM report notes that systematic reviews and
  meta-analyses provide information on areas for
  further study.
• Question can you use take a topic from the IOM
  priority list and identify a study for grant
  purposes?

26
Examples of CER Studies

• Detection of dementia in a community setting
• Systematic review
• Decision modeling
• Pragmatic trial
• Patient falls
• Systematic review
• Patient record review
• Randomized controlled trial with pragmatic
  aspects

27
Detection of dementia in a community setting
28
The original CMS-sponsored TA

• Aim Assess the benefits of FDG-PET scanning in
  patients with dementia, with mild cognitive
  impairment (MCI) and in asymptomatic patients
  with a family history of AD, subsequent to the
  standard evaluation as described in the American
  Academy of Neurology (AAN) guidelines.
• CMS requested that the AHRQ identify an Evidence
  Practice Center to perform a Technology
  Assessment (TA)
• Duke EPC assigned the TA in 2001

29
Methods of the original TA

• Literature review
• Decision model to provide an understanding of the
  decisional context

http//www.cms.hhs.gov/coverage/download/id64.pdf
30
Direct inference
Test
31
Indirect inference causal pathway
Test
32
Model Part 1
AD, treat
AD, no treat
No AD, treat
No AD, no treat
AD, treat
No AD, treat
AD, no treat
No AD, no treat
33
Test performance
AD by clinical evaluation No AD by clinical evaluation
Test True False
Test - False - True -
34
ROC curve of PET test accuracy based on the
literature review
35
Tree results mild dementia
True () False () False () True () Correct ()
No PET/ Rx 56 44 0 0 56
PET/ Rx 49 6 7 38 87
No PET/ no Rx 0 0 56 44 44
36
Asymptomatic
MCI
Model Part 2
Mild Dementia
Dead
Moderate Dementia
Severe Dementia
37
Illustrative patient history
38
Markov results mild dementia
QALY LE SDFLE
No PET/ Rx 4.10 7.89 4.02
PET/ Rx 4.09 7.88 4.00
No PET/ no Rx 4.02 7.82 3.86
39
Primary conclusion

• PET could improve the overall accuracy compared
  to accuracy of an exam based on AAN guidelines.
• Treatment based on an AAN-recommended examination
  leads to better health outcomes than treatment
  based on PET results

40
How can this make sense?

• While net accuracy with PET may be better, this
  is because there are many fewer false positives
  but a few more false negatives
• Incorrectly not treating (due to a false negative
  result a patient misses an opportunity for a Rx
  benefit) is worse than incorrectly treating (the
  patient unnecessarily receives medication,
  however the Rx is relatively benign, may be
  beneficial even if they dont have AD, and the
  personal downside is that their cognitive
  impairment/disability is not correctly labeled)

41
When testing is preferred

• 1. If a new treatment becomes available that is
  not only more effective than AChEIs but is also
  associated with a risk of severe adverse effects.

42
When testing is preferred

• 2.If testing could be demonstrated to be a better
  reference standard than an examination based on
  AAN guidelines. (i.e., testing would need to
  better distinguish patients who respond to
  therapy than is possible with a standard
  examination.)

43
When testing is preferred

• 3.If the results have demonstrable benefits
  beyond informing AChEI use.
• This value of knowing could have both positive
  and negative components.

44
A research agenda in service to decision-making

• Designs
• Minimize bias
• Maximize generalizability
• Why not a trial?
• In particular, why not a pragmatic clinical trial?

Kulasingam et al. Am J Alzheimers Dis Other
Demen. 2006
45
Design of a pragmatic clinical trial
Matched Communities
R
Patients identified, 1 page evaluation
completed
PET not reimbursed
PET reimbursed
Medicare claims (primary outcome resource use
from index date to 3 months)
Medicare claims at the community-level for
individuals with relevant ICD codes will be
examined (see Methods)
46
Design of a pragmatic clinical trial

• Design A demonstration project in which matched
  communities are randomly assigned to have FDG-PET
  reimbursed by Medicare or not.
• Allocation Communities are allocated to
  intervention or control by concealed
  randomization.
• Blinding Blinded outcome assessors/data
  collectors, biostatisticians.
• Follow-up period 3 years
• Setting Communities in which state-of-the-scienc
  e FDG-PET is reasonably available for various
  Medicare-covered clinical applications.
• Patients Patients will be enrolled based on (a)
  age 65, (b) free-living, (c) presenting without
  prior specific workup for a complaint of memory
  deficit, and (d) the physician specifies that
  some degree of workup is planned.

47
Design of a pragmatic clinical trial

• Intervention All participating communities will
  have a general education program regarding the
  diagnosis and evaluation of cognitive impairment,
  and will be informed how to enroll patients into
  the study. To ensure comparable patient
  identification in all communities, providers will
  be compensated on completion of a basic
  evaluation form for an eligible patient.
  Communities randomized to have FDG-PET reimbursed
  will have payment coordinated by the regional
  Medicare carrier. Communities randomized to not
  have FDG-PET reimbursed by Medicare will not have
  restrictions on FDG-PET if covered under other
  payment arrangements.

48
Design of a pragmatic clinical trial

• Measures Measures will consist of (a) a simple
  (i.e., 1 page) form completed on the date of
  presentation (index date) by the patients
  provider regarding basic demographic and clinical
  features, diagnosis/further diagnostic plan,
  treatment plan, and prognosis and (b) resource
  utilization related to initial evaluation and
  management of individuals with CI, as assessed
  via linked Medicare claims files. Cumulative
  resource costs from the index date to
  three-months (short-term) will serve as the
  primary outcome measure for purposes of sample
  size calculation. Additional measures will
  include resource counts (e.g., imaging studies,
  specialty referrals, laboratory testing, and so
  on), FDG-PET diffusion (in terms of proportion of
  candidates who have had a FDG-PET), as well as
  trajectory of resource use over time.

49
Review of Steps

• Priority topic
• Literature review/decision modeling to identify
  areas for further research
• Proposed pragmatic clinical trial

50
Patient Falls
51
Potential Areas of Research

• Patient falls are one example of how our quality
  work informs a CER priority. As an outcome of
  interest, falls are of critical importance,
  highlighted by CMS decision to include falls on
  the list of Hospital Acquired Conditions for
  which they no longer pay. There are many
  validated fall assessment tools, but there has
  not, to date, been any comparative research on
  the tools to determine which is more effective in
  determining fall risk assessment and which
  interventions are most effective for preventing
  falls.

MaryJane Schumann, ANA, 2009
52
Falls

• Coussement J et al. American Geriatrics Society,
  2008
• Interventions for Preventing Falls in Acute- and
  Chronic-Care Hospitals A Systematic Review and
  Meta-Analysis
• Goal To determine the characteristics and the
  effectiveness of hospital fall prevention
  programs
• Results Review showed that most studies were
  conducted on long-stay (mean length of stay (LOS)
  gt1.5 years) and rehabilitation units (mean LOS
  36.9 days).
• Results For analysis of the number of falls, one
  unifactorial and two multifactorial studies
  showed a significant reduction of 30 to 49 in
  the intervention group, with the greatest effect
  obtained in the unifactorial study that assessed
  a pharmacological intervention.

53
Falls

• Lakatos BE et al. Psychosomatics, 2009
• Objective to determine the prevalence of
  diagnosed and undiagnosed delirium in patients
  who fell during their hospital stay.
• Study design Retrospective chart review
• Methods Falls were categorized by their severity
  (i.e., minor, moderate, and major). Demographic
  information, patient outcomes, and diagnostic
  criteria for delirium (per DSMIV) were collected
  on the day of admission, the day of the fall, and
  the 2 days preceding the patients fall
• Results Falls in the general hospital were
  associated with delirium (both diagnosed and
  undiagnosed), advanced age, and specific surgical
  procedures

54
Falls

• Vass et al. Reducing Falls in In-patient Elderly,
  Trials, 2009
• Summary of an RCT that aims to reduce falls in an
  elderly in-patient population in an acute care
  setting.
• Background More than half of all in-patient
  falls in elderly people in acute care settings
  occur at the bedside, during transfers or whilst
  getting up to go to the toilet. In the majority
  of cases these falls are un-witnessed.
• Background New patient monitoring technologies
  have the potential to offer advances in fall
  prevention. Bedside sensor equipment can alert
  staff, not in the immediate vicinity, to a
  potential problem and avert a fall. However no
  studies utilizing this assistive technology have
  demonstrated a significant reduction in falls
  rates in a randomized controlled trial setting.

55
RCT for fall prevention

• The research design is an individual patient
  randomized controlled trial of bedside chair and
  bed pressure sensors, incorporating a
  radio-paging alerting mode to alert staff to
  patients rising from their bed or chair, across
  five acute elderly care wards in Nottingham
• University Hospitals NHS Trust.
• Participants will be randomized to bedside chair
  and bed sensors or to usual care (without the use
  of sensors). The primary outcome is the number of
  bedside inpatient falls.

56
RCT proposed data collection
Data collection Baseline Discharge
Demographics X
Previous falls/med. Hx. X
Cognitive Ability X X
Quality of Life X X
Activities of Daily Living X X
Discharge Destination X X
Length of stay X X
Fear of falling questionnaire X X
Total of in-patient falls X X
Vass et al. Trials, 2009
57
Review of Steps

• Priority topic
• Meta-analysis and chart review study to identify
  gaps
• Proposed clinical trial
• Pragmatic aspects are community setting
• Range of outcomes

58
Conclusions

• CER is new focus of funding at the NIH and AHRQ
• List of priority topics (IOM)
• Evidence reports and meta-analyses can provide
  information on gaps in knowledge base that
  require further study (AHRQ)
• Search grants.gov for RFAs or other announcements
  re new funding opportunities
• Lots of potential colleagues/collaborators at the
  U MN
• School of Public Health
• School of Medicine
• School of Dentistry

59
Funding and Acknowledgements

• Shalini Kulasingam is supported by NCI grant
  K07-CA113773
• Previously funded by
• Grants Merck, CSL-Australia, SP-MSD, CDC, NIH,
  mtm
• Consultant SP-MSD, CSL New Zealand, Medtronic
• Collaborators
• Evan Myers, Duke University
• George Sawaya, University of California, San
  Francisco
• Joy Melnikow, University of California, Davis
• Mark Schiffman, Philip Castle, NCI
• Eduardo Franco, Raghu Rajan, McGill University
• Laura Koutsky and Akhila Balasubramanian,
  University of Washington
• Patti Gravitt, Johns Hopkins University
• Levi Downs, Rahel Ghebre, Ruby Nguyen, Karen
  Kuntz, University of Minnesota