Rapid Cycle Analysis for Early Detection of Vaccine Adverse Events

1
Rapid Cycle Analysis for EarlyDetection of
Vaccine Adverse Events
Richard Platt, MD, MScfor the CDC Vaccine
Safety Datalink Investigators Harvard Pilgrim
Health Care and Harvard Medical School
2
Why Do We Need Early Detection Systems?

• Rare adverse events may be impossible to detect
  in pre-licensure studies
• Spontaneous reports to passive surveillance
  systems, e.g. VAERS, often need rapid follow-up
• Designing follow-up studies can take months to
  years using traditional approaches

3
Rapid Cycle Analysis

• A modern approach to surveillance that takes
  advantage of VSDs strengths
• Update data on all vaccines and all subsequent
  outcomes every week
• Choose vaccines and potential adverse events to
  monitor
• Conduct weekly analysis

4
Sequential Analysis Methods

• Early data contribute to every subsequent
  analysis
• Repeated statistical testing of the same data
  requires special methods
• New method Maximized SPRT (Kulldorff et al.,
  2004)
• A refinement of sequential probability ratio
  testing (Wald, 1955)

5
Example Rotavirus vaccine and intussusception
1999
Vaccine licensed Aug 98 15 VAERs reports through
Jul 99
Vaccine suspended
Withdrawn
6
Example Rotavirus vaccine and intussusception
Log likelihood ratio
Critical value of LLR 3.3
Vaccine suspended
Vaccine withdrawn
7
Example Rotavirus vaccine and intussusception
Vaccine licensed Aug 98 15 VAERs reports through
Jul 99
Vaccine suspended
Vaccine withdrawn
8
Rapid Cycle Analysis Ongoing Surveillance via
VSD

• Meningococcal conjugate vaccine and
  Guillain-Barre syndrome
• Rotavirus vaccine and intussusception,
  gastrointestinal bleeding, and other events
• HPV, Tdap, MMRV, influenza vaccines being
  implemented

9
Implementing Rapid Cycle Analysis

• For each vaccine, choose the outcomes of interest
• Choose the comparison method concurrent
  controls, historical rates, or both
• Create programs to generate aggregate data from
  the 8 VSD sites
• Program analysis and run weekly

10
Choosing Outcomes

• Select outcomes that are
• Clearly defined
• e.g., Guillain-Barre syndrome or seizures rather
  than neurologic problems
• Acute-onset
• Relatively uncommon
• Plausible

11
Concurrent Comparison Analysis

• Uses matched controls, e.g., patients making
  preventive visits
• Advantage Avoids false signaling or missed
  signals due to secular trends
• Limitations
• Need to define appropriate control groups not
  simple!
• Vaccines may be adopted rapidly, leaving few
  controls

12
Results of Concurrent AnalysisOutcomes within 42
days after index event, VSD population, May
2005-March 2007

• The index event in concurrent controls was a
  preventive visit (N262,102) the analysis
  selects 1 control per vaccination visit.

13
Historical Comparison Analysis

• Uses incidence rates from existing data
• Advantage Knowing the historical rate of rare
  events allows earlier recognition that a small
  number of cases among vaccine recipients is
  unusual
• Example 4 cases of Guillain-Barre syndrome
  occur in vaccinees, 0 in controls
• Limitation Secular trends

14
Results of Historical AnalysisThrombocytopenia
within 42 days after meningococcal vaccination,
May 2005-March 2007
LLR for one-sided test, critical value of
B2.87, plt.05
15
What if Historical maxSPRT analysis of low
platelet countCritical value B 2.87 for ?.05
with upper bound1
LLR for one-sided test Plt.05 Plt.001
16
Limitationsof Rapid Cycle Analysis using maxSPRT

• Signals are not definitive, and follow-up may
  require
• Alternative analyses
• Evaluation of temporal clustering
• Chart review
• Unanticipated adverse events are better evaluated
  using data mining methods, e.g., temporal scan
  statistic

17
Next Steps

• VSD plans to implement surveillance rapidly
  whenever a new vaccine is introduced
• Refine methods of analysis for new situations,
  e.g. zoster
• Add new populations

18
Coming soon CERTs Health Plan Consortium for
Public Health

• Goal Improve the safety and safe use of marketed
  vaccines and prescription drugs by studying their
  use in health plan members
• Target population 100 million
• A planned activity of the Centers for Education
  and Research on Therapeutics (CERTs)
• Created under Congressional mandate to be a
  trusted national resource in therapeutics
• Administered by AHRQ in consultation with FDA
• Accepted processes for administering
  public-private partnerships

19
CERTs Health Plan Consortium for Public Health
Aims

• Timely risk identification and quantification
• Prospective evaluation of new therapeutics
  captured by health plan data
• focus on pre-defined list of potential problems
• Detailed followup of selected problems
• Identification of potentially unsafe use of
  preventive and therapeutic agents
• Other topics, subject to Board approval

20
CERTs Health Plan Consortium for Public Health

• Structure Public-private partnership
• health plans, federal agencies, industry,
  professional societies, public, foundations,
  academic community
• Data health plans automated data (claims) with
  access to full text medical records
• Transparency
• Proposed protocols available in advance for
  public comment
• Final protocols publicly available
• Final results publicly available
• Confidentiality and privacy
• Individuals HIPAA/IRB compliant
• Health plans identity and proprietary data
  protected

21
CERTs Health Plan Consortium for Public Health
Funding

• Infrastructure requires core funding
• Individual projects will require separate funding
  

22
CERTs Health Plan Consortium for Public Health

• Existing health plan data allow substantial
  enhancement of timeliness, power, and efficiency
  of post-marketing studies of therapeutics
• This information can/should complement other
  sources Medicare/Medicaid, VA, Vaccine Safety
  Datalink

23
Collaborators partial list

• James Baggs, CDC
• Jeff Brown, Harvard
• Arnold Chan, CERT
• Bob Davis, CDC
• Inna Dashevsky, CERT
• Rich Fox, Harvard
• David Graham, FDA
• Margarette Kolczak, CDC
• Martin Kulldorff, Harvard

• Ned Lewis, Kaiser
• Kim Lane, CERT
• Renny Li, Harvard
• Tracy Lieu, Harvard
• Parker Pettus, CERT
• Irene Shui, Harvard
• Eric Weintraub, CDC
• Katherine Yih, Harvard
• Ruihua Yin, Harvard
• Health plan-based VSD and CERT teams