Incorporating Epidemiologic Context into a Clinical Prediction Model for Pertussis

1
Incorporating Epidemiologic Context into a
Clinical Prediction Model for Pertussis
Andrew M. Fine, Ben Y. Reis, Lise Nigrovic, Don
Goldmann, Karen L. Olson, Tracy LaPorte, Kenneth
D. Mandl December 4, 2008 International Society
for Disease Surveillance Raleigh, NC
2
Bordetella Pertussis

• Incidence
• Impact
• Difficult to diagnose early
• Delayed test results
• Outbreaks have temporal and geographic
  variability

3
Research Approach

• Develop model to adjust standard clinical risk
  with epidemiological context

Epidemiological Context
Standard Clinical Predictors
Contextualized Risk
4
Low risk
No recent cases
1 week of cough
Low to normal risk
5
Elevated contextualized risk
Recent of local cases exceeds threshold
1 week of cough
Normal to elevated risk
6
Weekly Proportion Positive
Pertussis season
7
Pertussis cases in Massachusetts

• 2001 2003

8
Objective

• To improve identification of pertussis cases by
  developing a decision model that incorporates
  data on recent, local population-level disease
  incidence

9
Methods

• Retrospective cohort study
• Patient population
• Under 12 months old
• Presented to a single urban tertiary care
  pediatric emergency department and were tested
  for pertussis by culture
• 2003-2007

10
Data Sources

• Clinical data
• Hospital electronic medical records
• Demographics, historical features, physical exam
  findings, laboratory data, treatment,
  disposition, outcome
• Public health surveillance data
• State laboratory query of all pertussis culture
  results 2003-07 (all ages)
• Date sent and test result

11
Definitions

• Pertussis positive or negative
• Based on culture results
• PCR, serology, DFA not used in definition

12
Clinical Data (443 infants)

• Demographics
• Age
• Gender
• Visit date

• Signs/ symptoms
• Cough duration
• Fever duration
• History of apnea
• History of post-tussive emesis
• History of cyanosis
• History of seizure
• History of pertussis contact

13
Surveillance Data

• 19,907 culture results (2003-07)
• Created aggregate variables for state data
• Total tests sent
• Number positives
• Proportion positive
• OVER range of time 1, 2, 3, 4 weeks prior to
  each study date
• Each date from Jan 1, 2003 Dec 31, 2007 was
  assigned aggregate surveillance variables

14
Weekly Proportion Positive
Pertussis season
15
Weekly Proportion Positive
Week of Sept 9. 2005 6/45 0.13
Pertussis season
16
Weekly Proportion Positive
Week of Sept 9. 2005 6/45 0.13
Pertussis season
Week of Nov 24, 2006 10/285 0.035
17
Weekly Proportion Positive
Week of Sept 9. 2005 6/45 0.13
Pertussis season
Week of Nov 24, 2006 10/285 0.035
Week of Jan 5, 2007 6/188 0.032
18
Two Decision Models

• Clinical model
• -- Demographics and clinical data
• Contextualized model
• -- All clinical and PH surveillance variables

19
Statistical Methods

• All 443 infants were used to derive models with
  bootstrap validation
• Univariate analysis to determine variables
  significantly associated with pertussis
• Multivariate logistic regression to identify
  independent predictors of pertussis

20
Performance Metrics

• Sensitivity, Specificity, PPV, NPV, AROC
• Best model defined as greatest specificity among
  those with highest sensitivity
• Also compared actual performance by clinical
  experts from Childrens Hospital Boston emergency
  department with the 2 models
• Percent correct classification (utilization or
  omission of antibiotics)

21
PertussisResults
n 443
n 19,908
22
Notable ResultsUnivariate analysis
23
Logistic RegressionClinical only model
24
Logistic RegressionContextualized model
25
Performance of Decision Models
p0.04
plt0.001
P0.02
26
Misclassification Tables (1)Missed Opportunities
27
Misclassification Tables (2)Negative cultures
but treated
28
Limitations

• Retrospective
• Single site
• Culture only
• Other potential predictors immunization, cough
  descriptors, lymphocytosis

29
Discussion

• Incorporating PH surveillance data improved
  ability of decision model to correctly classify
  patients with and without pertussis
• Supports idea of bidirectional exchange between
  clinicians and public health
• Supports further exploration of methods for
  integrating large PH data sets with rich clinical
  data sets to improve decision making and PH

30
Idealized Scenario

• Jerry, a 4 mo M, presents to the ED with 2 days
  of cough and runny nose. Initially, the
  clinician thinks Jerry has a common cold. Upon
  opening the electronic chart, his doctor is
  presented with Jerrys chief complaint and
  standard VS, but based on recent epidemiological
  trends, he also is presented with a message
  there has been an increased incidence of
  pertussis in Jerrys county over the past month.
  Based on awareness of recent epidemiological
  trends in the area, the physician now considers
  obtaining pertussis tests, initiating antibiotics
  on the patient and close contacts, and reporting
  to public health.

31
Acknowledgements

• Mentor Ken Mandl
• Funder Centers for Disease Control (K01)
• Childrens Hospital Boston - Ben Reis, John
  Brownstein, Lise Nigrovic, Karen Olson, Don
  Goldmann
• MA DPH - Jim Daniel, Tracy Laporte, Gillian
  Haney, Linda Han, Stephanie Schauer, Susan Lett