What EHRs Can Deliver that Randomized Clinical Trials Cannot

1
What EHRs Can Deliver that Randomized Clinical
Trials Cannot

• Retrospective studies with long-term follow-up
• Robert W. Grundmeier, MD
• July 13, 2009

2
Disclosures

• No conflicts of interest
• No off-label uses of commercial products will be
  discussed

3
Overview

• Re-use of existing clinical data in electronic
  health records for research
• The potential and challenges
• Past experience with successful retrospective
  cohort studies in the EHR
• Urinary tract infections
• Asthma
• The path ahead

4
Before Paper-Based Poetry
Fabricated Chart...Based on a true story
5
After Detailed Semi-Structured EHR Template
6
The PotentialRich and Fresh Data

• Thousands of repeated observations recorded for
  each potential subject over time
• Longitudinal health problem diagnoses
• Billing diagnoses
• Vital signs and measurements
• Prescriptions
• Immunizations
• Structured preventive health visits
• Laboratory and radiology data
• Procedures
• And many more!

7
The PotentialLarge Volume of Data

• 7 Years of data
• 9 Subspecialty centers
• 9 Subspecialty divisions
• 32 Primary care sites
• 300,000 Patients
• 2,500,000 Visits
• 65,000,000 Observations

8
Case Study 1

• Antibiotics for UTI prophylaxis
• (Conway PH. JAMA 2007)
• Pediatric Asthma Hospitalizations and the Quality
  of Ambulatory Care

9
Urinary Tract Infection (UTI) Study Cohort

• Almost 75,000 subjects
• 30 practices
• 5 years of EHR data
• Urine culture results from 3 laboratories
• Hospital and specialty center radiology data
• One Robert Wood Johnson Fellow

10
Urinary Tract Infection (UTI) Study Findings

• 12 annual incidence of recurrent UTI in children
  with an initial UTI
• Significantly higher rates in children with high
  grades of vesico-ureteral reflux
• Antimicrobial prophylaxis
• Did not change the rate of recurrent UTI
• Increased prevalence of resistant organisms in
  recurrent UTI from 53 to 90!

11
Urinary Tract Infection (UTI) Study Challenges

• Diagnosis codes only had moderate agreement
  with culture results
• Kappa 0.46
• Interpreting urine culture data required some
  natural language processing
• Validation proved this approach superior to using
  diagnosis

Cx Cx - Total
Dx 1,401 1,629 3,030
Dx- 738 12,536 13,274
Total 2,139 14,165 16,304
These kids really had a UTI
12
Urinary Tract Infection (UTI) Study Challenges

• Uncertainty about whether or not we had
  accurately identified the first UTI
• Considered using a birth cohort with complete
  data in EHR
• Instead chose to review all paper charts for
  patients with UTI (N775)
• 91 cases excluded due to documentation of prior
  UTI before EHR implementation
• Only 1 case considered a false positive

13
Case Study 2

• Antibiotics for UTI prophylaxis
• (Conway PH. JAMA 2007)
• Pediatric Asthma Hospitalizations and the Quality
  of Ambulatory Care

14
Asthma Study

• Questions
• Does quality of asthma care affect
  hospitalization rate?
• Are there disparities in asthma healthcare?
• Methods
• Almost 6,000 subjects from 5 practices
• 5 years of EHR data
• 24 independent variables
• 1 outcome (hospitalization)
• One AHRQ Contract

15
Asthma Study Preliminary Results
16
Univariate Predictors of Asthma Hospitalization

• Age lt 4 years
• .128 vs .063 hospitalizations per subject per
  month
• Moderate to severe persistent asthma
• .075 vs .044 hospitalizations per subject per
  month
• African American Race
• .072 vs .055 hospitalizations per subject per
  month
• Public Insurance
• .073 vs .065 hospitalizations per subject per
  month

17
Asthma Study Challenges Unmeasured Attributes

• Marginally adequate socioeconomic status (SES)
  markers for retrospective studies
• Public vs. private insurance is about as good as
  it gets
• Geocoding may help
• Median census tract income
• Housing type

18
Bad Luck Simultaneous QI Efforts Inseparable
19
Asthma Misclassification across time and space

• Common conditions are coded commonly, and
  reasonably well
• 57,820 Patients billed for asthma care
• 53,824 Patients with asthma on problem list
• 54,993 Patients with at least 2 albuterol
  prescriptions
• This is EXCELLENT correlation

20
Persistent Asthma

• What about persistent asthma?
• 16,949 Patients billed for persistent asthma
• 11,943 With persistent asthma as a problem
• But
• 23,673 Patients with at least 2 inhaled
  corticosteroid prescriptions which implies
  persistent asthma
• And
• Only 3,553 With persistent symptoms based on
  questionnaire
• Huh?

21
Non-Random Misclassification By Care Location

• It is OK to compare organizations using their
  electronic data because everyone has the same
  problems with their data the playing field is
  level
• Anonymous (Hospital Executive)
• Oh, really? Svetlana (CBMi Data Analyst)

Persistent Asthma PC KF Allergy TOTAL
Encounter Dx 8,087 8,167 754 16,949
Problem List 6,399 5,229 1,038 11,943
Inhaled Steroid 7,073 11,439 3,862 23,673
Persistent Symptoms 2,461 843 261 3,553
22
Non-Random Misclassification By Care Location
(Mistake in Query)

• Svetlana And do you really think that all the
  players will write their queries correctly?

Persistent Asthma PC KF Allergy TOTAL
Encounter Dx 8,087 8,167 754 16,949
Problem List 6,399 5,229 1,038 11,943
Inhaled Steroid 7,458 12,754 4,003 23,673
Persistent Symptoms 2,461 843 261 3,553
WRONG!
23
Non-Random Misclassification Over Time

• And, the playing field changes over time
• In 2004 one could have been lulled into a false
  sense of security over the reliability of
  encounter or problem list data Actually, WE
  WERE!

Persistent Asthma 2004 2005 2006 2007
Encounter Dx 3,695 6,026 7,923 8,707
Problem List 3,188 4,866 6,508 7,689
Inhaled Steroid 3,696 6,237 10,067 13,980
Persistent Symptoms 0 0 982 2,450
24
Good News! Statistical Magic for Missing Data

• Asthma severity is correlated with many variables
  available in the EHR
• Frequency, type, and dose of preventive treatment
• Frequency of quick relief prescriptions
• Frequency of oral steroid prescriptions
• Hospitalizations
• We recently imputed severity for the 20 of our
  population that is unclassified
• The results were unbelievably accurate like magic

25
The Bottom Line

• Retrospective studies can and should be done with
  EHR data captured for routine care
• When data are suspicious or missing, look for
  corroborating evidence
• You dont know what you dont know, until you
  read the charts
• Find cohorts enriched in the disease, brew some
  strong coffee, and read!
• Pound the pavement, go to where data are
  collected

26
The Way Forward Improve Data Collection

• Must think about how to make the clinician want
  to use the new data capture tool
• We are doing a comprehensive decision support
  intervention regarding ear infections for this
  reason

27
Thank You