Massachusetts Quality eMeasure Validation Study MQeVS

1
Massachusetts Quality e-Measure Validation
Study(MQeVS)

• Eric C. Schneider, MD, MSc
• Brigham and Womens Hospital
• Harvard School of Public Health

Sponsor AHRQ (R18 HS017048)
2
Performance Measurement Fundamental to
Improvement
Performance Visibility
Performance Rewards
Peers
Patients
Public
Purchasers
Performance Feedback
Payments Penalties
Market Share
Report Cards
P4P
3
Quality Measurement and Reporting 1997

• Limited public demand
• Few standardized quality measures
• Few organizations engaged
• Few physicians aware
• Public disclosure rare
• Few patients aware

4
2007 Dramatic Increase in Standardized Measures

• Standard-setting organizations
• National Quality Forum (NQF)
• Ambulatory Quality Alliance (AQA)
• Engagement of Medical Profession
• Physician Consortium for Performance Improvement
• AHRQs National Quality Measures Clearinghouse
  (www.qualitymeasures.ahrq.gov)
• Access - 22 measures
• Outcome - 204 measures
• Patient Experience - 298 measures
• Population Health - 33 measures
• Process - 636 measures
• Structure - 44 measures
• Use of Services - 33 measures

5
Implementation of Measurement and Reporting
Remains Controversial

• Measurement programs based on
• Administrative data
• Hybrid method (supplemental med record review)
• Patient/enrollee surveys
• Skepticism about validity of performance results
• Burdensome, costly data collection

6
Colorectal Cancer Screening Results by Method
Schneider et al, Under Review
7
Envisioning the EHR forPerformance Measurement

• Detailed, structured clinical data
• Unobtrusive data collection
• Performance data aggregated across care settings
  to enable sophisticated measures (e.g. care
  coordination, safety)
• Performance results at physician group rather
  than health plan level

Schneider et al, Enhancing performance
measurement NCQAs Roadmap for a Health
Information Framework. JAMA 19992821184
8
Studies of Performance Measurement Using EHR and
HIE

• Single institution
• Single care setting
• Single IT platform
• Limited number of performance measures
• Local, rather than national standards

9
Massachusetts e-Health Collaborative (MAeHC)

• 2004 Demonstration Project
• 50 million from Blue Cross Blue Shield of MA
• Universal EHR adoption in 3 MA communities
• Intra- and inter-community data exchange (HIE)
• Integrated performance measurement/reporting
• Massachusetts Health Quality Partners (MHQP)
• AQA ambulatory performance measure starter set
  (26 measures)
• Quality Data Warehouse
• Receives HIE data as needed to calculate measure
  results

10
Courtesy of Micky Tripathi
11
MA Quality e-Measure Validation Study (MQeVS)

• To compare a quality measurement method using
  structured, coded EHR data with
• 1) a hybrid method involving a combination of
  aggregated claims data and medical record review.
• 2) a claims-only method based on a novel
  database that aggregates claims data from
  commercial health plans and Medicare.

12
MQeVS Highlights

• Implementation
• MAeHC Pilot (www.maehc.org)
• MHQP (www.mhqp.org)
• Evaluation
• Community ambulatory practices similar to U.S.
• Broad clinical and research expertise among
  research partners
• Harvard School of Public Health
• MHQP
• Partners Healthcare
• Harvard Medical School
• Center for Survey Research (U Mass, Boston)

13
MQeVS Sample

• Two Specific Aims assure that study addresses
  broad populations and measures
• Aim 1 900 patients with EHR-HIE data, patient
  survey, medical record review, health plan
  administrative data
• Aim 2 All measure eligible patients with
  EHR-HIE data and health plan administrative data

14
Data Collection Protocol Privacy/Confidentiality
HSPH (PI) Analysis Team
Mass E-Health Collaborative
Med Record Extract (Aim 1)
8
Claims Data Extract (Aims 12)
EHR Quality Measure Extracts (Aims 1 2)
9
10
Med Record Review Team (Partners)
Medical Record Requests
6
Physician Offices
CSC/ QDW
Medical Record Copies
MHQP
Medical Record Consent Cohort (Study IDs)
5
Survey Data Extract (Aim 1)
7
3
De-identified Study Cohort List (Aim 1)
Opt-out Step
1
Pre-notification
EHR Data
Survey Team (CSR)
Study Consent, Survey, and Med Record Review
Consent
2
EHR VENDORS
4
Patients
Re-identified Study Cohort Pt Contact Info
Completed Surveys and Consents
15
Analysis Availability of Data for Measure
Components

• Five key steps comprise a quality measure
  calculation algorithm, determining whether a
  patient meets criteria
• 1. for inclusion in the preliminary denominator
• 2. to be excluded from the preliminary
  denominator
• 3. for membership in the final denominator (after
  exclusions)
• 4. for membership in the measure numerator
• 5. for selection for both the numerator and
  denominator

16
Quality Measures Deconstructing Data
NeedsEexclusion criteria Ddenominator
inclusion Nnumerator inclusion Varvaries
17
Data Adequacy Assessment

• (1) Data Present, Event Confirmed a lipid
  lowering medication is recorded in the patients
  data
• (2) Data Present, Event Not Confirmed there is
  no lipid lowering medication, but data indicate
  that the patient is taking other medications
• (3) Data Not Present there are no data regarding
  medications making it uncertain whether the
  patient is taking a lipid lowering medication.

18
Analysis
Where Availability through the EHR (ac) /
(abcd) 92 And Availability
through Hybrid method (ab) / (abcd) 98
19
Challenges

• Logistical
• HIE implementation
• Data sharing (privacy/confidentiality)
• Analytic
• Lack of a gold standard
• Complex correlation among data sources
• Identifying and interpreting missing data
• Small sample sizes for some measures

20
Opportunities

• NQF-endorsed national standard measure set
• Measure set creates need for broad range of data
  types
• Direct comparison of HIE to two widely-used
  measurement methods
• Deconstruction of measure components identifies
  data problems that may affect future performance
  measures

21
Crossing the Quality Chasm?