Early Evidence of the Leapfrog Groups Impact on Hospital Safety

1
Early Evidence of the Leapfrog Groups Impact on
Hospital Safety

• 2004 Academy Health Annual Research Meeting
• Dennis Scanlon
• Eric Ford
• Penn State University
• Jon Christianson
• University of Minnesota
• This project was supported by an Investigator
  Award in Health Policy Research from the Robert
  Wood Johnson Foundation

2
What is the Leapfrog Group and How is it
Organized?

• Up to 98,000 Americans die each year from
  preventable medical mistakes they experience
  during hospitalizations, according to the
  Institute of Medicine
• Medical errors are a leading cause of death in
  America -- there are more deaths in hospitals
  each year from preventable medical mistakes than
  there are from vehicle accidents, breast cancer,
  or AIDS.
• Leapfrog, a coalition of more than 150 public and
  private organizations that provide health care
  benefits, is a voluntary program aimed at
  mobilizing large purchasers to alert the
  healthcare industry that big leaps in patient
  safety and customer value will be recognized and
  rewarded with preferential use and other
  intensified market reinforcements.

3
What is the Leapfrog Group and How is it
Organized?

• Leapfrog works with medical experts to identify
  problems and propose solutions that will improve
  hospital systems. Leapfrog focuses on the quality
  of certain aspects of care relevant to urban area
  hospitals. Leapfrogs initial safety agenda
  included three areas
• Computerized Physician Drug Order Entry Systems
• Using Intensivists in hospital ICUs
• Evidence based referrals for seven conditions and
  procedures
• Hospitals are invited to complete Leapfrogs web
  survey and share information with their
  communities about their efforts to reduce
  preventable medical mistakes.
• Leapfrog has chosen a Regional Rollout strategy
  to advance its safety agenda in specific
  communities. To date, 24 RROs have been selected
  across four waves.

4
Objectives of the RRO Case Study Project

• Describe/Document RRO structure and activities
• Obtain feedback from various stakeholders
  belonging to or affected by the RROs
• Provide feedback to Leapfrog and the RRO leaders
• Evaluate the impact of the RRO strategy for
  accomplishing Leapfrogs objectives
• Realize the capabilities and limitations of
  voluntary collective health purchasing efforts
  more broadly

5
Study Design

• Multiple case studies where the RRO is the case
• Data sources
• Interviews with key stakeholders in each RRO
• Secondary documents (e.g., RRO application, news
  articles)
• Leapfrog hospital disclosure results
• Secondary data sources (e.g., Interstudy and AHA
  data)
• Developed (with peer review) five protocols for
  common key stakeholders
• RRO leader
• Hospital executive
• Health plan executive
• Employer/purchaser
• Benefits consultant

6
Study Design

• Logic Model developed to guide the analysis
  (Sofaer et al.)
• Selected 7 RROs and identified key stakeholders
  for interview
• 2 non Leapfrog RRO control sites to be added
• Generalizability
• 4 researchers conducted interviews - two in each
  market
• tape recorded and summary notes produced by each
  interviewer
• Analysis of qualitative and secondary data by
  team of five researchers using standard
  approaches
• Preliminary results provided to RRO leaders for
  feedback and accuracy verification
• Produce final reports and products

7
Leapfrog Regional Rollout Logic Evaluation Model

• Assumptions
• The Institute of Medicines Errors and
  Quality reports has opened a window of
  opportunity to focus on health care safety and
  quality in the United States.
• Private and public sector purchasers can be
  catalysts for increased emphasis on patient
  safety by identifying a small number of safety
  leaps that evidence suggests will reduce the
  number of errors and deaths occurring in
  hospitals. Purchasers will also shift their
  focus from a pure dollar perspective to a value
  perspective.
• The Leapfrog Group is the formal entity that will
  organize purchasing activity around patient
  safety issues. The Leapfrog group will do this
  by identifying the relevant safety leaps and by
  enlisting and supporting purchasers in their
  efforts to influence/change hospital behaviors
  and decisions.
• Health care delivery is local, therefore a
  successful national safety initiative must
  recognize the localized nature of hospital care
  and health services delivery.
• The Leapfrog Group is intended to be a short-term
  rather than perpetual organization, with
  operations ceasing once patient safety has
  received significant attention by relevant
  stakeholders.

• Leapfrog Central Activities
• Determines the patient safety leaps based on
  expert advice and scientific evidence.
• Translates and communicates the leaps to
  purchasers and hospitals by providing information
  and technical assistance/support.
• Develops (in conjunction with Medstat) a hospital
  survey to measure hospital progress on the
  recommended leaps.
• Organizes purchasing power by recruiting
  individual public and private sector purchasers
  and selecting regions and leaders willing to
  promote the Leapfrog standards in local
  communities.
• Engages various stakeholder groups (e.g.,
  hospitals, employers, public sector purchasers,
  health plans, consumers, regulators) in feedback
  and discussion regarding patient safety generally
  and the chosen leaps specifically.
• Advocates for patient safety nationally through
  media exposure and by developing a reputation as
  the authoritative watchdog on hospital patient
  safety.

• RRO Mission
• Agree to Leapfrogs purchasing principles and
  accept the role as Leapfrogs agent in the
  local market.
• Develop a work plan for Leapfrog activities
  including goals and objectives, staffing,
  resources and a timeline.
• Choose the approach (e.g., punitive or
  collaborative) to engaging and influencing
  hospitals behavior.
• Identify and garner local support in the
  purchasing community for the Leapfrog Group
  nationally, and for the local Leapfrog initiative.

• RRO Process Activities
• Hospital completion of Leapfrog survey indicating
  status regarding Leaps
• Number and participation of local employers
  recruited to support RRO effort
• Tangible and significant health plan support for
  RRO efforts
• Public announcement of Leapfrog imitative in
  region.

• Final Outcomes (Goals)
• Medical errors in community hospitals are reduced
  and patient risk of being harmed by adverse
  hospital events declines
• Hospital costs become more efficient as cost
  savings from error reduction exceed costs of
  implementing the leaps.

• Intermediate Outcomes
• Hospitals
• Increased number of hospitals adopt CPOE
• Increased number of hospitals use intensivists
• Fewer EHR surgeries performed at low volume
  hospitals
• Hospital admissions increases at compliant
  hospitals and declines at non-compliant hospitals
• Employers
• Improved employer understanding of patient safety
  and medical errors, and awareness of variation
  across local hospitals in performance regarding
  patient safety
• Employers provide financial incentives that
  reward use of Leapfrog compliant hospitals
• Employers contract with health plans that support
  Leapfrog standards
• Employers and purchasers educating their
  employees on hospital safety.
• Health Plans
• Health plans develop hospital contracting
  incentives linked to Leapfrog leap compliance.

8
Leapfrog Regional Rollout Logic Evaluation Model

• Assumptions
• The Institute of Medicines Errors and
  Quality reports has opened a window of
  opportunity to focus on health care safety and
  quality in the United States.
• Private and public sector purchasers can be
  catalysts for increased emphasis on patient
  safety by identifying a small number of safety
  leaps that evidence suggests will reduce the
  number of errors and deaths occurring in
  hospitals. Purchasers will also shift their
  focus from a pure dollar perspective to a value
  perspective.
• The Leapfrog Group is the formal entity that will
  organize purchasing activity around patient
  safety issues. The Leapfrog group will do this
  by identifying the relevant safety leaps and by
  enlisting and supporting purchasers in their
  efforts to influence/change hospital behaviors
  and decisions.
• Health care delivery is local, therefore a
  successful national safety initiative must
  recognize the localized nature of hospital care
  and health services delivery.
• The Leapfrog Group is intended to be a short-term
  rather than perpetual organization, with
  operations ceasing once patient safety has
  received significant attention by relevant
  stakeholders.

• Leapfrog Central Activities
• Determines the patient safety leaps based on
  expert advice and scientific evidence.
• Translates and communicates the leaps to
  purchasers and hospitals by providing information
  and technical assistance/support.
• Develops (in conjunction with Medstat) a hospital
  survey to measure hospital progress on the
  recommended leaps.
• Organizes purchasing power by recruiting
  individual public and private sector purchasers
  and selecting regions and leaders willing to
  promote the Leapfrog standards in local
  communities.
• Engages various stakeholder groups (e.g.,
  hospitals, employers, public sector purchasers,
  health plans, consumers, regulators) in feedback
  and discussion regarding patient safety generally
  and the chosen leaps specifically.
• Advocates for patient safety nationally through
  media exposure and by developing a reputation as
  the authoritative watchdog on hospital patient
  safety.

9
Leapfrog Regional Rollout Logic Evaluation Model

• RRO Mission
• Agree to Leapfrogs purchasing principles and
  accept the role as Leapfrogs agent in the
  local market.
• Develop a work plan for Leapfrog activities
  including goals and objectives, staffing,
  resources and a timeline.
• Choose the approach (e.g., punitive or
  collaborative) to engaging and influencing
  hospitals behavior.
• Identify and garner local support in the
  purchasing community for the Leapfrog Group
  nationally, and for the local Leapfrog initiative.

• RRO Process Activities
• Hospital completion of Leapfrog survey indicating
  status regarding Leaps
• Number and participation of local employers
  recruited to support RRO effort
• Tangible and significant health plan support for
  RRO efforts
• Public announcement of Leapfrog imitative in
  region.

10
Leapfrog Regional Rollout Logic Evaluation Model

• Final Outcomes (Goals)
• Medical errors in community hospitals are reduced
  and patient risk of being harmed by adverse
  hospital events declines
• Hospital costs become more efficient as cost
  savings from error reduction exceed costs of
  implementing the leaps.

• Intermediate Outcomes
• Hospitals
• Increased number of hospitals adopt CPOE
• Increased number of hospitals use intensivists
• Fewer EHR surgeries performed at low volume
  hospitals
• Hospital admissions increases at compliant
  hospitals and declines at non-compliant hospitals
• Employers
• Improved employer understanding of patient safety
  and medical errors, and awareness of variation
  across local hospitals in performance regarding
  patient safety
• Employers provide financial incentives that
  reward use of Leapfrog compliant hospitals
• Employers contract with health plans that support
  Leapfrog standards
• Employers and purchasers educating their
  employees on hospital safety.
• Health Plans
• Health plans develop hospital contracting
  incentives linked to Leapfrog leap compliance.

11
RRO Case Study Sample
12
Interviews Conducted( multiple respondents in
some interviews)
13
Atlanta Market Factors
Sources AHA Guide, 2002. Atlanta Business
Chronicle, 2003. Atlanta Business Journal,
2002. Does not include newborns.
Includes only Leapfrog urban and targeted
hospitals
14
Rochester RRO Organization

• 12 Initiatives
• Community-wide clinical guidelines
• Patient Safety
• Healthcare worker shortage
• Monroe County Health Action
• End-of-life / Palliative care
• LEAPFROG
• Addressing Chronic Diseases
• Efficient and Effective Use of Resources
• Increasing Administrative Efficiency
• Increasing Access to Quality Services
• Benchmarking and Reporting Performance

Rochester Health Commission
Twelve Initiatives
Ed Black, MD / Cathy Kausch Excellus RRO Lead

• Community-wide advisory committee
• Key business groups
• Local employers
• Monroe County Medical Society

• Two major health insurers
• Three large urban hospital systems
• Smaller rural hospitals

New Leaps Subcommittee
15
How Should Success be Measured???

• Increased collaboration among hospitals and
  payors in the community
• Awareness and assessment by various stakeholders
• of targeted hospitals that disclose to the
  Leapfrog survey
• of hospitals compliant on the various leaps
• of hospitals making real progress towards leap
  adoption
• of admissions to hospitals meeting relevant
  leaps
• Degree to which purchasers/plans incent use of
  Leapfrog compliant hospitals

16
Hospital Comments about the Leapfrog Group
(examples)

• I have seen no appreciable change in the
  community as a whole regarding Leapfrog, but our
  hospital will purchase a CPOE system at least in
  part due to the impetus of Leapfrog
• Leapfrog has really helped getting some of the
  leaders minds at least directed towards quality
  as an important issue to payers
• Its instigated change in the provide community
  vis-à-vis physician order entry and increased
  focus on outcomes. It hasnt really changed
  purchaser behaviors, but Im not sure it was
  designed to.
• I applaud the payers demanding safety, but I
  still dont understand how they picked the leaps
  (e.g., esophageal surgery volume) relative to
  other viable candidates (e.g., smart pumps)

17
Hospital Comment about the RROs

• If their (the RRO) true goal is to improve
  quality, they did get peoples attention. They
  also caused hospitals to form several committees
  that are sharing information. The most
  improvement I think it has spawned relates to ICU
  care. I believe they are measuring important
  things internally and seeing improvement in many
  hospitals. Of course, these measures are not
  part of Leapfrog, because they (Leapfrog) are not
  measuring things that are important relative to
  patient care, just what they can easily promote
  and reference some literature with. Interesting
  though, to fight where Leapfrog wanted to go, the
  hospitals and doctors have gotten together,
  looked at articles, and are busy trying on their
  own to improve quality. Frankly, I think going
  at this directly would have been more successful,
  but will hand it to them (the RRO and LF), they
  have found a way to hype their effort, and it has
  helped push some people along.

18
Hospital Survey Reporting by RRO
Source The Leapfrog Group, January 2004
19
Dallas Ft. Worth Evidence Based Hospital
Referral. 57 Hospitals Targeted 36
Hospitals Responded
20
In your view, what has been the impact of the
Leapfrog safety agenda in your market? Please
mark the answer that best characterizes your view.
21
RRO Performance Ratings By Location
22
Summary (preliminary)

• Leapfrog has captured hospitals attention but
  impact on final outcomes is hard to discern
  (still to early to tell)
• Different response to the three leaps
• Lack of broad purchaser involvement in many
  markets.
• Activity led by small number of individual
  corporate champions or coalitions
• Small share of hospital admissions represented by
  RRO employers
• RROs captured by hospitals in some markets?
• Strict adherence to leaps vs. local development
• Relationship to other patient safety efforts
  (crowding out or being crowded out)
• Rural hospital concerns
• Health plan response
• Sentinel events

23
Future Work and Caveats

• Add control sites and additional RRO sites to
  test hypotheses
• Assess impact with expanded leaps
• Quantitative analysis of hospital reporting over
  time
• A true assessment will take time!