Ontario Preventable Death Study Investigators Veena Guru, MD Jack V' Tu, MD PhD Edward Etchells MD M

1
Ontario Preventable Death Study
InvestigatorsVeena Guru, MD Jack V. Tu, MD PhD
Edward Etchells MD MSc Geoffrey M. Anderson, MD
PhD C. David Naylor, MD DPhil Richard J. Novick
MD MSc Christopher M. Feindel, MD MSc Fraser D.
Rubens MD MSc Kevin Teoh MD MSc Andrew
Hamilton, MD Daniel Bonneau MD Charles Cutrara,
MD Peter C. Austin, PhD Stephen E. Fremes, MD
MSc Avdesh Mathur, MD MSPublished in
Circulation. 2008 1172969-2976. (June 2008)

• Relationship Between Preventability of Death
  After Coronary Artery Bypass Graft Surgery and
• All-Cause Risk-Adjusted Mortality Rates
• www.qualitycabg.org

2
Educational Objectives

• Identify the proportion of preventable deaths
  after CABG surgery
• Identify limitations to the reporting of
  all-cause mortality as a measure of quality of
  care

3
Background

• Hospital report cards are a popular tool for
  promoting improvements in quality of care
• Reports commonly monitor outcomes such as
  in-hospital mortality that are readily measured,
  and can be compared at a physician or
  institutional level using risk adjustment
• Providers that perform differently from the
  average performance of a region may be identified
  as statistical outliers

4
Background

• Two basic problems with the paradigm of outcomes
  reports
• It is never clear how many deaths associated with
  an outlying provider are preventable
• Compared to a handful of outlier providers, the
  total number of preventable deaths may actually
  be much higher if cumulated across the majority
  of non-outlier hospitals or providers
• Thus quality of care is likely best enhanced by
  dealing with all institutions rather than
  isolating and addressing outliers

5
Hospital Specific Mortality Trends in Ontario
Guru V et al. Public versus private institutional
performance reporting. Am Heart J. 2006
Sep152(3)573-8.
6
How is poor quality measured through outcomes?

• Outcome
• Preventable Unpreventable
• Related mainly to Related mainly to
• delivery of care patient factors
• Must judge if an outcome is PREVENTABLE
• ie could have been avoided if optimal care had
  been delivered.

7
Hypothesis

• A measure such as preventable mortality should
  have a stronger link than all cause in-hospital
  mortality in measuring institutional level
  quality performance.

8
Methods Target Population

• Retrospective analysis of 40 randomly selected,
  in-hospital deaths following isolated CABG
  surgery at each of the nine cardiac surgery
  institutions between April 2000March 2002 in
  Ontario (CCN and CIHR used to identify patients).
  
• Additional cases were reviewed from fiscal years
  1998, 1999, 2002, and 2003 for hospitals with
  small volumes or with new cardiac surgery
  programs.

9
Methods Chart Review

• Hospital charts of each in-hospital death were
  reviewed by trained nurse abstractors for
  baseline demographic and clinical data using a
  standardized database
• The nurse-abstracted chart summary was reviewed
  by two cardiac surgeons who were blinded to the
  identity of the patient, attending surgeon, and
  hospital

10
Methods Surgeon Reviewers

• Surgeon reviewers were experienced staff surgeons
  and/or division chiefs
• Each surgeon reviewer was trained to apply a
  standardized implicit tool to identify
  preventable deaths
• Most surgeons completed between 70-90 cases
  during the study

11
Methods Review Tool

• The implicit review tool was modified in previous
  adverse event audits (Canadian Adverse Event
  Study, UK Medical Review, US Medical Review Form,
  NNE Mode of Death Form)
• Preventability was scored on a 7 point LIkert
  scale, and then redefined as a binary outcome
  (preventable, not preventable)
• Surgeon reviewers were aware of the interpretive
  definitions in advance of reviewing
• Inter-rater reliability was assessed with the
  kappa statistic
• Correlation between all-cause risk-adjusted
  mortality and preventable deaths identified at
  each hospital using Spearman ranking of
  correlation coefficient

12
Methods Preventability Rating

• Preventability was judged from facts available in
  the chart, and reviewers were instructed to judge
  issues related to care once the surgeon had
  accepted the patient termed decision to operate
• Reviewers were advised when rating
  preventability, they should apply the concept of
  Optimal Care, what one would want for oneself,
  in accordance with the best evidence available at
  the date of the particular patient admission

13
Methods Preventability Rating

• Reviewers were required to provide a list of
  quality of care problems that contributed to
  deaths judged as preventable and potential
  solutions
• For cases where there was a disagreement on the
  decision to operate and/or whether the death
  was preventable, a third surgeon reviewer
  reviewed both primary physician reviews as well
  as the original chart to provide a final judgment

14
Results

• 347 deaths (40 per hospital for 8 hospitals, 27
  deaths at a new cardiac surgery program) were
  each reviewed by 2 of 9 physician reviewers to
  give a total of 694 single reviews.

15

• Top Five Modes of Death Post CABG
• Often more than 1 preventable cause
• Sudden Death/Ventricular Arrhythmia 28
• Cardiac Ischemia/Infarction/Failure 20
• Ischemic Bowel/Hepatic Failure/MOF 21
• Coma/Stroke 11
• Respiratory Disease 7

16
Figure Breakdown of Implicit Surgeon Review of
Isolated CABG Deaths (n347)
32 (111/347) of deaths deemed preventable
61 ICU Problem Related To Death
86 Operative Problem Related To Death
15 Ward Problem Related To Death
Diagnosis of life threatening event 35 Response
of nurse and/or physician 28 Patient monitoring
17 Decision or timing of reoperation
14 Medication dosage and administration
10 Communication 10
Operative judgment 46 Detection of graft patency
43 Completeness of revascularization
27 Communication 27 Separation from
cardiopulmonary bypass 18 Completion of
anastomosis 18 Assessment of appropriateness of
graft for particular target 18 Assessment of
hemodynamic stability for off-pump surgery
15 Initiation and maintenance of cardiopulmonary
bypass 14 Assessment of aorta 12 Selection and
preparation of bypass graft 11 Inspection for
hemostasis 10
Diagnosis of life threatening event 10 Response
of a nurse and/or physician 8 Patient monitoring
8 Communication 4 Obtaining help from another
member of healthcare team 4
17
Quality of care problems identified

• Review Section Double Review Single
  Review Kappa, Agreement Post Third Review
• Inappropriate timing of surgery
• 4 (14) 15 (52) 0.26 (0.11-0.40), 85 8
  (27)
• A reasonable surgeon would not repeat surgery
• 2 (7) 12 (42) 0.17 (0.01-0.33), 89 5
  (17)
• 3. There was significant deviation in
  perioperative management from the accepted norm
• 32 (112) 42 (146) 0.16 (0.05-0.26),
  58 ----
• 4. There was evidence of preventability
• 15 (52) 33 (115) 0.24 (0.13-0.34), 67 32
  (111)

18
Figure Correlation of risk-adjusted all-cause
hospital mortality rates versus preventable
death judgments through implicit review.
Spearman Correlation Coefficient -0.42
(p-value0.26)
Preventable Death Proportion
Risk-adjusted all cause mortality rate is
calculated using the observed all cause mortality
divided by the predicted mortality rate for a
particular hospital multiplied by the average
crude provincial mortality rate for isolated CABG
surgery.
19
General Areas of Improvement Identified

• Quality Assurance (38 of Preventable Deaths)
• eg ensure better intraop hemostais.before
  leaving the OR
• eg doppler verification of graft flows
• Education (29 of Preventable Deaths)
• eg delayed recognition of sepsis two full days
  between onset of sternal drainage, suspicion of
  sepsis and definitive surgical treatment
• eg too much IV propanolol given to this patient
  with grade 3 LVEF, a tight left main lesion and
  signs of heart failure, leading to an asystolic
  cardiac arrest

20
General Areas of Improvement Identified

• Communication (23 of Preventable Deaths)
• eg complex cases require a thorough discussion
  and coordination strategy between the surgeon,
  the anesthetist, and perfusionist. All redo cases
  with patent grafts should have vascular access
  prior to sternotomy
• eg the decision to remove an intra aortic balloon
  pump should be clearly delineated and the
  discussion and consensus between the surgeon and
  intensivist documented
• Resources (24 of Preventable Deaths)
• eg patient needed to be placed on a left
  ventricular assist device
• eg invest resources to purchase flow probe systems

21
General Areas of Improvement Identified

• Retraining (29 of Preventable Deaths)
• eg the angiogram should be reviewed in a rounds
  format with experienced surgeons
• eg the technical aspects of the proximal
  construction (of grafts) should be reviewed in
  detail
• Credentialing (12 of Preventable Deaths)
• eg surgical unit needs to review conversion rates
  and focus expertise techniques for off-pump
  need to be reviewed and further documented on the
  operative record
• eg if aorta so severely calcified consider
  circulatory arrest for replacement of ascending
  aorta. Refer case to high risk centre

22
Table Preoperative patient predictors of
preventable death

• Patient Factor Coefficient (p-value) OR (95
  CI)
• Age (continuous variable) -0.027 (0.05) 0.97
  (0.95-1.00)
• Female Sex 0.38 (0.12) 1.47 (0.90-2.39)
• Left main coronary disease -0.39 (0.12) 0.67
  (0.41-1.11)
• Three vessel coronary disease -0.31 (0.27) 0.73
  (0.42-1.27)
• Emergent status (ORlt24hrs) -0.98 (0.06) 0.38
  (0.13-1.04)
• Diabetes(Rx) -0.96 (0.0003) 0.38 (0.22-0.65)

23
Conclusions

• Despite a decade of mortality report card
  feedback and some measurable lowering of
  provincial CABG mortality rates there is still
  room for improvement in the processes of care
  with 32 of deaths judged preventable.
• Inappropriateness in the timing and decision to
  operate are not prevalent quality of care
  problems in Ontario

24
Conclusions

• In 74 there were deviations of care from
  accepted norm and best practices.
• Preventable mortality rates were not strongly
  correlated with all-cause risk-adjusted mortality
  rates.
• Preventable deaths were more commonly identified
  in those patients with lower predicted operative
  risk. One way to focus quality improvement
  efforts is to look most closely at deaths that
  are statistically unexpected, i.e. occurring
  among those who were expected to have
  uncomplicated post-operative courses and
  excellent outcomes.

25
Risk of Estimated In-hospital DeathVS
Preventability/Appropriateness
26

• IMPLICATIONS
• Our findings suggest that all-cause,
  risk-adjusted mortality statistics are unlikely
  to be either precise or accurate as tools for
  screening quality of surgical care at the
  individual or institutional level.
• Reliance on institutional outcome report cards
  alone may provide misleading impressions of the
  quality of hospital care, and paradoxically
  reduce the incentive for providers working in
  non-outlier institutions to address suboptimal
  care.

27
Additional Resources

• 1. Deadly Medical Errors Still Plague U.S.
  Report Shows 10-Year Effort to Curb
• Medical Errors Yields Few Results. (May
  19, 2009)
• http//www.webmd.com/news/20090519/deadly-medic
  al-errors-still-plague-us?page2
• 2. Publicly reported provider outcomes The
  concerns of cardiac surgeons in a single-payer
  system. V. Guru et al. Can. J. of Cardiology
  200925(1)33-38.
• 3. Ranking Hospitals on Surgical Quality Does
  Risk-Adjustment Always Matter? J.B. Dimick et
  alJ. American College of Surgeons Volume 207,
  No.3,Sept. 2008.
• 4. The public health hazards of risk avoidance
  associated with public reporting of risk-adjusted
  outcomes in coronary intervention. Resnic and
  Welt. J. AM. Coll. Cardio 2009 53831-833.
• 5. The Wisdom and Justice of Not Paying for
  Preventable Complications P. Pronovost et al.
  JAMA. 2008299(18)2197-2199.