Risk stratification in adult cardiac surgery

1
Risk stratification in adult cardiac surgery

• Bruce E. Keogh
• University Hospital Birmingham

2
The importance of comparative data quality in
life events

• Television
• Car
• House
• Spouse
• Medical treatment

3
What is healthcare data used for?

• Policy planning
• Performance assessment
• Health Authorities
• Hospitals
• Specialties
• Consultants
• Governance
• Research
• Audit
• Public and patient information

4
Why risk stratify?

• Prerequisite for effective comparisons between
• Countries
• Hospitals
• Individual clinicians

5
Approach to risk stratification

• To establish baselines for
• Patient variation

6
Approach to risk stratification

• Depends on what the data is to be used for
• Outcomes
• Death on waiting list
• Death in post-operative period
• Long term survival
• Processes
• Treatment variation
• Institutional process

7
Release of surgical data
Secretary of States Ministerial Response to the
Bristol Royal Infirmary Inquiry - 18th July, 2001
For data on surgical outcomes to be published,
of course, it needs to be robust, rigorous and
risk-adjusted. That will take inevitably time.
The report does recommend publication to give
both NHS staff and the public accurate
information. It recommends the establishment of
a new independent Office for Information on
Healthcare Performance within the Commission for
Health Improvement to co-ordinate the collection
and publication of data.
8
The first step data quality

• A count of patients and procedures
• A patient procedure map

9
Small errors are magnified
Treatment Outcome
Performance Measure
10
How good is our corporate data?
11
Data validation and presentationNuffield Trust
Rand Organisation study
12
Missing EuroSCORE data
13
The second step clear dataset

• Clinically relevant, defined dataset
• Adequate contemporary risk stratification
• Surveillance
• Anticipates change

14
McNamaras Fallacy

• US secretary of defense 1961-8
• Do you make what is measurable important, or
• Do you make what is important measurable ?

15
The National Adult Cardiac Surgical Database
Minimum Dataset

• Patient Data
• Demographics
• Cardiac history
• Co-morbidities
• Preoperative investigations
• Preoperative support

• Operative Data
• Operative priority
• Procedure data
• Training data
• Outcome Data
• Complications
• Mortality

Harmonised with NSF, cardiology and the NHS data
dictionary
16
Changing age spectrum for isolated CABG patients
100
80
gt75 years
71-75 years
60
66-70 years
61-65 years
Percentage of patients
56-60 years
40
lt56 years
Unconfirmed
20
0
1994
1995
1996
1997
1998
1999
2000
2001
Financial year ending
17
Mortality for isolated CABG by age (n22,487)
18
(No Transcript)
19
(No Transcript)
20
Age and gender distributions for isolated CABG
patients (n97,714)
21
Combining variables
22
The increasing incidence of diabetes in coronary
patients
20
Diet controlled
15
Oral therapy
10
Percentage of patients
Insulin
Any diabetes
5
0
1994
1995
1996
1997
1998
1999
2000
2001
Financial year
23
Confounding factors in diabetics

• More women with ethnic specificity
• More triple vessel disease with worse angina
• More diffuse in nature
• More previous MIs and a Lower ejection fraction
• More congestive heart failure
• More hypertension
• More vascular disease
• More chronic renal failure

J TCVS 198385264-71 Circulation 199184(S
III)275-84 Am J. Cariol 1998817-11 Ann Thor
Surg 1999671045-52 JACC 2000351122-9 JACC
2000351116-21
24
Risk Stratification Systems

• Some basic variables
• Type of operation
• Age, Gender, Re-operation, Urgency
• Cardiac function, recent MI
• Comorbidities diabetes, hypertension, vascular
  disease etc
• Score allocated to each variable
• Simple additive systems
• Parsonnet Score, EuroSCORE
• Complex statistical systems
• Logistic regression, Bayesian modelling

25
How good are risk stratification systems?
(n26,842)
26
Risk Adjusted DisplaysCoronary Valve Surgery,
UHB, 1999-2000
27
Receiver Operating Characteristic curve for the
complex CABG Bayes score applied to 2001 data
100
80
ROC area0.778 (n22,531)
60
Percentage true positives
40
20
0
0
20
40
60
80
100
Percentage false positives
28
Is more data better?
29
Evolution of Risk Factor Influence
1980's
2000
Odds ratio
0
1
2
3
4
5
6
7
8
30
Is more data better?

• More data points give better predictive and
  comparative power
• Data completeness falls off
• Validation slightly less robust
• Different statistical methodologies required

31
Calibration plot Complex 9-factor Bayes score
for isolated CABG re-trained on the on data from
the financial years 1998-1999Financial years
1998-1998 (n33,392)
25
20
15
Percentage mortality
Observed
10
Predicted
5
0
0-1
1-2
2-3
3-5
5-10
gt10
All
Risk groupings
32
Factors in high risk avoidanceThe immeasurable
33
Conclusions

• Some risk stratification is possible
• Comparative pointers
• Personal monitoring systems
• Perfect risk stratification is not possible

34
Why is Validation Necessary?
Risk factor variation in New York Before and
after report cards
New York DoH spends 3 years validating data
before release
35
EuroSCORE groupings for isolated CABG patients by
Contributing Centre for financial years 1997-00
(n26,840)
36
Validation Key Issues

• Professionally credible
• Specialty input
• Publicly credible
• Independent
• Professional protocol

37
Death difficult to validate

• Small number of mis-coded outcomes makes a
  difference
• Mortuary records poor
• Hand written ledgers
• Not computerised
• Not linked to PAS
• ONS takes time

38
Kappa coefficients for submitted vs.
Re-abstracted data elements of Parsonnet and
Euroscore
39
U.K. Cardiac Surgical RegisterIsolated coronary
surgery, 1977-2001
40
Patient Orientated National Databases
41
Data uses Performance Assessment Framework

• Health Improvement
• Fair Access to Services
• Effective Delivery of Healthcare
• Efficiency
• Patient and Carer Experience
• Health Outcomes of NHS Care

42
The Parsonnet Scoring System
Female Age Morbid obesity Diabetes Hypertension
LV dysfunction Renal failure Failed
intervention IABP Catastrophic state Other rare
circumstances
43
The EuroSCORE System