Optimal Organ Allocation Policies: An application of discrete event simulation

1
Optimal Organ Allocation PoliciesAn
application of discrete event simulation
University of Pittsburgh

• Mark S. Roberts, MD, MPP
• Associate Professor of Medicine, Health Policy
  and Management and Industrial Engineering
• Chief, Section of Decision Sciences and Clinical
  Systems Modeling
• University of Pittsburgh School of Medicine

2
The general problem of Organ Allocation

• Organs are a scare resource, and waiting lists
  are increasing
• The debate (in the US) surrounds
• Who gets transplanted? (or retransplanted)
• What determines selection priority and site?
• UNOS (Has changed allocation rules 3 times)
• HCFA (rules about success rates required for
  sites)
• What level of regional preference is appropriate?
• Organs to the sickest or to those who would
  benefit most?
• What are the appropriate methods to analyze this
  problem?
• As much a talk about the value of simulation as a
  talk about transplantation allocation

3
What is the Clinical and Policy Problem?

• There are two very distinct questions that can be
  asked regarding transplantation
• CLINICAL Question given a specific patient with
  a specific disease and a set of clinical
  characteristics, what is the optimal timing in
  the declining course of disease to transplant
  that specific patient?
• POLICY Question What set of selection, listing
  criteria and allocation rules maximize the
  utility of the limited supply of organs? (What is
  the optimal timing from the point of view of the
  society?)

4
Allocation of organs to Patients

• In the US, organs are allocated (in theory) with
  a balance of efficiency and fairness
• The United Network for Organ Sharing (UNOS) is
  responsible for implementing and setting
  allocation policy
• Current rules are essentially a combination of
  two concepts
• PRIORITIZATION where a patient falls in the
  waiting list
• ALLOCATION how many lists there are in the
  country
• For example, there are different lists for each
  region, and acute liver disease is treated
  differently from chronic liver disease

5
US Organ Procurement Organizations (OPOs)
OPOs are aggregated into 11 regions
6
Prioritization

• As changed several times over the past decade
• Prior to 2002
• 4 Status groups
• 1 (acute, fulminate liver failure)
• 2a (chronic liver failure, need ICU care for
  survival)
• 2b (chronic liver failure, need hospitalization)
• 3 (chronic liver failure, not in hospital)
• Several other status levels for special
  circumstances
• Status 7 (too sick at the moment)
• Order within status mainly determined by waiting
  time
• Allows for gaming the system

7
Prioritization (MELD score since 2002)

• Prioritization was changed to rank people based
  on level of illness (transplant the sickest
  first)
• Model for End-stage Liver Disease (MELD score)
• Predicts the probability of survival for the next
  three months
• Scaled to an integer between 6 (lowest
  probability of death) and 40 (highest probability
  of death)
• Status 1s are the same (fulminate, with p(death
  in 7 days) gt 50)
• The remainder are grouped by MELD, rank within
  MELD is where wait time, blood type compatibility
  matter

MELD Score 10(0.957 x ln(creatinine) 0. 378
x ln(bilirubin) 1.120 x ln(INR) 0.643)
8
Allocation Hierarchy OPO to Region to Nation
Status 1

• Livers are divided into two major groups
• STATUS 1 patients with acute liver failure with
  a LE of lt7 days
• CHRONIC patients, allocated by MELD Score (a
  statistical score representing probability of
  dying in 3 months) that varies between 6
  (healthiest) and 40 (sickest)

5
6
2
4
1
3
OPO
Region
Nation
MELD Score
9
Goal of overall modeling effort

• Goal is to build a model that represents the
  clinical natural history of ESLD and then
  superimpose selection, timing and allocation
  policies on top of that model
• Requires a clinically robust, detailed model of
  the progression and natural history of liver
  disease

Different rules will imply that different
patients receive organs at different times in
their disease post transplant success is a
function of clinical characteristics of the
recipient and the donor
Waiting list
Organ
10
ESLD Clinical Model Chronic Disease
COMPLICATIONS
Imagine there was a single marker of liver
function that could be tracked over time. As
liver function declines, various clinical events
begin to occur
SYMPTOM DEVELOPMENT
Natural History
"LIVER FUNCTION"
THERAPY REQUIRED
DEATH
TIME
CHRONIC DISEASE
11
Average Natural History
12
Effect of Natural HX on Transplant Success

• As liver disease progresses,the success of
  transplantation changes
• Increase operative death
• Decreased post-op survival

100
POST TRANSPLANT SURVIVAL
80
OPERATIVE MORTALITY
60
POST-TRANSPLANT SURVIVAL (YRS)
40
OPERATIVE MORTALITY
20
0
DECLINING LIVER FUNCTION (PROGRESSION OF DISEASE)
13
Post Transplant Survival by Severity of Disease

• If transplanted early, there is little operative
  death
• As disease progresses, operation carries higher
  mortality risk, and post TX survival declines
• Eventually patients become extremely high risk

14
Natural History and Post TX Survival

• Transplantation early may provide less post-tx
  survival than Nat Hx
• Transplanting too late may provide post tx
  survival that is to short

15
Markov Model Initial attempts to model
16
Optimal Timing asking the wrong question

• We (society) doesnt chose a time, we choose a
  strategy
• When different people are transplanted is a
  function of the system
• This question is much more relevant in living
  donor transplants
• You have seen the work by Oguzhan Alagoz, PhD (a
  former student) on optimizing this problem
• So, we wanted to look at the societal question
  what are the consequences of various allocation
  rules

17
Discrete Event Simulation

• Methodology directly applicable to the problem
• Can model the queues formed, and the other
  characteristics of the natural history, survival,
  etc.
• DES simulation allows for competition between
  resources
• DES models the specifics of the situation
• Actual number of people on the list
• Number of transplants
• Number on waiting list
• These are questions that CANNOT be addressed by
  RCTs or standard statistical methods

18
Discrete Event Simulation The Liver Transplant
Model

• Model individual patients presenting with liver
  disease
• Model individual organs generated by donors
• Model individual transplant centers
• Model pre and post-transplant survival
• Model natural history

19
Basic Model Structure
Patient Generator
Organ Generator
Model Outputs
User-defined Inputs
Survival Quality-Adjusted Survival Costs spent
on ESLD Number of deaths waiting Average waiting
time Number of wasted organs
Selection and Allocation Rules
Discrete Event Simulation Model
Survival Module
Disease Progression Module
Resource Use Module
Quality of Life Module
20
Patient Generator

• Disease (10 Groups)
• Gender
• Age
• Race
• Blood Type
• Laboratory values
• Bilirubin
• Creatinine
• PT
• Albumin

OPO1
Organ Procurement Organizations (OPOs) (which are
clustered into Regions)
21
Organ Generator

• Gender
• Age
• Race
• Blood Type

OPO1
Organ Procurement Organizations (OPOs)
22
Data Dependencies
23
Regional/Geographic Overlay
24
Model incorporates current regional preference
Status 1

• Model could arbitrarily change to any level of
  regional prioritization or not

5
6
2
4
1
3
OPO
Region
Nation
MELD Score
25
Allocation Mechanism Generic structure
26
UNOS Algorithm (1999-2002)
Ranked by points (ABO relative time on list)
Priority
In ICU LE lt 7 days .
1
1
2
2a
3
Regions 10 OPOs Status Levels 4
2b
4
3
NOT USED
NOT USED
27
MELD Algorithm (post 2002)
Priority
In ICU LE lt 7 days .
1
Ranked by points (relative time on list at that
score or worse)
MELD 40
MELD 39
MELD 38



MELD 6
28
Disease Progression Module

• Modeling natural history
• Discrete event simulation requires the ability to
  predict (quantitatively) the changes in clinical
  parameters over time
• Traditional statistical methods are not suited to
  do this concurrently
• where
• Data available is likely biased

X(t1) f(X(t)) or X(t?t) f(X(t),?t)
Clinical covariates of interest
X (x1x2,x3, xn)
29
Natural History Estimating Problem
Evaluation
Natural history according to NIDDK
Bilirubin
Transplant
Observed values of variable
Average Natural History
Time
30
Natural History Prior Simulation Efforts
In earlier simulation (ULAM), by Pritzker and
UNOS, the natural history model is directly tied
to the allocation/selection model
Status 1
Time 2
Status 2a
1 2a 2b 3
p13
1 2a 2b 3
p11
p12a
p12b
Status 2b
p2a3
p2a1
p2a2a
p2b2b
Time 1
p2b3
p2b1
p2b2a
p2b2b
Status 3
p33
p31
p32a
p32b
Cannot modify this to assess the effect of the
change to the MELD score, for example
31
Natural History Modeling

• Laboratory data does not come at regular
  intervals
• More dense when patient is sick (over-sampled)
• Less dense when patient is healthy
  (under-sampled)
• Actual laboratory data is interpolated using
  cubic splines

Bilirubin
t1
t2
t3
t4
t5
t6
t7
t8
t9
t10
t11
t12
t13
t14
t15
Observed bilirubin
Time
Cubic spline estimated bilirubin Estimated cubic
spline
32
Maintaining correlations in the data

• All of the laboratories are sampled at the same
  time, keeping relationships between laboratories
• Each persons laboratory history is decomposed
  into a series of overlapping triplets
• Each triplet characterizes a short time interval
  for that patient labs yesterday, today, and
  tomorrow

33
Natural History Stratification
Patients with Primary Biliary Cirrhosis
34
Natural History, Stratification
Five Disease Groups
Out of Hospital
DZ 1
DZ 2
DZ 3
DZ 4
DZ 5
In Hospital
DZ 1
DZ 2
DZ 3
DZ 4
DZ 5
In Intensive Care
DZ 1
DZ 2
DZ 3
DZ 4
DZ 5
35
Disease Progression Mechanism
1
t
-
1
t
t1
1.2
1.3
Creat
?
4.2
4.0
ALB
?
1.0
1.8
tBILI
?
15
18
PT
?
t
-
1
t
t1
30
45
ALT
?
0.9
1.0
1.2
Creat
A 42 year old male with hepatitis C
3.9
3.9
3.6
ALB
1.2
1.3
1.8
tBILI
17
18
21
PT
28
29
35
ALT
3
t
-
1
t
t1
Creat
1.2
1.3
1.2
Among all 40-50 old male patients with hepatitis
C, find one with similar laboratory profile
2
ALB
4.2
4.0
3.6
tBILI
1.0
1.8
1.8
PT
15
18
21
ALT
30
45
35
The similar patients time
t1
values
become the current patients
time t1
values.
36
Determining similarity

• Assessed TX surgeons, gastroenterologists
• Determined how different each lab had to be to be
  clinically important

37
Modeling Natural History
50 year old Hispanic female with Alcoholic Liver
disease
38
Modeling Natural History
50 year old Hispanic female with Alcoholic Liver
disease
39
Modeling Natural History
40
Modeling Natural History Results
Shown only for one group of ESLD diagnoses
(primary biliary cirrhosis, primary
sclerosing cholangitis, alcoholic liver disease,
and autoimmune disorders). Differences
between actual and simulated change are within
levels considered clinically
insignificant by clinical advisory group.
41
Modeling Natural History Results
Correlations between Clinical Covariates
42
Pre-transplant survival
1.0
0.9
0.8
Proportion Surviving
0.7
Actual
0.6
Model
p0.26
0.5
0
100
200
300
400
Days
43
Post Transplant Survival

• Estimated disease-specific post transplant
  survival curves from sample of 17,000
  transplants from UNOS 1991-1996 w/ follow-up to
  1999
• Cox proportional Hazards models
• Model transplants the patient at a given time,
  and knows the clinical covariate vector at that
  time
• age, gender, bilirubin, creatinine, PT, albumin,
  encephalopathy
• Model generates a covariate-adjusted cumulative
  hazard
• Hazard function is randomly samples to arrive at
  a specific survival time
• Re-estimated for Cox model predicting graft
  survival

44
Post Transplant Survival
45
Post Transplant Survival

• Cox proportional Hazard models by disease

Organ characteristics
If Patient Survival gt Graft Survival, patient is
RELISTED at time of graft failure If Patient
Survival lt Graft Survival, patient dies at
survival time
Patient Survival
Patient characteristics
Graft Survival
46
Cost Module

• Not yet implemented
• Extract of 2000 patients from UNOS matched to
  CMS claims
• Costs of care
• Pre-transplant/transplant/post-transplant
• Disease-specific, location (in hospital out of
  hospital) specific

47
Quality of Life module

• Prospective evaluation of patients awaiting
  transplant
• Formal Utility Assessments
• Standard Gamble
• Time Trade off
• Visual Analog Scale
• NIDDK Quality of Life Questionnaire
• Hoped to predict utility from QOL question
  responses
• Entered 130 patients

TTO .74 .54 .40 .84
SG .67 .41 .40 .64
Out of Hospital In Hospital In ICU Post Transplant
From literature
48
Patient Generator
Arbitrary priority scheme
removed from list

• Bilirubin
• Albumin

• Creatinine
• INR

Improved
Waiting List Removals
Alive on Waiting List
Quantitative Natural History
Too sick
Refused Transplant
of organs wasted
Unused
Arbitrary allocation rules
Organ Generator
Organ Match
Die While Waiting
Organ Failure
of organs transplanted
Post Transplant Graft Survival
died prior to transplant

• Distribution by
• Region/OPO
• Age
• Race
• Gender
• CMV
• ABO

Alive Post Transplant
Post Transplant Survival
Die Post Transplant
Post Transplant Patient Survival
Dead
49
Calibration and Validation
Variable of Transplants Waiting Deaths
while waiting Waiting Time (median)
1992 2614 2599 1903 1880 516 473 124 142
1993 2947 2946 2809 2548 567 514 175 193
1994 3129 3124 3838 3544 671 589 242 217
1995 3470 3460 5365 5072 835 754 345 316
1996 3567 3583 7203 6795 1000 919 n/a n/a
Model UNOS Model UNOS Model UNOS Model UNOS
50
Allocation Rule Predictions

• We have examined 4 alternative strategies
• Original UNOS ranking, local preference
• Original UNOS ranking, national List
• Current MELD ranking, local preference
• Current MELD ranking, national List
• Compare several outcome between multiple scenario
  runs under each set of conditions
• Use the model to develop (calculate) EMERGENT
  PROPERTIES
• these are properties that are measurable in real
  world but are calculated by the model, not used
  as inputs

51
Results
UNOS Regional 1622 3589 0.84 0.78 252 9.51 6.65
MELD Regional 1670 3612 0.84 0.78 181 9.32 6.67

UNOS National 1947 3169 0.82 0.75 346 9.63 6.56
MELD National 1985 3149 0.82 0.75 284 9.46 6.65
Outcome measure Patients relisted Deaths while
waiting 1 year patient survival 1 year graft
survival median wait time (days) mean survival
(years) Mean survival (QALYS)
52
Geographic Variability

• Model captures the remarkable geographic
  variability in waiting times, which is eliminated
  with move to national list

53
SRTR efforts Simulation Allocation Models (SAMs)
Transplant Candidates
Post Transplant Events
Disease Progression
Outcomes Under Policy A
Simulation Allocation Model (SAM)
Compare Policies
Outcomes Under Policy B
Waiting List
Donor Organs
Unused Organs
54
SRTR Natural History Model
Bilirubin
time
Albumin
time

• Pick one individual, use that persons actual
  history
• What do you do when model ? actual history?
• How to interpolate?

Prothrombin time
time
55
SRTR evaluation of transplant policy Lung

• Lung transplant rules used waiting time as major
  prioritization
• Recently (2005) changed from longest wait first
  to sickest first
• Results have dramatically changed the survival in
  chronic progressive lung disease

56
SRTR evaluation of transplant policy Lung
September 24, 2006 Lung Patients See a New Era
of Transplants By Denise Grady A quiet
revolution in the world of lung transplants is
saving the lives of people who, just two years
ago, would have died on the waiting list.
57
Current allocation question
14
15
17
19
21
24
25
26
28
31
34
36
Patients ranked by MELD score (probability of
death in 3 months)
Sickest first
14
15
17
19
21
24
25
26
28
31
34
36
Largest Net Benefit First
6.9
7.2
6.7
6.3
7.2
7.6
7.3
8.1
8.2
7.4
7.8
7.5
NB
Survival with THIS organ (Stx)
14
NET Benefit Stx Sno-tx
Survival with NO organ (Sno-tx)
58
Summary

• Simulation methods match the problem in this
  context
• DES allows for queues, waiting times, etc to be
  emergent properties of the model
• Example of biological modeling with a policy
  overlay
• So, why is it so accepted?

59
Simulation model acceptance by Transplant
community

• Clear standard research methods wont work
• Impractical (and likely illegal) to randomize
• Model was built with clinical oversight and
  assistance
• Model demonstrates predictive validity
• Model predicts the effects of rules change
• Rules are changed
• Observe the actual results

60
(No Transcript)