Markov Chain Population Models

1
Markov Chain Population Models in Medical
Decision Making
Gordon Hazen Min Huang Northwestern University
2
Markov models (individual-level) in medical
decision making
Intervention that reduces disease mortality rate
3
Conventional outcome measureQALYs for an
individual (or a cohort)
4
From individual to population
Motivation To study a whole population

• Equilibrium distribution of a population
• 2. Equilibrium measure of effectiveness of
• an intervention

1. no equilibrium 2. no births
Individual-level models
5
Augment model by allowing births
Intervention that reduces disease mortality rate
6
Population model and its routing
Population model
Routing process
7
Population no longer dies outreaches new
equilibrium after intervention
8
Time-homogeneous individual-level Markov models
Individual Markov model
State space
0,1,2,..,J,-1, where -1 representing Death
is an absorbing state
Transition rates
9
Population models
Population Markov model
State space
Transition rates
Open Jackson processes
Serfozo
Serfozo R. Introduction to Stochastic Networks.
Springer 1999.
10
Routing processes
Individual-level model
State space
0,1,2,..,J,-1, where -1 is a source/sink node
Transition rates
11
Properties
is irreducible, then at equilibrium
If

• are independent,

)
Poisson(
equilibrium population means

• Conditional on total population size n, n is
• multinomial

equilibrium population proportions

12
Equilibrium population means
is the unique collection of positive numbers
that satisfy balance equations of routing process
i.e.
.
Here Q is a submatrix of the rate matrix of the
routing process, and also a submatrix of the rate
matrix of the underlying individual model,
corresponding to all nonabsorbing states, i.e.,
health states 0,1,,J.
13
What measures of quality are possible at the
population level?
Measures of health
Individual QALYs
QALYs for an individual starting in state j
Equilibrium population measures
14
Average Lifetime QALY ALQ
Mean QALY of randomly selected individual from
equilibrium population
15
Total Lifetime QALY TLQ Mean total
QALYs of all individuals in equilibrium
population
16
Average QALYs per Year AQ/yr One-year
QALY of randomly selected individual from
equilibrium population
17
Total QALYs per Year TQ/yr One-year QALY
of all individuals in equilibrium population
18
Discounted Total QALYs DTQ Mean total
discounted QALYs for this and all subsequent
generations of population.
19
Relationships between measures
DTQ
TQ/yr
if the population is in equilibrium from t0.
TLQ
ALQ
TQ/yr
AQ/yr
TQ/yr
AQ/yr
20
The simple illustrative example differences
among measures
Intervention that reduces disease mortality rate
21
Evaluating interventions using these measures
22
Insight

• Problem average measures do not account for
  population size increase due to better survival.

• Caution in choosing population measures

23
Example tamoxifen use to prevent breast
cancerCol
Col N.F., Orr R.K., Fortin J.M. Survival impact
of tamoxifen use for breast cancer risk
reduction projections from a patient-specific
Markov model, Med Decis Making 2002 22 386-393.
24
Non-homogeneous individual-level Markov models
1. Human background survival
Background mortality rate
(Gompertz)
2. The other factor a homogeneous Markov process
25
Population models
Mean density with respect to age a of the
population in state j at time t
Theorem
26
Notations
equilibrium mean density with respect to age a of
the population in state j,
equilibrium expected total population count in
state j.
Conclusions
27
Measures of health
Individual QALYs
QALYs for an individual starting from age a0 in
state j
Equilibrium population measures
ALQ TLQ AQ/yr TQ/yr TLQ
28
Example tamoxifen use to prevent breast
cancerCol
29
Summary

• Population Markov models for medical decision
  making.
• Population measures of interventions
• Age-dependency.