Markov Chains

1
Lecture 9

• Markov Chains
• Examples

2
Moodiness

• On a given day Vince is either cheerful (C),
  so-so (S), or glum (G).
• If he is C today, then he will be C, S, or G
  tomorrow with respective probs 0.5, 0.4, 0.1.
• If he is feeling S today, then he will be C, S,
  or G tomorrow with probs 0.3, 0.4, 0.3.
• If he is G today, then he will be C, S, or G
  tomorrow with probs 0.2, 0.3, 0.5.
• Let Xn denote Garys mood on the nth day
• Then Xn , n 0 is a three state Markov chain
• Write the transition probability matrix

3
Weather

• Suppose that whether or not it rains today
  depends on previous weather conditions through
  the last two days
• If it has rained for the past two days, then it
  will rain tomorrow with probability 0.7
• If it rained today, but now yesterday, then it
  will rain tomorrow with probability 0.5
• If it rained yesterday, but not today, then it
  will rain tomorrow with probability 0.4
• If it did not rain in the past two days, then it
  will rain tomorrow with probability 0.2

4
Weather

• If we let the state at time n depend only on
  whether or not it is raining at time n, the this
  is not a Markov chain
• Why not?
• But, we can make a Markov chain if we consider 4
  states (determined by the weather conditions from
  both today and yesterday)

5
Weather

• Now we have a 4 state Markov chain
• Write the transition probability matrix

6
Squirrels

• The American gray squirrel
• Introduced in Great Britain by a series of
  releases from various sites
• Starting in the late 19th century (1876)
• They have now spread throughout England, Wales
  and part of Scotland and Ireland
• The native red squirrel
• Considered the endemic subspecies
• Has disappeared from most of the areas colonized
  by the gray squirrels
• In the last century the popn has consistently
  declined
• Extinct in many areas of England and Wales
• Some popns still exists on offshore islands in
  southern England and mountainous Wales

7
Squirrels

• Introduction of gray squirrels continued until
  1920
• By 1930 it is considered a pest in deciduous
  forests and control measures were attempted
• National distribution surveys were taken
• They found that the red squirrel was being lost
  from areas that had been colonized by gray
  squirrels during the past 15-20 years
• Questionaires are given to foresters to fill out
  regarding the squirrel populations
• Changes in squirrel abundance, tree damage,
  squirrel control measures, number of squirrels
  killed

8
Squirrels

• Using the data collected we can construct a model
  to predict the trends in distribution of both
  species of squirrels in Great Britain
• Usher et al
• Used a map overlay technique to extract data from
  distribution maps made for the forestry
  commission
• The maps were divided in to 10-km square sections
• Each 10-km square grid is classified as either
• R red only squirrels recorded that year
• G gray only
• B both species
• O neither species

9
Squirrels

• To satisfy Markov assumptions we only consider
  squares in two consecutive years
• There are 16 classes

10
Squirrels

• Write down the transition probability matrix
• This is based on conditional probabilities
• Draw the state diagram
• What happens to the squirrel population over a
  long period of time?

11
Aged Structured Populations
12
Aged Structured Populations

• Population with non-overlapping generations
• We may be concerned with several previous
  generations OR entire age distribution
• This is typically dealt with using a Leslie
  matrix
• Developed by PH Leslie (1945)
• Let xit be the of individuals of age i in year
  (generation) t

13
Aged Structured Populations

• Let xit be the of individuals of age i in year
  (generation) t
• Newborns are age 0
• Max age is w
• If we are modelling a sexual species then we only
  consider the female indivs and offspring
• We assume that there are enough males to go around

14
Leslie Matrix

• net fecundity of age i indivs mip0
• (how many offspring)(how many survive
  (probability))
• of age 1 indivs in t1 per age i indiv in t
• So

15
Leslie Matrix

• The leslie matrix for

16
Leslie Matrix

• Then
• Note that we never get x0 with this formulation
• Is this a Markov process?
• No transposed i,js so that Lij gives
  contribution xi from xj
• Also, column sums reflect the of indivs in
  generation t1 per indiv of age j in t
• This sum not necessarily 1 (as opposed to Markov
  processes, which are generally conservative

17
Leslie Matrix

• Many assumptions hidden in this
• Age alone is the dominant predictor of fecundity
  and survival probability
• (ignores any effects of total population size)
• other assumptions
• Analogously with Markov chains we can solve for
  the stable age distribution,

18
Leslie Matrix

• Analogously with Markov chains we can solve for
  the stable age distribution
• Proportion of total population that is age i
• If the age distribution is stable, it must be
  true that

19
Leslie Matrix

• So, is an eigenvector of L
• If L has a unique, real, dominant (largest
  magnitude) eigenvalue ? corresponding to the
  right eigenvector then for t1 sufficiently
  large
• This gives us the stable age distribution
• is giving us the stable age distribution

20
Leslie Matrix

• Then, for
• But,
• So
• Distribution at time t is given by the stable age
  distribution scaled by ?t and c1
• So if ?gt1, all the age classes and total
  population size will grow together, geometrically
  by ? each year, but the distribution in age stays
  the same

21
Leslie Matrix

• Remember that this is only true for tgtt1 (i.e.
  after stable age distribution is reached)
• But what is c1?
• Suppose that is a row vector given by the
  left eigenvector of L ( L? where ? is the
  eigenvalue) scaled such that x1
• Now

22
Leslie Matrix

• So given L, we can solve for ?, , , then
  given we know c1
• We know stable age distn, amount population
  grows or declines each year (?) and we can
  compute the actual distn for any t
• What is intuitively?
• gives the relative importance of individuals
  of different ages in x0 to the total population
  size in future

23
Leslie Matrix

• Example
• Suppose 1 1.6 1.4 1.3/X
• Means- a two year old at time 0 would have 1.6
  time as many offspring (descendants) in the
  distant future as a 1 year old at time 0
• I thought the initial conditions didnt affect
  the long run distribution
• True for irreducible, aperiodic, closed set
  Markov process
• For these Markov processes the total of indivs
  is constant
• But with Leslie matrix, columns usually dont sum
  to 1
• Popn could increase with time

24
Survival Probability

• Let ?ip0p1p2pi-1 be the probability that a
  newborn survives to age i
• Recall that xi,t age i in genn at t
• Newborns in genn t are given by
• Individuals of age i in genn t were born in t-i
  and survived

25
Survival Probability

• If we have reached the stable age distribution,
  each age group is increasing geometrically by ?
• of age i at time t in terms of of newborns at
  t

26
Survival Probability

• If we have reached the stable age distribution,
  each age group is increasing geometrically by ?
• of age i at time t in terms of of newborns at
  t

27
Survival Probability

• So,
• From this we can compute ? explicitly