Elementary Introduction to Markov Chains

1
Elementary Introduction to Markov Chains
2
Stochastic (random) processes
3
Integer-valued stochastic process
càdlàg continus à droite, limités à gauche,
trajectories
4
Markov property in discrete time

• Sequence of random variables
• If the process is nonnegative integer-valued,
  then
• where
• the state space of the process (chain).

5
Transition probabilities
6
Marginal probabilities
7
Example 1 Irreversible mutations in discrete
generations

• Let us assume that state 1 can mutate into state
  2 but not conversely
• After a long time, nothing is left in state 1
  (state 2 is absorbing)

8
Example 2 Reversible mutations in discrete
generations

• Let us assume that state 1 can mutate into state
  2 and conversely
• We expect a stationary distribution

9
Markov property in continuous time
10
Transition probabilities in continuous time
11
Transition intensities
12
Matrix of transition intensities
13
Relationship between transition intensities and
transition probabilities
14
Stationarity in time-continuous processes
15
Example 3 Mutations in continuous time
16
Reversibility
17
(No Transcript)