Problems 10/3

1
Problems 10/3

• 1. Ehrenfasts diffusion model

2
Problems, cont.

• 2.
• Discrete uniform on 0,...,n

3
Problems, cont.

• 3.
• where
• k0?

4
Classification of states

• A state i for a Markov chain Xk is called
  persistent if
• and transient otherwise.
• Let
• and .
• j is persistent iff fjj1.
• Let

5
Some results

• Theorem
• Pii(s)1Fii(s)Pii(s)
• Pij(s)Fij(s)Pij(s) for i ? j.
• Proof
• As for the random walk case we deduce from the
  Markov property that
• Multiply both sides by sm, sum over m1 to get

6
Some results, cont.

• Corollary
• State j is persistent if and then
  for all i.
• State j is transient if and
  then for all i.
• Proof
• SInce we see that
• But (by Abels thm)

7
A final consequence

• If i is transient, then
• Why?
• Example Branching process
• What states are persistent? Transient?
• State 0 is called absorbing, since once the
  process reaches 0, it never leaves again.

8
Mean recurrence time

• Let Ti minngt0 Xn i and ?i E(TiX0i).
• For a transient state ?i 8.
• For a persistent state
• We call a recurrent state positive persistent if
  ?i lt 8, null persistent otherwise.
• Example Simple random walk
• positive recurrent non-null persistent

9
A model forradiation damage

• Initial damage from radiation can either heal or
  get worse until it is visible.
• 0 is a healthy organism (absorbing)
• 3 visible damage (absorbing)
• 1 initial damage
• 2 amplified damage

10
Radiation damage, cont.

• Recovery probability ?0 is probability of
  reaching 0 before 3.
• Last step must go
• Thus

11
Communication

• Two states i and j communicate, , if
  for some m.
• i and j intercommunicate, , if
  and .
• Theorem is an equivalence relation.
• What do we need to prove?

12
Equivalence classes of states

• Theorem If then
• i is transient iff j is transient
• i is persistent iff j is persistent
• Proof of (a) Since there are m,n with
• By Chapman-Kolmogorov
• so summing over r we get

13
Closed and irreducible sets

• A set C of states is closed if pij0 for all i in
  C, j not in C
• C is irreducible if for all i,j in C.
• Theorem
• STC1C2...
• where T are all transient, and the Ci are
  irreducible disjoint closed sets of persistent
  states
• Note The Ci are the equivalence classes for

14
Example

• S0,1,2,3,4,5
• 0,1,4,5 closed irreducible persistent
• 2,3 transient. Why?

15
Long-term behavior

• Recall from the 0-1 process that
• When does this not depend on n?
• p01 p11

16
Stationary distribution

• Case (b) is the general one. Here is the idea
  Recall that ?(n) ?(0)Pn. In order to get the
  same distribution for all n, we use ?(0)??where
  ??solves ??P ??
• ?(1) ??P ?
• ??P2 ??P ?
• ...
• ??Pn ?

17
Snoqualmie Falls

• so
• or