Governor

1
Governors School for the Sciences

• Mathematics

Day 7
2
MOTD Andrei A. Markov

• 1856 to 1922
• Stochastic processes
• Markov Chains

3
Markov Process

• State Diagram
• Represent by System of Linear Difference
  Equations
• Determine changes in the probabilities for being
  in a certain state

4
Key Transition Matrix

• N states gt NxN Transition matrix T
• T(i,j) probability of changing from
  state j to state i
• X(n) vector of probabilities (at time n)
• X(n1) TX(n) (Markov Process)
• Properties 1. T never changes 2. At each
  time, all the probabilities in X(n) sum to
  1 3. Columns of T sum to 1

5
Boardwork

• Keywords States, Distribution, Transition Matrix
• Disease model (well, sick, dead)
• Dice game (roll 1-5 in order before getting a 6)
• Best of 3 Series

6
Disease Model
7
Dice Game
8
Dice Game Average Position
9
Best of 3 Series

• Equal prob of winning ¼ probability of
  ending at (0,2), (1,2), (2,1), (2,0)
• Team A has prob ¾ of winning game 9/16 prob
  of (2,0), 9/32 (2,1) 1/16 prob of (0,2), 3/32
  (1,2)

10
Exercise

• Two states Not dormed Dormed
• Transition Probabilities (per day) ND -gt ND
  0.8 ND -gt D 0.2 D -gt ND
  0.9 D -gt D 0.1
• Construct the transition matrix
• Determine if you start as Not dormed what is the
  prob of you being dormed after 1 day, 2 days, 3
  days

11
Results

• A state is an absorbing state if once you get
  there you cant leave
• If a system has only one absorbing state, then
  the distribution will tend towards that state
• If it has 2 absorbing states, then the
  distribution will tend towards some split of
  those two states (depending on the initial
  distribution)
• If it has no absorbing states then the
  distribution will tend towards some stable
  distribution

12
Break Time
13
Higher order DEs

• 2nd order x(n1) f(x(n), x(n-1))
• Fibonacci sequence x(n1) x(n) x(n-1)
• Take x(0) 1, x(1) 1, then we get 2, 3, 5,
  8, 13, 21, 34, 55, 89,
• Take x(0) 2, x(1) 4, then we get 6, 10,
  16, 26, 42, 68, 110,

14
Interesting facts

• For any sequence generated by the Fibonacci rule
  x(n1)/x(n) -gt g (g (1sqrt(5))/2 the
  Golden ratio)
• Many things in nature seem to follow the
  Fibonacci rule seeds in a sunflower, leaves on
  a pine cone (and other plants), etc.

15
Solving Linear DEs

• Solve x(n1) ax(n)
• Solve x(n1) ax(n) bx(n-1)
• Solve x(n1) x(n) x(n-1)
• Solve x(n1) ax(n) b
• Solve x(n1) ax(n) bx(n-1) c

16
System of Difference Equations

• Two sequences x(n) and y(n)
• Difference equation relating both x(n1)
  f(x(n), y(n)) y(n1) g(x(n), y(n))
• Example Predator-Prey x(n1) 1.1x(n)
  0.2y(n) y(n1) 0.2x(n) 0.95y(n) x
  rabbits, y - foxes

17
(No Transcript)
18
Lab

• Building transition matrices for a Markov process
  and studying the results
• T 0.2, 0.3 0.8, 0.3 sum(T)
• X 10
• TX
• X(,i1) TX(,i)