Chapter 10 Markov Chains

1
Chapter 10Markov Chains

• Section 10.2
• Regular Markov Chains

2
Short-Term Predictions

• Recall from the previous section that short-term
  predictions can be made for n repetitions of an
  experiment using the initial probability vector
  and the transition matrix.

3
KickKola Example

• Also from the previous section, we found that
  KickKola sodas market share increased after each
  of two following purchases from their original
  market share of 14. (See Section 10.1, Example
  2 notes)
• What will happen to their market share after a
  large number of repetitions?
• The answer to this question requires us to make
  a long-range prediction and find the level at
  which the trend will stabilize.

4
Long-Range Predictions

• In this section, we try to decide what will
  happen to the initial probability vector in the
  long run that is, as n gets larger and larger.
• Assumption transition matrix probabilities
  remain constant from repetition to repetition.

5
Regular Transition Matrices

• While one of the many applications of Markov
  chains is in finding long-range predictions, its
  not possible to make long-range predictions with
  all transition matrices.
• Long-range predictions are always possible with
  regular transition matrices.

6
Regular Transition Matrices

• A transition matrix is regular if some power of
  the matrix contains all positive entries.
• A Markov chain is a regular Markov chain if its
  transition matrix is regular.

7
Determining if Transition Matrices are Regular

• If some power of the matrix has all positive,
  non-zero entries, then the matrix is regular.
• If a transition matrix has some zero entries and
  if these zeros occur in the identical places in
  both P and P for any k, then the zeros
  will appear in those places for all higher powers
  of P. Therefore, P is not regular.

k1
k
8
Example 1

• Decide whether the following transition matrices
  are regular.
• a.) b.)

9
Equilibrium Vectors
10
Equilibrium Vector

• If a Markov chain with transition matrix P is
  regular, then there exists a probability vector V
  such that
• VP V
• This vector V gives the long-range trend of the
  Markov chain. Vector V is found by solving a
  system of linear equations.
• If there are two states, then V x y
• If there are three states, then V x y z

11
Example 2

• Find the equilibrium vector for the transition
  matrices below.
• a.) b.)

12
Properties of a Regular Markov Chain
13
Steps for Making a Long-Range Prediction

• 1.) Create the transition matrix P.
• 2.) Determine if the chain is regular.
  
• (This occurs when the transition matrix
  is regular.)
• 3.) Create the equilibrium matrix V.
• 4.) Find and simplify the system of
• equations described by VP V.
• 5.) Discard any redundant equations
• and include the equation x y 1
• related to V x y. (Or xyz1)
• 6.) Solve the resulting system.
• 7.) Check your work by verifying that VP V.

14
Example 3 - KickKola Revisited

• A marketing analysis shows that 12 of the
  consumers who do not currently drink KickKola
  will purchase KickKola the next time they buy a
  cola and that 63 of the consumers who currently
  drink KickKola will purchase it the next time
  they buy a cola.
• Make a long-range prediction of KickKolas
  ultimate market share, assuming that current
  trends continue.

15
Example 4

• A census report shows that currently 32 of the
  residents of Metropolis own their home and that
  68 rent.
• Of those that rent, 12 plan to buy a home in
  the next 12 months, while 3 of the homeowners
  plan to sell their home and rent instead.
• Make a long-range prediction of the percent of
  Metropolis residents who will own their home and
  the percent who will rent, assuming current
  trends hold.