Agenda for Next Week

1
Agenda for Next Week
2
Chapter 17

• Markov Processes Part 2

3
Review

• A Markov Process describes a situation where a
  system is in one state at a time
• Switching between states is probabilistic
• The state of the system is dependent ONLY on the
  previous state of the system

4
Example Machine Adjustment
In adjustment (state 1) 0.7 0.6
Out of adjustment (state 2) 0.3 0.4
To From In adj. (1) Out of adj. (2)
5
Example Machine Adjustment
If the machine is found to be in adjustment on
day 1, what is the likelihood it will be in
adjustment on day 3? Not in adjustment?
Day 1
Day 2
.7
1
.7
1
.3
.3
2
6
Example Machine Adjustment
Day 3
.49
Day 2
1
.7
Day 1
.3
.21
1
2
.67
.7
1
.6
.3
.18
2
1
.33
.4
.12
2
7
.7
1
Day 4
1
2
.3
.7
.6
1
.3
1
2
.7
.4
2
1
.7
1
.6
.3
2
1
2
.3
.4
1
.6
2
2
.4
8
.7
1
Day 4
1
2
.3
.7
.6
1
.3
1
2
.7
.4
2
1
.7
1
.6
.3
2
1
2
.3
.4
1
.6
2
2
.4
9
Example Machine Adjustment

• Day 4
• P(S1S1) .7(.7)(.7) .7(.3)(.6) .3(.6)(.7)
  .3(.4)(.6)
• .667
• P(S2S1) .7(.7)(.3) .7(.3)(.4) .3(.6)(.3)
  3(.4)(.4)
• .333

10
Day 5
.7
.7
.3
1
1
.6
.4
2
.3
.7
.7
.6
.3
1
.3
1
2
.6
.7
.4
.4
2
1
.7
.7
.3
1
.6
.3
.6
2
1
.4
2
.3
.7
.4
.3
1
.6
.6
2
.4
2
.4
11
Example Machine Adjustment

• Day 5
• P(S1S1) .7(.7)(.7)(.7) .7(.7)(.3)(.6)
• .7(.3)(.6)(.7) .7(.3)(.4)(.6)
• .3(.6)(.7)(.7) .3(.6)(.3)(.6)
• .3(.4)(.6)(.7) .3(.4)(.4)(.6)
• .666
• P(S2S1) .7(.7)(.7)(.3) .7(.7)(.3)(.4)
• .7(.3)(.6)(.3)
  .7(.3)(.4)(.4)
• .3(.6)(.7)(.3)
  .3(.6)(.3)(.4)
• .3(.4)(.6)(.3)
  .3(.4)(.4)(.4)
• .334

Notice anything interesting?
12
Steady State Probabilities

• These probabilities are called steady state
  probabilities
• The long term probability of being in a
  particular state no matter which state you begin
  in
• Steady state prob. (state 1) .667
• Steady state prob. (state 2) .333

13
Example Machine Adjustment
In adjustment (state 1) 0.7 0.6
Out of adjustment (state 2) 0.3 0.4
To From In adj. (1) Out of adj. (2)
14
Example Machine Adjustment
In adjustment (state 1) p11 p21
Out of adjustment (state 2) p12 p22
To From In adj. (1) Out of adj. (2)
15
Steady State Probabilities
P(S1 Day n1S1) .7 P(S1 Day nS1) .6
P(S2 Day nS1)
P2
P1
P(S2 Day n1S1) .3 P(S1 Day nS1) .4
P(S2 Day nS1)
P2
P1
16
Steady State Probabilities
P1 p11P1 p21P2
P1 (1 - p12 ) P1 p21 (1- P1)
P1 P1 - p12 P1 p21 - p21 P1
p12 P1 p21 P1 p21
p21
P1
p12 p21
17
Steady State Probabilities
p21 p12 p21
.6 .3.6
.6 .9
2 3
P1
p12 p12 p21
.3 .3.6
.3 .9
1 3
P2
18
Example Steady State

• Let p1 long run proportion of refused calls
• p2 long run proportion of accepted calls
• Then,

.70 .30 p1 p2
.60 .40 p1 p2
19
Example Steady State

• .70p1 .60p2 p1 (1)
• .30p1 .40p2 p2 (2)
• p1 p2 1 (3)
• Solve for p1 and p2

Can be restated as
p1 1 p2 p2 1 p1
20
Example Steady State

• Using equations (2) and (3), substitute p1 1
  p2
• into (2)
• .30(1 - p2) ?40p2 p2
• This gives p2 .3333
• Substituting back into equation (3) gives
• p1 .67
• Thus the expected number of accepted calls per
  year is (.76471)(52) 39.76 or about 40

21
Example

• Henry, a persistent salesman, calls North's
  Hardware Store once a week hoping to speak with
  the store's buying agent, Shirley. If Shirley
  does not accept Henry's call this week, the
  probability she will do the same next week (and
  not accept his call) is .35. On the other hand,
  if she accepts Henry's call this week, the
  probability she will not accept his call next
  week is .20.

22
Example Transition Matrix
Next Weeks Call
This Weeks Call
23
Example

• What is the probability Shirley will accept
  Henry's next two calls if she does not accept his
  call this week?

24
Example
Refuses
.35
P .35(.35) .1225
Refuses
Accepts
.35
Refuses
P .35(.65) .2275
.65
Refuses
P .65(.20) .1300
Accepts
.20
.65
Accepts
P .65(.80) .5200
.80
25
Example

• What is the probability of Shirley accepting
  exactly one of Henry's next two calls if she
  accepts his call this week?

26
Example
Refuses
.35
P .20(.35) .07
Refuses
Accepts
.20
Accepts
P .20(.65) .13
.65
Refuses
P .80(.20) .16
Accepts
.20
.80
Accepts
P .80(.80) .64
.80
27
Example Steady State

• How many times per year can Henry expect to talk
  to Shirley?
• Answer To find the expected number of accepted
  calls per year, find the long-run proportion
  (probability) of a call being accepted and
  multiply it by 52 weeks.

28
Example Steady State

• Let p1 long run proportion of refused calls
• p2 long run proportion of accepted calls
• Then,

.35 .65 p1 p2
.20 .80 p1 p2
29
Example Steady State

• .35p1 .20p2 p1 (1)
• .65p1 .80p2 p2 (2)
• p1 p2 1 (3)
• Solve for p1 and p2

Can be restated as
p1 1 p2 p2 1 p1
30
Example Steady State

• Using equations (2) and (3), substitute p1 1
  p2
• into (2)
• .65(1 - p2) ???p2 p2
• This gives p2 .76471
• Substituting back into equation (3) gives
• p1 .23529.
• Thus the expected number of accepted calls per
  year is (.76471)(52) 39.76 or about 40