ST3236: Stochastic Process Tutorial 3

1
ST3236 Stochastic ProcessTutorial 3

• TA Mar Choong Hock
• Email g0301492_at_nus.edu.sg
• Exercises 4

2
Question 1

• A markov chain X0,X1, on state 0, 1, 2 has the
  transition
• probability matrix
• and initial distributions, p0 P(X0 0) 0.3,
• p1 P(X0 1) 0.4 and p2 P(X0 2) 0.3.
• Determine P(X0 0, X1 1, X2 2) and draw the
  state-
• diagram with transition probability.

3
Question 1
P(X0 0,X1 1,X2 2) P(X0 0)P(X1 1
X0 0)P(X2 2 X0 0,X1 1) P(X0 0)P(X1
1 X0 0)P(X2 2 X1 1) p0 x p01 x
p12 0.3 x 0.2 x 0 0.
4
(No Transcript)
5
Question 2

• A markov chain X0,X1, on state 0, 1, 2 has the
  transition
• probability matrix
• Determine the conditional probabilities
• P(X2 1,X3 1X1 0) and P(X1 1,X2 1X0
  0).

6
Question 2
P(X2 1, X3 1 X1 0) P(X2 1 X1
1)P(X3 1 X1 0, X2 1) P(X2 1 X1
0)P(X3 1 X2 1) p01 x p11 0.2 x 0.6
0.12 Similarly (or by stationarity), P(X1 1,
X2 1 X0 0) 0.12 In general, P(Xn1 1,
Xn2 1 Xn 0) 0.12 for any n. That is,
it doesnt matter when you start.
7
Question 3

• A markov chain X0,X1, on state 0, 1, 2 has the
  transition
• probability matrix
• If we know that the process starts in state X0
  1, determine
• probability P(X0 1,X1 0,X2 2).

8
Question 3
P(X0 1,X1 0,X2 2) P(X0 1)P(X1 0 X0
1)P(X2 2 X0 1,X1 0) P(X0 1)P(X1
0 X0 1)P(X2 2 X1 0) p1 x p10 x p02 1
x 0.3 x 0.1 0.03
9
Question 4

• A markov chain X0,X1, on state 0, 1, 2 has the
  transition
• probability matrix

10
Question 4a
Compute the two-step transition matrix
P(2). Note Observe that the rows must
always sum to one for all transition matrices.
11
Question 4b
What is P(X3 1X1 0)?
P(X3 1X1 0) 0.13 In general, P(Xn2 1
Xn 0) 0.13 for any n.
12
Question 4c
What is P(X3 1X0 0)? Note that
Thus, P(X3 1X0 0) 0.16 In general, P(Xn3
1 Xn 0) 0.16 for any n.
13
Question 5

• A markov chain X0,X1, on state 0, 1, 2 has the
  transition
• probability matrix
• It is known that the process starts in state X0
  1, determine
• probability P(X2 2).

14
Question 5

• Note that
• P(X2 2) P(X0 0) x P(X2 2 X0 0)
• P(X0 1) x P(X2 2 X0 1)
• P(X0 2) x P(X2 2 X0 2)
• p0p02 p1p12 p2p22
• 1 x p12 0.35

15
Question 6

• Consider a sequence of items from a production
  process, with each item being graded as good or
  defective.
• Suppose that a good item is followed by another
  good item with probability ? and by a defective
  item with probability 1-?.
• Similarly, a defective item is followed by
  another defective item with probability ? and by
  a good item with probability 1-?.
• Specify the transition probability matrix.
• If the first item is good, what is the
  probability that the first defective item to
  appear is the fifth item?

16
Question 6
Let Xn be the grade of the nth product. P(Xn1
g Xn g) ?, P(Xn1 d Xn g) 1
-? P(Xn1 d Xn d) ?, P(Xn1 g Xn d)
1 - ? Thus, the transition probability matrix
is
17
Question 6
The probability is, P(X5 d,X4 g,X3 g,X2 g
X1 g) P(X2 g X1 g) x P(X3 g X2
g) x P(X4 g X3 g) x P(X5 d X4 g)
(why?) pgg x pgg x pgg x pgd ?3(1 -?)
18
Question 7

• The random variables ?1, ?2, ... are independent
  and with
• common probability mass function
• Set X0 0 and let Xn max ?1, ?2, ... .
• Determine the transition probability matrix for
  the MC Xn.
• Draw the state-diagram associated with transition
  probability

19
Question 7

• Observe
• X0 0,
• X1 max X0, ?1,
• X2 max X1, ?2,
• Xn max Xn-1, ?n
• Hence, Xn recursively compares the previous
• maximum and the current input to obtain the new
• maximum.

20
Question 7
The state space is S 0, 1, 2, 3 P(Xn1 0
Xn 0) P(?n1 0) 0.1 P(Xn1 1 Xn 0)
P(?n1 1) 0.3 P(Xn1 2 Xn 0)
0.2 P(Xn1 3 Xn 0) 0.4 P(Xn1 1 Xn
1) P(?n1 0) P(?n1 1) 0.1 0.3
0.4 P(Xn1 2 Xn 1) 0.2 P(Xn1 2 Xn
2) 0.1 0.3 0.2 0.6 P(Xn1 3 Xn
3) 0.1 0.3 0.2 0.4 1 P(Xn1 j Xn
i) 0 if j lt i. (Cannot Happen!)
21
Question 7
The transition probability matrix is
22
(No Transcript)