Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Sections 2.9
Sections 2.9
Page 1/1
Sections 2.9
Page 2/1
Sections 2.9
Page 3/1
Sections 2.9
Page 4/1
The probability that it will be sunny on Sunday is 27.325%. Important fact: the sum of the entries of a state vector is always one.
Sections 2.9 Page 6/1
Sections 2.9
Page 7/1
Example (continued)
Initially, the customer is equally likely to eat at either restaurant, so S0 = 0.125 0.875 S4 = S6 =
1 2 1 2
S1 =
, S2 =
0.21875 0.78125 , S5 = , S7 =
, S3 =
0.1953125 0.8046875 , ,
Example (continued)
1 2 1 2
1 2 1 2
S1 = 0 , S2 = PS1 = 0
1 2 1 2 1 4 1 2 1 4 1 4 1 2 1 4 1 4 1 4 1 2 1 4 1 4 1 2
1 4 1 2 1 4
1 4 1 4 1 2 3 8 1 8 1 2 11 32 3 16 15 32
1 2 1 2
0 =
11 32 3 16 15 32
3 8 1 8 1 2
S3 = PS2 = 0
1 2 1 2
S4 = PS3 = 0
29 64
29 64 .
Sections 2.9
Page 10/1
Sometimes, state vectors converge to a particular vector, called the steady state vector.
Problem
How do we know if a Markov chain has a steady state vector? If the Markov chain has a steady state vector, how do we nd it? One condition ensuring that a steady state vector exists is that the transition matrix P be regular, meaning that for some integer k > 0, all entries of P k are positive (i.e., greater than zero).
Example
In 2.9 Example 1, P = 0 1 0 1
1 4 3 4 1 4 3 4
is regular because 0 1
1 4 3 4 1 4 3 4 3 16 13 16
P2 =
(I P )S = 0 This last line represents a system of linear equations that is homogeneous. The structure of P ensures that I P is not invertible, and so the system has innitely many solutions. Choose the value of the parameter so that the entries of S sum to one.
Sections 2.9 Page 12/1
Example
From 2.9 Example 1, P= 0 1
1 4 3 4
and weve already veried that P is regular. Now solve the system (I P )S = 0. I P = Solving (I P )S = 0: 1 1 4 1 1 4 0 0
1 0 1 4 0 0 0
1 0 0 1
0 1
1 4 3 4
1 1 4 1 1 4
Example (continued)
Since s1 + s2 = 1, 1 t +t = 1 4 5 t = 1 4 4 t = 5 Therefore, the steady state vector is S=
1 5 4 5
0.2 0.8
Sections 2.9
Page 14/1
2 P= 0
1 2
1 4 1 2 1 4
1 4 1 4 1 2
so
P2 = 0
1 2
1 2
1 4 1 2 1 4
1 4 1 4 1 2
1 2 1 2
1 4 1 2 1 4
1 4 1 4 1 2
5 8 1 8 1 2
5 16 5 16 3 8
5 16 1 4 7 16
Sections 2.9
Page 15/1
Example (continued)
Now solve the system (I P )S = 0.
1 2
1 4
0
1 2
1 2 1 4
1 4 1 4
1 2
1 1 0 4 2 1 0 0 2 1 0 0 2
1 4 1 4
1 4
1 0 1 1 0 2 2 1 1 0 0 2 4 0 0 0 0 0 0
1 0 0
0 3 1 0 3 4 0 4 0 1 1 1 2 4 0 0 1 2 0 0 0 0 0 0 0 0
Sections 2.9
Page 16/1
Example (continued)
Since s1 + s2 + s3 = 1, 1 3 t + t + t = 1, 2 4
S =
3 9 2 9 4 9
Sections 2.9
Page 17/1