TALLER CADENAS DE MARKOV DEL ESTADO ESTABLE

Presentado por:
Karen Ayala Del Toro
Esteban González Polo

Presentado a:
Ing. Katherinne Salas Navarro, PhD

Universidad de la Costa - CUC

Facultad de Ingenierías
Programa de Ingeniería Industrial
Investigación de Operaciones II
Barranquilla
2020
 TALLER CADENAS DE MARKOV DEL ESTADO ESTABLE

1. Given the following (one-step) transition matrix of a Markov chain

0 1 2 3 4 5
0, 0,
0 0 0 0
0 67 33
1,
0 0 0 0 0
1 00
1,
0 0 0 0 0
2 00
0, 0,
0 0 0 0
3 25 75
1,
0 0 0 0 0
4 00
0, 0,
0 0 0 0
5 5 5

a. Draw the state transition diagram for the Markov chain.

012345
1/3

2/3

3/4

1
1/4 1/2 1 1/2

b. Determine the period of each of the states in the Markov chain.

 Estado 0: 4 periodos
 Estado 1: 4 periodos
 Estado 2: 4 periodos
 Estado 3: 4 periodos
 Estado 4: 4 periodos

c. Find the steady-state probabilities.

π 0 ¿ π 0 ( 0 ) + π 1 ( 0 ) +π 2 ( 1 ) + π 3 ( 0 )+ π 4 ( 0 ) + π 5 ( 0 )
T 1 ¿ π 0 ( 0 ) +π 1 ( 0 ) + π 2 ( 0 ) + π 3 ( 1/ 4 ) + π 4 ( 0 ) + π 5 ( 1/2 )

π 2 ¿ π 0 ( 0 ) + π 1 ( 1 ) + π 2 ( 0 ) + π 3 ( 0 )+ π 4 ( 1 ) + π 5 ( 0 )

T 3 ¿ π 0 ( 2/3 )+ π 1 ( 0 ) + π 2 ( 0 ) +π 3 ( 0 )+ π 4 ( 0 )+ π 5 ( 0 )

T 4 ¿ π 0 ( 0 ) + π 1 ( 0 ) + π 2 ( 0 )+ π 3 ( 3/4 ) + π 4 ( 0 ) + π 5 ( 1/2 )

π 5 ¿ π 0 ( 1/3 ) + π 1 ( 0 )+ π 2 ( 0 ) + π 3 ( 0 ) +π 4 ( 0 ) + π 5 ( 0 )

π 0 + π 1+ π 2 + π 3 + π 4 + π 5=1

Simplificando las ecuaciones

π 0=π 2 ( 1 )

π 1=π 3 (1 /4 ) + π 5 ( 1/2 )

π 2=π 1 (1 ) + π 4 ( 1 )

π 3 ¿ π 0 ( 2/3 )

π 4 =π 3 ( 3 /4 ) + π 5 ( 1/2 )

π 5=π 0 ( 1/3 )

π 0 + π 1+ π 2 + π 3 + π 4 + π 5=1

Igualando a cero
π 2 ( 1 ) −π 0 ( 1 )=0

π 3 (1 /4 ) + π 5 ( 1/2 )−π 1 (1)=0

π 1 ( 1 ) + π 4 ( 1 )−π 2 (1)=0

π 0 ( 2/3 )−π 3 (1)=0

π 3 (3 /4 ) + π 5 ( 1/2 )−π 4 (1)=0

π 0 (1 /3 )−π 5 (1)=0

π 0 + π 1+ π 2 + π 3 + π 4 + π 5=1

Solución:
En calculadora de matrices:
1
π 0=
4
 133
π 1=
1600
1
π 2=
4
67
π 3=
400
2 67
π4=
1600
33
π 5=
400

Anexo. Capture de solución en calculadora

 2. A production process contains a machine that deteriorates rapidly in both
quality and output under heavy usage, so that it is inspected at the end of each
day. Immediately after inspection, the condition of the machine is noted and
classified into one of four possible states:

Estat
0 1 2 3
e
0,87 0,062 0,062
0 0
5 5 5
1 0 0,75 0,125 0,125
2 0 0 0,5 0,5
3 1 0 0 0

a. Draw the state transition diagram for the Markov chain.

0.0625

0.0625 0.125
0.875 0.125 0.5

0 1 2 3
0.75 0.5

b. Find the steady-state probabilities.

0 1 2 3

0 0 0.875 0.0625 0.0625

P= 1
2
3
[ 0 0.75 0.125 0.125
0
1
0 0.5
0 0 0
0.5
]
π 0=π 3

π 1=0.875 π 0 +0.75 π 1

π 2=0.0625 π 0 +0.125 π 1+ 0.5 π 2

π 3=0.0625 π 0 +0.125 π 1+ 0.5 π 2

1=π 0 + π 1 + π 2+ π 3
Simplificando las ecuaciones
π 0=π 3

π 1=0.875 π 0 +0.75 π 1

π 2=0.0625 π 0 +0.125 π 1+ 0.5 π 2

π 3=0.0625 π 0 +0.125 π 1+ 0.5 π 2

1=π 0 + π 1 + π 2+ π 3

Igualando a cero
π 0−π 3=0

−0.875 π 0 +0,25 π 1=0

−0.0625 π 0−0.125 π 1 +0.5 π 2=0

−0.0625 π 0−0.125 π 1−0.5 π 2 +π 3=0

π 0 + π 1+ π 2 + π 3=1

Solución
2
π 0=
13
7
π 1=
13
2
π 2=
13
2
π 3=
13
Anexo. Capture de solución en calculadora
 c. If the costs of being in states 0, 1, 2, 3, are 0, $1,000, $3,000, and $6,000,
respectively, what is the long-run expected average cost per day?

Π∗C ¿ π 0 ( 0 )+ π 1 ( 1000 ) +π 2 ( 3000 ) + π 3 ( 6000 )

Π∗C=0,154∗( 0 ) +0,538 ( 1000 ) +0,154∗( 3000 ) +0,154∗( 6000 ) =1923,077