Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Ramesh Johari
Problem Set 2
Due: February 4, 2015, 5:00 PM, in the basement of Huang Eng. Ctr.
(a). Draw the graph representing the Markov chain describing the number of customers in the
queue (with states 0, 1, . . . , N ), and label the probability of each transition.
(b). Is the chain irreducible? Which states are recurrent?
(c). Suppose the queue starts in state 1. Write down a set of equations that yield the probaiblity
that exactly three customers are blocked before the queue empties. (Hints: Note that a customer is
blocked if and only if the queue is in state N , and a new arrival occurs. Also, by the Strong Markov
Property, conditional on one customer just having been blocked, the process behaves like a queue
starting with N customers.)
(d). Explain why there is a unique invariant distribution , and write down a set of equations
you can use to answer the following question: Assuming the states are distributed according to the
invariant distribution , what is the average number of customers in the queue? (You dont have to
solve the equations.)
Note: Next week, you will learn that this is also the long run average number of customers in
the queue.
Problem 4. (Service staffing) Consider the following model of a manager handling temp
staffing in a service organization.
Suppose that if a worker comes in to work today, he comes in to work tomorrow as well with
probability p, and otherwise shirksi.e., skips workwith probability 1 p. If the worker
shirks today, he receives a phone call from the manager, and this induces him to return to work the
next day with probability r, or to shirk the next day with probability 1 r. Finally, if the worker
shirks D days in a row, the manager gets fed up and replaces the worker with a new worker the
next day.
(a) Give a Markov chain description of the position from the point of view of the manager, with
states W (worker is working) and S1 , . . . , SD (where Si represents the worker having shirked for i
consecutive days). You may assume that a new worker always starts in state W.
(b) Show the chain has a unique invariant distribution , and compute it. Hint: You might find
the following identity useful:
1 + x + x2 + + xD1 = (1 xD )/(1 x).
Note: the remainder of the problem is OPTIONAL on this weeks problem set; I will be assigning parts (c)-(f) on Problem Set 3. Part (g) will remain optional even on Problem Set 3.
Now suppose that a cost c1 is incurred each day that the worker doesnt come in to work, and
that a replacement cost c2 is incurred if we have to hire a new worker (e.g., this might be the cost
we pay to a temp staffing firm to send us a new worker).
(c) (OPTIONAL this week) Define a function C(i) that gives, as a function of the state i, the
cost incurred in each state.
(d) (OPTIONAL this week) Determine the long run average cost (due to both shirking and
replacements).
(e) (OPTIONAL this week) Show that if c2 > c1 /r, then the average cost optimal decision (i.e.,
the decision which minimizes long run average cost) is to never replace the worker. (Hint: Show
that the long run average cost is decreasing in D.)
(f) (OPTIONAL this week) Show that there is a constant , such that if c2 /c1 > , then it is
optimal never to replace the worker; and if c2 /c1 , then it is optimal to replace the worker on
the first day he shirks.
(g) (OPTIONAL) Suppose that instead of the model described above, we were allowed to
choose each day whether to replace the worker; i.e., we are not required to fix D in advance, but
can make decisions dynamically. Are the policies of part (f) still optimal? Why or why not?
Problem 5. (Adapted from B&T) Consider the Markov Chain in Figure 1. We refer to a
transition that results in a state with a higher index as birth and that to state with a lower index as
death. Calculate the following quantities, assuming that when we start observing the chain, it is
already in steady-state.
(b) The above property is useful for designing randomized algorithms to uniformly sample
objects with certain properties. We now demonstrate this for the case of sampling independent sets
from a graph. An undirected graph G(V, E) is a collection of nodes V = {1, 2, . . . , n}, and edges
E = {(i, j)} connecting some pairs of nodes. An independent set of G is a set of nodes I V
such that no two edges in I are connected by an edge; see Figure 2 for examples.