MS&E 321

Stochastic Systems
Prof. Peter W. Glynn

Section 9:

Spring 12-13
June 1, 2013
Page 1 of 7

Renewal Theory

Contents
9.1

Renewal Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9.2

Solving the Renewal Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9.3

Asymptotic Behavior of the Solution of the Renewal Equation . . . . . . . . . . . . .

9.1

Renewal Equations

Let X = (X(t) : t 0) be a non-delayed regenerative process and suppose that f : S R+ .

Then, if
a (t) = Ef (X(t)),
a (t) can be expressed as follows:
Z

Ef (X(t))I(1 ds).

a (t) = Ef (X(t))I(1 > t) +

[0,t]

But
Z

Z
Ef (X(t))I(1 ds) =

[0,t]

E[f (X(1 + t s))|1 = s]P(1 ds)

[0,t]

Z
Ef (X(t s))P(1 ds)

=
[0,t]

Z
=

a (t s)P(1 ds).

[0,t]

In other words, a = (a (t) : t 0) satisfies the linear integral equation

a(t) = b(t) + (F a)(t),
for t 0, where
F (t) = P(1 t),
b(t) , Ef (X(t))I(1 > t),
and

Z
(G h)(t) ,

h(t s)G(ds)
[0,t]

is the convolution operation (that is well-defined for any non-decreasing function G and non-negative
h).
1

 SECTION 09:

RENEWAL THEORY

Definition 9.1.1 Given a function b and a non-negative G, a linear integral equation of the form
a=b+Ga
is called a renewal equation for a non-decreasing function G. Define the n-fold convolution of G via
G(n) (t) = I(t 0)
for n = 0 and

G(n) (t) =

G(t s)G(n1) (ds)

[0,t]

for n 1. Note that when G is the distribution function F of a positive rv 1 ,

F (n) (t) = P(T (n) t),
where T (n) = 1 + 2 + + n , and the i s are iid copies of 1 .
Proposition 9.1.1 The function (a (t) : t 0) = (Ef (X(t)) : t 0) is the minimal non-negative
solution of
a=b+F a
and is given by

a =

F (n) b.

n=0

Here are some additional examples of renewal equations:

Example 9.1.1 Let a (t) = P(T (N (t) + 1) t > x) be the tail probability of the residual life
process T (N (t)+1)t corresponding to a non-delayed renewal counting process N = (N (t) : t 0).
Note that a satisfies the renewal equation
a = b + F a,

(9.1.1)

where
b(t) = P(1 > t + x)
and
F (t) = P(1 t).
The function a is the minimal non-negative solution of (9.1.1) and is given by

a =

F (n) b.

n=0

Example 9.1.2 Let X = (X(t) : t 0) be a non-delayed regenerative process and let a (t) =
P(t T (N (t)) x) be the distribution of the current age process t T (N (t). Here, a satisfies
a = b + F a,
where
b(t) = I(t x),
2

(9.1.2)

 SECTION 09:

RENEWAL THEORY

F (t) = P(1 t).

The function a is the minimal non-negative solution of (9.1.2) and is given by
a =

F (n) b.

n=0

Example 9.1.3 Let X = (X(t) : t 0) be a non-delayed S-valued regenerative process and let
T = inf{t 0 : X(t) A} be the first hitting time if some subset A S. Put a (t) = P(T > t).
Then, a satisfies
a = b + G a,
(9.1.3)
where
b(t) = P(T 1 > t),
G(t) = P(1 t, T > 1 ).
The function a is the minimal non-negative solution of (9.1.3) and is given by
a =

G(n) b.

n=0

Note that G is not a probability distribution function here (since G() = P(T < 1 ) < 1 in
general).
Definition 9.1.2 A renewal equation is said to be a proper renewal equation if G is a probability
distribution function; otherwise, the renewal equation is said to be an improper renewal equation.
In particular, if G(0) = 0 and G() < 1, the improper renewal equation is said to be defective; if
G(0) = 0 and G() > 1, the improper renewal equation is said to be excessive.
Improper renewal equations can frequently be transformed into proper renewal equations via
use of the following trick. Note that if a is given by
a =

G(n) b,

n=0

then
e
a(t) =

e (n) eb)(t),
(G

n=0

where
e
a(t) = et a(t)
eb(t) = et b(t)
e
G(dt)
= et G(dt),
and e
a satisfies the renewal equation
So if we can find such that

ee
e
a = eb + G
a.

e
G(dt)
= e t G(dt)

is a probability distribution function, (9.1.4) becomes a proper renewal equation.

3

(9.1.4)

 SECTION 09:

RENEWAL THEORY

Exercise 9.1.1 a.) Prove that if a renewal equation is excessive, there always exists a unique
e is a probability distribution function.
such that G
e is a
b.) Prove that if a renewal equation is defective, there need not exist a such that G
probability distribution function.

9.2

Solving the Renewal Equation

If F is the distribution of an exponential rv with parameter > 0, then

F (n) (dt) =

(t)n1
exp(t)dt
(n 1)!

so

F (n) (dt) = 0 (dt) + dt.

n=0

Hence,

(F (n) b)(t) = b(t) +

b(s) ds.
0

n=0

So, the renewal equation can be explicitly solved when F is exponential.

It can also be solved in closed form in a limited number of other cases. Let
Z
e
a() =
et a(t) dt,
0

Z
eb() =

et b(t) dt,

e
G()
=

et G(dt),

e are the Laplace transforms of a, b, and G, respectively. If a satisfies

and note that e
a, eb, and G
a = b + G a,
it follows that
e a,
e
a = eb + Ge
so that
e
a=

eb
e
1G

e can be computed in closed form, then a can potentially be computed in closed form
If eb and G
e 1 . For example, this can
by calculating the inverse Laplace transform corresponding to eb(1 G)
e
generally be done when G has a rational Laplace transform (i.e. is the ratio of two polynomials in
).
4

 SECTION 09:

9.3

RENEWAL THEORY

Asymptotic Behavior of the Solution of the Renewal Equation

Typically, the solution to the renewal equation can not be computed in closed form. In such
settings, one usually must be satisfied with computing the limiting behavior of

(F (n) b)(t)

n=0

when t tends to infinity.

Definition 9.3.1 The function
U (t) =

F (n) (t)

n=0

is called the renewal function and the corresponding distribution/measure is called the renewal
measure. If F has a density f ,

X
u(t) =
f (n) (t)
n=1

is called the renewal density. If F is the distribution function of an integer-valued rv 1 , then

un =

n
X

pn(k)

k=1
(k)

(where pn = P (1 + + k = n)) is called the renewal mass function.

We focus first on the case where F is the distribution function of an integer-valued rv 1 . Note
that
un =

P (1 + + k = n)

k=1

= E

I(T (k) = n)

k=1

= EI(regeneration at time n).

It turns out that the asymptotic behavior of un as n can be analyzed via consideration of an
appropriate Markov chain X = (Xn : n 0). In particular, let
Xn = inf{T (k) n : T (k) n}
be the residual life process corresponding to the sequence of event times T (1), T (2), . . .. Note that
un = P0 (Xn = 0),
where X = (Xn : n 0) is an Z+ -valued Markov chain with transition probabilities given by
P (i, i 1) = 1
for i 1, and
P (0, i) = P (1 = i + 1).
5

 SECTION 09:

RENEWAL THEORY

Note that the state 0 is always recurrent for X. Furthermore, X is positive recurrent if and only if
E1 < , in which case is given by
P
P (1 > i)
j>i P (1 < j)
=
.
(i) =
E1
E1
Finally, X is aperiodic if and only if
gcd{k 1 : P (1 = k) > 0} = 1.
If X is aperiodic and E1 < , it follows that
un = P0 (Xn = 0) (0) =
as n . It follows that if

1
E1

|bk | < ,

(9.3.1)

k=0

then the Dominated Convergence Theorem implies that

n

X
X
1 X
unk bk
(F (k) b)(n) =
bk
E1
k=0

k=0

k=0

as n .
Theorem 9.3.1 (Discrete Renewal Theorem) Suppose E1 < and
gcd{k 1 : P (1 = k) > 0} = 1.
If (9.3.1) holds, then
n
X

unk bk

k=0

1 X
bk
E1
k=0

as n . If E1 = and (9.3.1) holds, then

n
X

unk bk 0

k=0

as n .
Let X = (Xn : n 0) be a S-valued non-delayed regenerative sequence and suppose f : S R
is a bounded function (i.e. sup{|f (x)| : x S}). As noted earlier,
an = Ef (Xn )
satisfies a renewal equation, namely
an

Z
= bn +

a (t s)P (1 ds) = bn +

n
X
j=1

[0,n]

anj pj ,

 SECTION 09:
so
an

n
X

RENEWAL THEORY

unj bj ,

j=0

where
bj = Ef (Xj )I(1 > j).
The discrete renewal theorem guarantees that if 1 is aperiodic and has finite mean, then
P
j=0 bj

an
E1
as n . In other words,
P
Ef (Xn )

j=0 Ef (Xj )I(1

E1

> j)

P1 1
j=0

f (Xj )

E1

as n .
Exercise 9.3.1 Prove that if X = (Xn : n 0) is a delayed S-valued regenerative sequence with
E1 < and f : S R is a bounded function, then
P 1 1
f (Xj )
E j=0
Ef (Xn )
E1
as n , provided that 1 is aperiodic.
All the above discussion for discrete-time renewal theory extends to the continuous-time setting.