Poisson Approximation for Some Point Processes in Reliability

Author(s): Jean-Bernard Gravereaux and James Ledoux

Source: Advances in Applied Probability, Vol. 36, No. 2 (Jun., 2004), pp. 455-470
Published by: Applied Probability Trust
Stable URL: http://www.jstor.org/stable/1428462
Accessed: 13/09/2010 06:23

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=apt.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

Applied Probability Trust is collaborating with JSTOR to digitize, preserve and extend access to Advances in
Applied Probability.

http://www.jstor.org
 Adv.Appl. Prob. 36, 455-470 (2004)
Printed in NorthernIreland
@AppliedProbabilityTrust2004

POISSON APPROXIMATION FOR SOME

POINT PROCESSES IN RELIABILITY

JEAN-BERNARDGRAVEREAUX* AND
JAMESLEDOUX,*** INSAand IRMAR,Rennes

Abstract
Inthispaper,we considera failurepointprocessrelatedto theMarkovian arrivalprocess
definedby Neuts.Weshowthatit convergesin distribution to a homogeneous Poisson
process.Thisconvergence takesplacein thecontextof rareoccurrences of failures.We
alsoprovidea convergencerateof theconvergence in totalvariationof thispointprocess
usinganapproach developedby Kabanov, LiptserandShiryaevforthedoublystochastic
Poissonprocessdrivenby a finiteMarkovprocess.
Keywords: Compensator;softwarereliability;
Markovian-arrival process;doublystochas-
tic process
2000Mathematics SubjectClassification:
Primary 60G55;60J25
Secondary 60J75;90B25;93E11

1. Introduction
This work originates in Littlewood's papers [12] and [13] on a Markov-typemodel for
reliability assessment of modularsoftware. Basically, for a piece of software with a finite
numberof modules:
1. the structureof the software is representedby a finite continuous-timeMarkovchain
(CTMC) (Xt)t, where Xt is the active module at time t;
2. when modulei is active,failurestimes arepartof a homogeneousPoisson process (HPP)
with intensity
/(i);
3. when controlswitches frommodulei to module j, a failuremay happenwith probability
g (i, j);
4. when anyfailureappears,it does not affectthe softwarebecausethe executionis assumed
to be restartedinstantaneously.Such an event is referredas to a secondaryfailurein [9].
An extension of such a model was considered in [9], taking into account the influence of
failures on the execution dynamic of the software and dealing with the delays in recovering
an operationalstate. Transientanalysis was providedby means of resultsfrom [10]. Roughly
speaking, the failurepoint process was a Markov-arrivalprocess (MAP) as definedby Neuts
(see e.g. [14]). It is well knownthatwe then obtainas particularinstancesof ourfailureprocess
a phase-typerenewalprocess, a doubly stochasticPoisson process with a stochasticintensity
drivenby a CTMC(also called a Markov-modulatedPoisson process (MMPP)in the queueing
literature),etc.

Received 28 February2002; revision received 27 October2003.

* Postal address:Centrede Math6matiques,INSA, 20 avenuedes Buttes de Codsmes, 35043 Rennes Cedex, France.
** Email address:james.ledoux@insa-rennes.fr

455
456 J.-B.GRAVEREAUX
ANDJ. LEDOUX

An importantissue in reliabilitytheory,specifically for softwaresystems, is what happens

when the failureparametersbecome smallerand smaller. Littlewoodstatedin [12] (or in [13]
for a semi-Markovprocess (Xt)t) that,
as all failure parameters (i), p(i, j) tend to zero, the failureprocess described
above is asymptoticallyan HPP with intensity

X= Q(i, j)(i, j) +
r(i)•
(i)'
i distributionand Q the generatorof the CTMC (Xt)
where 7r is the stationary
(assumed to be irreducible).
This statementis well known in the software-reliabilitycommunityand has been widely used
to justify the hierarchicalapproachto modelling modularsoftware (see e.g. [6] for details).
However,to the best of the authors'knowledge,no proof of this fact is reportedin the applied
probabilityliterature.The aim of this note is to provideprecise statementsand proofs for the
asymptoticof the generalfailurepoint process definedin [9] for which Littlewood'smodel is
a particularcase. Specifically,we show that the countingprocess correspondingto this point
process convergesin distributionto the countingprocess of an HPP when failure parameters
tend to zero but at a specific time-scale. Roughly speaking,we introducea small parameter8
into the failureparametersand the convergencetakes place at time-scale t/e (in other words,
when failure parametersare small and on a large horizon time). Proving this result is easy
using the criterionof convergencein distributiongiven in [8] for instance. It is based on the
convergencein probabilityof compensatorscorrespondingto the variouscountingprocesses.
In fact, the countingprocess convergesin variationto the countingprocess of an HPP and the
convergencerate will be shown using a methoddevelopedin [8] for the MMPP.
Note that the class of MMPPs is widely used to model trafficstreamsfor communication
systems. It is easily seen that dealing with the presentissue is equivalentto consideringan
asymptoticMMPP with a fast modulatingMarkovchain. Thus, we retrievethe more or less
known fact that, when jittering takes place, the arrivalprocess tends to Poissonian (see [15,
p. 116] for a partialdiscussion).
The articleis organizedas follows. Section 2 recalls some backgroundon the point process
studiedhere. In addition,the compensatorof the point process is derivedin a straightforward
manner.In Section 3, we reportresults aboutconvergencein distributionof the point process
to an HPP. Connectionto the problemof fast modulationin the case of an MMPP is briefly
addressedin Subsection3.3. The rate of convergencein total variationof the point process is
given in Section 4. AppendixA recalls an estimate of the convergencerate of the singularly
perturbatedgeneratorprovided in [18]. The derivationof an inequality used in the text is
reportedin AppendixB.

2. Definition of the counting process and compensator

2.1. Definition of the model
We do not reportthe rationaleunderlyingthe definitionof the reliabilitymodel discussed
here;insteadwe referthe readerto [9] for details. Wejust need its mathematicalformulation.
The parameters .) and are as in the introduction. These failures were called
/t(., /t(.)
secondaryevents in [9]. The process X = (Xt)t is an irreducibleCTMC with infinitesimal
generatorQ = (Q(i, j))i,jad, where A( is the finite set {1 ..., M}. The termXt is the active
Poissonapproximation 457

componentat time t for a failure-freesystem, that is, X is the execution process. The vector
a = (a(i))isA denotesthe distributionof randomvariableX0. The new parametersX(i, j) and
X(i) (i, j E AM)are definedto have the same meaningas tg(i, j) and g (i). But, when failure
of this type happensin module i or duringa transitionfrom module i, there is a probability
p(i, k) that execution restartsin module k. So, for each i E NM,(p(i, k))kE is a probability
distribution.Such an eventwas referredas to a primaryfailurein [9]. Simultaneousoccurrence
of primaryand secondaryevents is neglected. For simplicity,we do not considerthe delay in
recoveringan operationalstate as in [9].
Then, taking into account failure occurrences, the random variable Xt* gives the active
componentat time t. The randomvariableNt counts the numberof (primaryand secondary)
failuresin the interval]0, t] (No = 0). Thus, N = (Nt)t is the countingprocess of the failure
point process. Underthe variousassumptionsin [9], the bivariateprocess Z := (Nt, XT)t can
be consideredas a CTMCover the statespace S = N x . Its infinitesimalgenerator,denoted
by G, has the special structure
Do DI 0

G 0 Do DI

using a lexicographicorderon the state space S. The matricesDo and D1 are definedby

IQ(i,j)(1 - X(i,j))(1 - pt(i,j)) ifi j,

Do(i, j) :=
_ Q(i, j) - X(i) -L (i) if i = j,

+1 k)p(ij)+ j) if j,
[(i) Q(i,
k)X(i, j)[1-X(i,
Q(i, j)]pt(i,
+ Q(i,k)X(i,k) p(i, i) + [(i) if i = j.
W.(i)k/i
Note thatmax IG (x, x)I < +oo. The structureof the generatorG shows thatN is the counting
processof a MAP. Finally,X* is a CTMCwith statespaceA, initialdistributiona andgenerator
Q* := Do + D1.
The CTMC X* is supposed to be right continuous with left limits (cadlig). If the failure
parametersare assumedto be such that X(i, j) < 1 for any (i, j), then X* is irreduciblesince
X is. This assumptionis not very stringent.

Example 2.1. (Littlewood'smodel.) Assume thatthereareno primaryfailures,thatis, k(i) = 0

and .(i, j) = 0 for all i, j E M; this is the model of Littlewood. The matricesDo and D1 are
given by
Q(i, j)(1 - p(i, j)) if i
j,
I) = I Q(i, j)Qu(i,
j) if i j, =
Dl(i,
Sr (i) ifi = j.
Furthermore,Q*= Qand we retrievethe fact thatfailuresdo not affect the executionprocess.
458 ANDJ. LEDOUX
J.-B.GRAVEREAUX

Example 2.2. (Markov-modulated Poissonprocess.) Anotherinterestingpoint process is that

obtainedby assumingin Littlewood'smodel thatthe probabilityof a secondaryfailureduring
a control transferis 0. In this case, setting A(i, j) = 0 for all i, j E N in the previous
expressions,we get Do = Q - diag(A(i)) and D1 = diag(A(i)). This is an MMPP.
Note that Z' = (X*, Nt)t is also a Markovprocess with homogeneous second component
as definedin [5], or a Markov-additiveprocess of arrivalsdiscussed in [16].
2.2. Compensator and intensity of the counting process
The basic facts on point processes, martingales,compensatorand intensity used in this
paperarereportedin [2]. A nice surveyon point processes is [17]. For any process V = (Vt)t,
7(V) = (!(V))t will denote its internalhistory,i.e. t(V) := r(Vs, s _ t).
Consideringthe bivariateprocess Z in orderto analyse the countingprocess N allows us
to deal with a Markov process with discrete state space. Thus, we can take advantageof
the powerful analytic theory and the computationalmaterialdeveloped for such a class of
processes. This fact was exploitedin [10] and [9] to assess variousreliabilitymeasuresrelated
to the transientbehaviourof the countingprocess. Due to the special structureof the generator
G of Z, N may be interpretedas the counterof specific transitionsin Z. More precisely,if we
are interestedin the set

T := {((n, i); (n + 1, j)), i, j E A, n > 0}

of pairsof statesin S, then

Nt= 1{Zs_=x,Zs=y}
(x,y)ET O<s<t

- Nt(x, y). (2.1)

(x,y)E T

In otherwords, Nt counts the transitionsfrom statex to y (x, y e T) in the interval]0, t]. It is

well known (see e.g. [2]) that, for a counterN(x, y) = (Nt(x, y))t of transitionsfrom statex
to y in a Markovchain,
Nt(x, y) - G(x, y) ds
l{zs_=x}
is an F-•z) = (Ft())t martingale.The randomfunction

Xt(x, y) := 1{Z,_=x}G(x, y) (2.2)

is the F•(Z)-intensityof N (x, y) and

At(x, y) j s (x, y) ds

is the ?(z)-compensator (7F(z) dual predictableprojection)of N(x, y). Then it is easily seen
that M = (Mt)t with Mt = Nt - At and

At
= j At(x,y)
(x y) ~ T
Poissonapproximation 459

is an $(z)-martingale. The F(Z)-intensity of N is

t := l{z,_=x)G(x,y)
(x,y)eT

= X X X G((n, i); (n + 1, j)) 1{(Nt_,X_)=(n,i)1

n>O ie~Mje~M

= X X DI(i, j)
1{(Nt_,X*_)=(n,i)}
n>O0ieM jeM

= = D(X*,
lxt=i)[ Dl(i, j)] j).

In the case of Example 2.1, the intensity of the countingprocess correspondingto Little-
wood's model is
AM(X_)+ Q(Xt-, j)(Xt-,
jAxt- j).
In the case of Example2.2, we retrievethe well-knownexpressionfor the $(N,X)-intensity
for an MMPP (Xt-).

3. Convergence of the counting process

3.1. What does it mean for the failure parameters to be small?
A basic way to representsmallerand smallerfailureparametersis to multiplyeach of them
by a scalare and to investigatethe behaviourof the countingprocess N as e tends to 0. Thus,
we considerthe perturbatedfailureparameters

eX(i), ex(i, j), ep(i), eL(i, j), i, jeEM. (3.1)

Let us considerthe example of an MMPP (see Example2.2) with associatedmatrices

Q=D(1 1 (1) 0
= -1 )' 0D1 (2)

(where A(1) :A(2)) and with a = x = (, 1) (i.e. with a stationaryenvironment). The

matrixDo is#
=
D(E)
o = (0
Q- (2,)(1)0E /4 2)E8
If T is the time to firstfailure,then

P{T > t} = P{Nt = 0} =-r exp(lD()t) 1T,

where 1 = (1 ..., 1). Settinga = 4 + - g(2))282, we have

(/(1)

reD
t
1T
=
[cosh
2
sinh t
( e-te-((1)(1)+(2)(2))et

As expected,P{T > t } convergesto 1 as s tends to 0. Therefore,convergencein distributionof

the counting process to an HPP,i.e. weak convergenceof the finite-dimensionaldistributions
460 ANDJ. LEDOUX
J.-B.GRAVEREAUX

of N to those of an HPP, cannot take place at the currenttime-scale. If we investigatethe

asymptoticdistributionof T at time-scalet/s, then,fromthe previousexpressionfor P{T > t),
lim P{T > t/s) = exp(-(t(1)7r(1) + g(2)zr(2))t).

We will thereforedeal with the countingprocess N(E)= (N('e))tdefinedby

NE) = Nt/e,

whereNt countsthe numberof failuresin the interval]0, t] for the reliabilitymodel of Section2
with the system (3.1) of perturbatedfailure parameters. The matrices D(0e, D E) are those
associatedwith the model N. Note that

D1e) = EB + E2L, (3.2)

with
-Q(i, j) if j 0 i,
L(i, j) = L(i,) j))g(i, j)(i,
O if j = i,
+ 1 ] Q(i, k) X,(i,k) ]p(i, j) + Q(i, j) L(i,j) if ]j 0 i,
B(i, j)= LI
tF)(i)
k?i if j = i.
DDi(i,i)
Note that B is a nonnegativematrix.The Markovprocesses X* :- and X**E:- (X*/)t
(Xt)t
have generatorsQ* = Doe)+ -)" and Q*.E:= Q*/s respectively.
3.2. Convergence of compensators
of N(E)is
Using the developmentof Subsection2.2, the F(N(E),'X*')-compensator

A() D (X*_,j) ds

-=
f B(X*_,j) ds + 82 L(Xj, j) ds (from(3.2))
jEeM jEiM

B(i, j) E2 L(i, j). (3.3)

ds + 1{X*=i} ds1jE
iieM =EEl{X_=i} jeM
ie"
Since X* is chdlg, X*(w) = (w) for almostall w e 2, except for countablymanys. Thus,
we can replace X* with X*X>_ in the above integrals. This observationwill be used to equate
similarintegralsthroughoutthis paper.
Now, let us considerLittlewood'smodel as given in Example2.1. Failuresdo not affect the
executionprocess, thatis, X* = X and Q* = Q. Thus,

A(E) - e ds C B(i, j).

l{Xs=i}

It follows from well-knowntime-averagepropertiesof the cumulativeprocess f f (Xs) ds for

an irreducibleMarkovprocess X (see [3]) that (8/t) l(xi) ds converges almost surely
foi
Poissonapproximation 461

to 7r(i), where 7ris the stationarydistributionof X. Thus,

Ae) -+
t r(i) 1 B(i, j) = Xt almost surely as E -- 0,
iEM j]EM

with Xas in (1). In particular,this implies thatA) convergesin probabilityto Xt. We recognize
the compensatorof an HPP with intensityX. It follows fromTheorem 1 of [7] that

as e -+ oo,
N() -- P
where P = (Pt)t is the counting process of an HPP with parameterX. We have shown that
Theorem 3.1 below holds for an MMPP and Littlewood's reliability model (Examples 2.1
and 2.2).
Since convergencein the L2-normimplies convergencein probability,the following lemma
will give the convergencein probabilityof compensator to Xt as e tendsto 0 for the general
reliabilitymodel of Section 2. Ate)

Lemma 3.1. If theprobabilityvector 7r is such that nrQ = 0, then

limE e] -r(i))ds = 0.
e--+0 0O
(1{xs=i}
Proof The perturbatedgeneratorQ* = D) + Di) of X* can be decomposedas follows:
Q* = Q + Re, (3.4)
where

Ix(i) + k)X(,
k)(i p(i j) - Q(i, j)X(i, j) if j i,
Q(i,
R(i,j)= ki
X(i) + E Q(i, k)X(i, k) p(i, i) - X(i) if j = i.
I k~i
The change of variablesu = se gives that

e1j ds= ds.

l{X=i) l{X*,=i}
Recall that (Xt'e)t has Q*/e = Q/e + R as generator.It is easily checked that R 1T = 0. In
sucha case,CorollaryC.1of [18,p. 349]givestheestimate
t 2
E] ( -r(i)) ds
l{x*,E=i}
< C(1 + t2)e.
The convergenceof fO1lX*,E=i}ds to 7r(i) in the L2-normas E tends to 0 follows from the
previousestimate.
The following theoremfollows from [7, Theorem 1].
Theorem 3.1. Theprobabilityvector r is such that r Q = O. As E tends to 0, the counting
process N(e) = (Nt/e)t converges in distributionto the counting process of an HPP with
intensity
= (i) 1(i) + ~Q (i, j)[(i, ) + (i, )]. (3.5)
i j1/
i~~M
462 J.-B.GRAVEREAUX
ANDJ. LEDOUX

3.3. Fast modulation in an MMPP

When we consideran MMPP (see Example2.2), the previousissue is equivalentto investi-
gatingthe asymptoticof a Poisson process with an intensitymodulatedby a fast Markovchain.
Indeed,the compensatorof N(e) is then

A(8) A=
- (Xs) ds

- j ds (with the change of variableu = Es).

(Xs/p_)
The Markov chain X(E)= (Xt/E)t has the generator Q/s. As E tends to 0, it is now clear
that the introductionof the small parameterE speeds up the rate of switching between states.
Theorem3.1 statesthatthe asymptoticprocess is an HPP with intensity

This fact is known and has been investigatedfor various applications. The closest context
to reliability theory is [1], where the underlyingPoisson process is either homogeneous or
nonhomogeneous.Proofsarebasedon asymptoticexpansionof the transitionsemigroupof the
bivariateMarkovprocess Z(e). Introductionof time-dependentintensityis not relevantin our
context. Indeed, the decrease of failureparameters,or the reliabilitygrowth,is alreadytaken
into accountby the small parametere.
The case where the modulatingprocess is a finite nonhomogeneousMarkovprocess X is
addressedin [4]. The asymptoticprocess is a nonhomogeneousPoisson process with suitable
ergodicityassumptionson X.
From a strictlymathematicalpoint of view, the issue addressedin this paperis equivalent
both to a fast modulatingMarkovprocess X* (with generatorQ*/E) and to the introduction
of a small scalare only in the failureparametersA(., .) correspondingto a transitionbetween
states. Speeding up the modulatingMarkovprocess X* at rate 1/E implies a speeding up of
the numberof transitionsbetween states of X*. Therefore,we have to compensatefor this
'explosion'by introducinga small factorE in /(., .).

4. Convergence rate
We provide in this section an estimate of the convergencerate of the finite-dimensional
distributionsof N(E)to thatof an HPP with intensityas in Theorem3.1. The countingprocess
of the HPP is denotedby P = (Pt)t. Note that Xin (3.5) is the scalarproduct(.r, B 1T). We
can write the limit compensatoras
At = (7r, B 1T)t. (4.1)
From (3.3), the F(N('), X*,E)-compensatorof N(e) is

A(E) -
i (B + EL) 1T(i) dO
ds
'l{Xs*_=i}

- L(B + EL)
1T)(i)
] ds
(setting u =
sE)
l{x.•=i}

- (B + L) 1T) ds, (4.2)

(Y_,
Poissonapproximation 463

where Ys(E):= (lX*,e=i))ijE . Hence, the

of N(e) is
YT(N(0')x*()-intensity
se) := (Ys(), (B + eL) 1T). (4.3)
Let T be any positive scalar and T := {to, t, ...., tn,}with 0 = to < t1 < -.. < tn
T. To evaluate proximity between the respective distributions?(Nr')) and ?(PT) of
(E) (E) Nt)(E)) and PT := (Pt,,
:=
Nj(Nt,.,_ Pt,), the distance in total variation, denoted
by dTv(.C(N(j)), ?Z(PT)),may be used, that .....
is

dTv(f(N)), ?Q(PT)) sup IP{N(E)E B) - P{Pr E B)}

BCNn

IP{N) = k}-P{PT= kI
kENn

(see [17]). For a locally boundedvariationfunctiont f (t), the total variationin the interval
[0, T] is
n

var[o,Tl(f) := sup f (t) - f (ti-1)

{ti,...,tn}jEP([O,T]) i=1

where P ([0, T]) is the set of all finite subdivisionsof the interval[0, T].
In this section, we show that the finite-dimensionaldistributionsof N(E) convergesin total
variation to those of an HPP with intensity X at rate e. The proof is borrowed from [8,
Theorem 6.1], where a similar result is given for an MMPP. This is heavily based on the
following estimate of the total variationbetween finite-dimensionaldistributionsof N(e) and
P [8, Theorem3.1]:

< - A),
dTV(G.(NE)),Z(PT)) Evar[o,T](A(e)
where A(e) is the F(N(*))-compensatorof N(e). The term A(e) is obtainedfrom (4.2) and

(t) j (Ys•) (B + eL) 1) ds

with
I(P{X'e = i N(e)))iEA
(see [11, Theorem 18.3]). Hence, the F(N(8))-intensityof N(e) is

E) ((S8) (B + eL) 1T). (4.4)

We thereforeobtainfrom (4.1) and (4.2) that

dTv((NNE)), X
?(P)) E [,
var[0,T]( (s(e) - B 1T) + e (s(), L 1T) ds

= E I((e) - r, B 1 ) + L 1 )Ids
e(Ys_),
<E - r, B 1T) Ids + eE I(s), L 1T) ds
I(_(e)

< E
0IT
I - r, B1T)I ds + eCT
(Ys(e) (.5
(4.5)
464 J.-B.GRAVEREAUX
ANDJ. LEDOUX

(since Y(E)is bounded). Since and 7r are stochasticvectors, it follows from (B.1) below
that
is
7 B - *,r(4.6)
1(Ys - r, 1T)[ < 2 11
where 3 := max(B 1T(i)) - min(B 1T(i)) and II IIIis the li-norm. Hence, it remains to
estimatein (4.5) the convergencerate of 1Iis•- Fr Ii to 0 when e -- 0. The firststep consists
in writinga filteringequationfor the vector Y0
4.1. Filtering equation for y(E)
)
Recall that =
(l{X*,e.=i})iE(. Note that each componentY (i) of Y( is a bounded
randomvariable. We now basically follow ChapterIV of [2].
YtE)
Lemma 4.1. Define Y E[Y(e) Let a be the probabilitydistributionof
Then,for all t > 0, ()]. Xof*.

It
jd8) =a +- i(t
Y) 1 Tf Q*ds +
v3)*
Vs(dN - X<ds)(4.7)
-
ds),
(4.7)

o ?(dN4s)
where (s) is the of N(') given in (4.4) and
F•(N(E))-intensity

() ~•(s) (B + eL) ( (4.8)

Proof Recall thatthe Markovprocess X*,' has the generatorQ*Q'= Q*/E. It follows from
the Dynkin formulathat

Y8) Yo)+ Y Q* ds + Mt, (4.9)

where M = (Mt) is an F(*x*'E)-martingale. Then, applyingTheorem 1 of [2, ChapterIV] to

the representation(4.9) of the boundedprocess y(E),we get that

^ *
1 rt
s ) Q ds +M,
0. =Y (6. +-
Yt f0
I
where M = (Mt) is an F(N('E)-martingale. Now, Theorem 17 of [2, ChapterIII] gives us the
following representationof the F(NE))E-martingaleM:

Gs(dN
(e) -
)ds),
where •(E) is the 3F(N() -intensity of N(W)and G = (Gt) is an .F(N E)-predictable process
called the innovationsprocess. We also know from Theorem2 of [2, ChapterIV] that Gt
- + G3,t, where the ith entryof the vectors G1,s and G3,s must be computedfrom
Gi•t )

E (i)X8")ds = E ) ds, (4.10)

CsYs CsGls(i)•~(
E 3 = EJ
CsG3,s(i)4 )ds, (4.11)
O<s<t CsAMs(i)AN(s)
Poissonapproximation 465

where C := (Cs)s is any nonnegativeF(N(e )-predictable process and AMs(i) = Ms(i) -

and AN(e) = are the jumps of martingaleM(i) and the countingprocess
Ms_(i) Nse) - N(
N(e) respectively.Therandomprocessesh(E)andX(E)aregiven by (4.3) and(4.4). Wedetermine
an explicit expressionfor G 1,tand G3,t. This is similarto the MMPPcase (see [2, p. 98]) except
that X*,' and N(e) may have commonjumps.
The left-handside of (4.10) can be rewrittenas

EJ CsY (i)•• ds -
E Y
(i)X0•• s
f(00T
Cs
= E CsY(s)(i)(Y E, (B + eL) )ds (from(4.3))
ft
= E CsY()(i)((B + eL) 1T)(i) ds

= E (i)((B + eL) 1T)(i) ds

CsY(E)
since Cs is Fs(N()) -measurable.We then deduce from (4.10) that

(E)fY(i)((B + eL) 1T(i)

G),s=
The countingprocess N(e) has the following representation(see (2.1)):
+oo

Nt(e)= ((n, j), (n + 1, k)),

Nt(e) (4.12)
n=0 jeAA kEAM

where Nt() (x, y) is the cumulativenumberof transitionsof the bivariateprocess (N(e), X*,e)
up to time t. Since AMs(i) = A Ys) (i),

CsAMs(i)AN(e)-- CsYs(e)(i)AN(e)- CsY (4.13)

O<s<t O<s<t O<s<t (e)(i)ANse)".
The firsttermon the right-handside of (4.13) can be rewrittenusing (4.12) as
+oo

CsYs(e)>(i)AN(e)
- CsYs(e)(i)- AN e)((n, j), (n + 1, k)).
O<s<t O<s<t n=0 jEAt kEAM

It is easily seen thatthe right-handside of the last equalityis

+oo

CsYY Y(s()j)ANs(e)((n, j), (n + 1, i)).

O<s<t n=0 jAE

So, from (4.13),

(i)jAN(e)
=j 3 Y2(j)dN~E)((n, j), (n + 1, i))
CsAMs 0 Cs
O<s<t n=0 jEA
- CsjYs(
C2(i) dNe).
466 J.-B.GRAVEREAUX
ANDJ. LEDOUX

Since the processes C and y(e) are F(N('),x*,E)-predictable, it follows from the definitionof
the compensator,(2.2) and (4.3) that

E L CsAMs(i)ANe)
O<s<t

= EE Cs
Y i) + eL(j, i)) - Y- (B + eL) 1T) dsds
c[ C ((j)(B(j, i)
Y-(j)(B(j, eL(j, i)) Ys (i (B +Li
(i)(Y5s_),
EEJ
= s (B (j, i) + EL(j, i))- Y (i) (B (i, j) + eL (i, j)) ds
Ys-(j)
jeM
je:
=E Cs j Ei Yf2(j)(B(j, i) + eL(j, i)) - Y i
i) j (B(i, j)) ds

jii Y [
(j)(B(j, i) + eL(j, i)) - f(-s0) j]i + eL(i, j))]ds
(i)L:(B(i,ij)
=sEE

since Cs is Fs(E)) -measurable.We then get from (4.11) that

i) + i)) - Y'- (i) -1~ i(B(i, j) +eL(i, j))

Ys('(J)(B(J, eL(j,
G3,s(i) = i•

Consequently,the gain Gs has the form given in Lemma4.1.

Lemma 4.2. Let a be theprobabilitydistributionof X *. Then(4.7) has the uniquesolution

't(= ta exp(Q*t/) 1 v(E exp(Q*(t - - X) ds). (4.14)

s)/e)(dN(sE)
Proof First,let us check that(4.14) is a solutionof (4.7). It has the form UtVt, where

Ut = a + V(E -(e) ds)

exp(-Q*s/E)(dNse)
Vt = exp(Q*t/e).

Using integrationby parts[2, Theorem2, p. 336], we obtain

t t
l
= a + 1t - (UsVs)Q*ds (dN ds).
-+
Second,notethat(177)- UrVt)t is a solutionof thehomogeneouslineardifferentialequation

y(t)= - y(s)Q* ds

with initial conditiony(O)= 0. Then it - Vt 0.

Ut =
Poissonapproximation 467

4.2. Convergence rate

Theorem 4.1. Theprocess P = (Pt)t is the countingprocess of an HPP with intensity

X
= (r, B 1T) = 7r(i)[I(i)+ Q(i,
X(i)+ E j)Q(Q(i, j) + L(i, j)),

where 7r is the probability distributionsuch that r Q = 0. For any T > 0, there exists a
constant CT such that
dTv(?(N(e)), ?(PT))
< Cre.
Proof Let us recall that

E
t
dTv(?(NTe)), ?(P)) 11j
-
7rillldt+ C1,e

(see (4.5) and(4.6)). Wejust haveto controlthe firsttermon theright-handside of the inequality.
Since v(e) 1T = 0, using (4.14) we can write

-
t(e) r = aexp(Q*t/e) - + (e)[exp(Q*(t - s)/e) - 1
r](dNs()e)
ds).

Using the triangleinequality,it is easily seen that

E II) - rIllldt < f Ia exp(Q*t/e) - r111

dt

t -
+E v(e)[exp(Q*(t s)/e)
-
1r r](dNs() - (e)ds) dt.

(4.15)

In a first step, let us considerthe firsttermon the right-handside of the previousinequality.

We have from (A.2) below with Q(E)= Q*/E = Q/e + R (see (3.4)) that,for all s > 0,

Ilaexp(Q*t/e) - r I1 < C1(E + exp(-pt/e)).

Then

T Ila - 7r11dt CiT +-e = C2,TE. (4.16)

exp(Q*t/e) <

In a second step, we have

E - - 1T
v(E)[exp(Q*(t s)/e) r](dN(e) - (e) ds)

h(E)ds)
<E Iv(')[exp(Q*(t - s)/e) - 1 r]IIl(dN(E)+

= 2E II [exp(Q*(t - s)/e) - 1 r]Jll^ ) ds

J0t
v_
468 J.-B.GRAVEREAUX
ANDJ. LEDOUX

since j1) is the (N''E))-intensity of N(E) and v(E is F(NE))-predictable. We recall that
I|xMII < 1lxlM for anyvectorx andmatrixM, where IMliM
lIIMII := maxi()j IM(i,j)1)
is the matrix 1-norm.Now,

[exp(Q*(t - s)/e) - 1T iexp(Q*(t - s)/e) - 1Tr 1.

1vs(E r][I v<1

It follows from (A.1) below that,for all s < t,

IIexp(Q*(t - s)/e) - 1T i + exp(-(t - s)p/e)).

< C2(e
Since v(E)is the difference between two probabilityvectors, Il <2. Using (4.4) and
Then Ilvse)
II()l - 1, weget IIv 11 21(B + L) 1 I.

I II1E) C3+ C4e

vs_ _<
is uniformlyboundedin s. Therefore,for all t > 0, we can write

E - s)/e) -_1Tr]l1k) ds
IIvs_[exp(Q*(t
S(C5+ EC6) f( + exp(-(t - s)p/e)) ds

< (C5 + eC6)(te + C7e)

< (C8t + C9)e (fore < 1).

We deduce from the previous estimate (and Fubini's theorem) that the second term on the
right-handside of (4.15) is such that

(0
(exp(Q*(t - s)/e)
E - ) dt
fT 1T r)(dN) - ds)
_
< (Cat+ C9)edt = C3,TE (fore < 1).

Theorem4.1 follows from (4.15) with the last inequalityand (4.16).

Remark 4.1. Withrespectto ourTheorem3.1, note thatTheorem1 of [7] would, in fact, give
convergencein distributionof the countingprocess N(W)to the Poisson process P in the space
of all counting processes equippedwith the Skorokhodtopology. Moreover,convergencein
variationalso takes place in this space. Indeed,the distancein total variationover the interval
[0, T] betweendistributionsof N(E)and P is also boundedfrom aboveby E var[o,r](A(E)- A)
(see [8, Theorem4.1]). Thus, it follows from Theorem4.1 thatthe rate of convergenceis e.

Remark 4.2. The orderof the convergenceratein Theorem4.1 cannotbe improvedin general.
This follows from [4, Section 5, Example 1], where the authorsreporta lower bound for the
distancein variationthathas order1 in e for a Poissonprocessmodulatedby a two-stateMarkov
process.
Poissonapproximation 469

AppendixA. Estimateof convergencerate

Consider a generatorQ(E)= Qi + Q2/e, where Q2 is assumedto be an irreduciblegen-
erator. There exists a probabilityvector 7r such that r Q2 = 0. The following estimates are
providedin [18, Lemma C.3, p. 346]: for all t > 0 and any probabilityvector oa,there exist
constantsK and p > 0 such that

IIexp(Q(")t) - 1T
K(e + exp(-pt/e)),
7rill (A.1)
-
Ilatexp(Q(e)t) xlr1 < K(e + exp(-pt/e)) (A.2)

and p only dependsupon Q2

Appendix B. A simple inequality

Let v be any M-dimensionalvector. Any convex combinations(ul1, v) and (u2, v) of scalars
(v(i), i = 1, so I(ul - u2, v)1 < 8 with
M) are in the interval [min(v(i)), max(v(i))],
8 := max(v(i)).... - min(v(i)).
Let al and U2 be two stochastic vectors. Write al - U2 = - U2)+ - - U2)-,
(ol (ol
where w+ = (max(w(i), 0))i=1,... M and w- = -(min(w (i), 0))i=1,... M for any vector w =
((i))i=l,...,M. Since - a2) 1T = 0,
(otl

Ilil -
ot2
Il = (otl - U2)+ 1T +(otl - o2)- 1T = 2(otl - a2)+ 1T = 2(aot- a2)- 1T,
where II illiis the 11-norm.We may then write the vectoral - as (ul - u2)IIcll- U2111/2,
o2
where u i, u2 are the stochasticvectors

- - 2)
(atl (otl
Ul o2)+ 1T'
a2)+ U2-
(at - (l - 2)- 1T'

Next, it follows from the firstpartthat

- 2 8
- = 10li 2111I(Ul - ( - (B.1)
I(Oti U2, 1V)I 2 u2, V1)1 2 Illil 22111.

References
[1] ANISIMOV, V. V. ANDSITYUK, V. N. (1979). Asymptoticbehaviorof inhomogeneousPoisson flow with conductor
function,dependenton a small parameter,controlledby a Markovchain. Cybernetics15, 526-532.
[2] BREMAUD,P. (1981). Point Processes and Queues. Springer,New York.
[3] CINLAR, E. (1975). Introductionto StochasticProcesses. Prentice-Hall,EnglewoodCliffs, NJ.
[4] DI MASI,G. B. ANDKABANOV, Y. (1993). The strongconvergenceof two-scale stochasticsystems and singular
perturbationsof filteringequations.J. Math.SystemsEstimationControl3, 207-224.
[5] Etov, I. I. ANDSKOROKHOD, A. V. (1969). Markovprocesses with homogeneoussecond component.II. Theory
Prob.Appl. 14, 652-667.
[6] GOSEVA-POPSTOJANOVA, K. ANDTRIVEDI, K. S. (2001). Architecture-basedapproachto reliabilityassessment
of softwaresystems. PerformanceEvaluation45, 179-204.
[7] KABANOV, Y. M., LIPTSER,R. S. ANDSHIRYAEV, A. N. (1980). Some limit theoremsfor simple point processes
(martingaleapproach).Stochastics3, 203-216.
[8] KABANOV, Y. M., LIPTSER,R. S. ANDSHIRYAEV, A. N. (1983). Weakand strongconvergenceof the distributions
of countingprocesses. TheoryProb.Appl. 28, 303-336.
[9] LEDOUX, J. (1999). Availabilitymodeling of modularsoftware.IEEE Trans.Reliab. 48, 159-168.
[10] LEDOUX, J. ANDRUBINo,G. (1997). Simple formulaefor countingprocesses in reliabilitymodels. Adv.Appl.
Prob. 29, 1018-1038.
[11] LIPTSER,R. S. ANDSHIRYAEV, A. N. (2001). StatisticsofRandom Processes, Vol. II, 2nd edn. Springer,Berlin.
470 J.-B. GRAVEREAUX AND J. LEDOUX

[12] LITTLEWOOD, B. (1975). A reliabilitymodel for systemswith Markovstructure.J. R. Statist.Soc. C 24, 172-177.
[13] LITTLEWOOD, B. (1979). Softwarereliability model for modularprogramstructure.IEEE Trans.Reliab. 28,
241-246.
[14] NEUTS,M. F. (1989). StructuredStochasticMatrices of M/G/1 Typeand TheirApplications.Marcel Dekker,
New York.
[15] NEUTS,M. F., LIU, D. ANDNARAYANA, S. (1992). Local Poissonificationof the Markovianarrivalprocess.
Commun.Statist.Stoch.Models 8, 87-129.
[16] PACHECO, A. ANDPRABHU, N. U. (1995). Markov-additiveprocesses of arrivals.In Advancesin Queueing,ed.
J. Dshalalow,CRC Press, Boca Raton,FL, pp. 167-194.
[17] SERFOZO, R. F. (1990). Pointprocesses. In StochasticModels (HandbookOperat.Res. ManagementSci. 2), ed.
D. P. Heymanand M. J. Sobel, North-Holland,Amsterdam,pp. 1-93.
[18] SETHI, S. P. AND ZHANG, Q. (1994). HierarchicalDecision Making in Stochastic ManufacturingSystems.
Boston, MA.
Birkhdiuser,