The Role of the Group Generalized Inverse in the Theory of Finite Markov Chains

Author(s): Carl D. Meyer, Jr.

Reviewed work(s):
Source: SIAM Review, Vol. 17, No. 3 (Jul., 1975), pp. 443-464
Published by: Society for Industrial and Applied Mathematics
Stable URL: http://www.jstor.org/stable/2028885 .
Accessed: 07/12/2011 16:49

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

Society for Industrial and Applied Mathematics is collaborating with JSTOR to digitize, preserve and extend
access to SIAM Review.

http://www.jstor.org
SIAM REVIEW
Vol. 17, No. 3, July 1975

THE ROLE OF THE GROUP GENERALIZED INVERSE IN

THE THEORY OF FINITE MARKOV CHAINS*
CARL D. MEYER, JR.t

Abstract.For an m-statehomogeneousMarkov chain whose one-steptransitionmatrixis T,

thegroupinverse,A#,of thematrixA = I - T is shownto playa centralrole.For an ergodicchain,
it is demonstratedthat virtuallyeverythingthat one would want to know about the chain can be
determinedby computingA#.Furthermore, it is shown thatthe introductionof A# into the theory
ofergodicchains providesnot onlya theoreticaladvantage,but it also providesa definitecomputa-
tionaladvantagethatis not realizedin the traditionalframework of the theory.

1. Introduction and notations.The theoryof finitehomogeneousMarkov

chainsprovidesone ofthemostbeautifuland elegantapplicationsofthetheoryof
matrices.In recentyears,one ofthemostactiveareas ofresearchin thetheoryof
matriceshas been the developmentof the theoryand applicationsof generalized
inversionofmatrices.The purposeofthisarticleis to perform a marriagebetween
the theoryof finiteMarkov chains and the theoryof generalizedinversionof
matrices.
This is not a newidea. Decell and Odell [1], [5] proposedtheintroduction of
generalizedinversesinto thetheoryof Markov chains.However,theyconcerned
themselveswith only the Moore-Penrose inverseor else a generalizedinverse
whichwas definedbya subsetofthefourPenroseconditions.These are thewrong
typesofgeneralizedinversesto use. Theyare "forced"intothetheorybecause of
theirabilityto produce solutionsof consistentsystemsof linearequations.The
resultsthatDecell and Odell obtainedby usingthemare verylimitedand seemto
lead to a dead end as far as the theoryof finiteMarkov chains is concerned.
Furthermore, any real computationaladvantagethatthe Moore-Penroseinverse
affords is highlyquestionable.Experiencehas shownthatthecomputationswhich
involvecomputingthe Moore-Penroseinversecan be quite unstable.
The correctgeneralizedinverseto use in connectionwithfiniteMarkovchains
is thegroupinverse[3], whichis a special case of the Drazin inverse[2]. In this
paper,it willbe shownthatthegroupinverseplaysa centralrole in thetheoryof
finiteMarkovchains.If Tis theone-steptransitionmatrixofa finitehomogeneous
Markovchainand ifA = I - T, it willbe shownthatonce thegroupinverse,A#,
ofA is known,thentheanswerto everyimportantquestionconcerningthechain
can be obtainedfromA*. In additionto formulating thetheoryoffiniteMarkov
chains in a naturalway in termsof A , we will show that A' providesa very
practicalaspect to the subject.The introductionof A * will providea definite
computationaladvantagethatis not realizedin thetraditionalframework of the
theory.
The textby Kemenyand Snell [4] has been chosen as a guide,and their
erminology(but not theirnotation)will be adopted in this paper. One of the
objectivesof thispaperwillbe to showthatforan ergodicchain,thematrixwhich

* Receivedby the editorsFebruary22, 1974,and in revisedformSeptember30, 1974.

t Departmentof Mathematics,NorthCarolina State University,Raleigh,NorthCarolina 27607.
443
444 CARL D. MEYER, JR.

Kemenyand Snell call the "fundamentalmatrix"need neverbe introducedand

thatwherever the"fundamental replacedbyA -*
matrix"appears,itcan be directly
The followingnotationsand definitions will be used throughoutthispaper:
E a field;
C thefieldofcomplexnumbers;
m,
xn __the set ofm x n matriceswhose entriesbelongto .
For Xe Crxmr:
Xi -the elementin theithrow and jth column;
X' thetranspose;
X* theconjugatetranspose;
Xdg the diagonal matrixobtained fromX by settingall elementsoffthe
main diagonal equal to 0;
N(X) thenullspaceofX;
p(X) thespectralradiusofX;
a(X)-the spectrumofX;
00--thenumbermaxiZk IXjkl.
X 11
11
For a Markovchain:
T theone-steptransitionmatrixofan m-statehomogeneousMarkovchain;
I an identitymatrix;
ei theithcolumnofI;
A thematrixI -T;
j a columnvectorin whicheach componentis equal to 1;
J thematrixj';
w' thefixedprobability(row)vectorforthetransitionmatrixofan ergodic
chain;
W thematrixI - AA#, whereA#is givenin thefollowingdefinition.
DEFINITION. For A cFm
E xm,thegroupinversefor A, whenitexists,is definedto
*
be thematrixA whichsatisfiestheconditions
AA#A= A, A#AA#= A# and AA#= A#A.
For A E E9mXm,it can be shown(see [3]) thatthethreeequationsAXA = A,
XAX = X, and AX = XA possess a solution for X if and only if rank(A)
rank(A2).Furthermore, whenit exists,thesolutionis unique.
The term "group inverse" is used because if A E ,9mXm is such that rank(A)
rank(A2),thenitis wellknown(see [10,p. 273]) thatA belongstoa set (?i?c mxm
such that N formsa multiplicative group underordinarymatrixmultiplication.
Thus A has a multiplicativeinverse,Ag,withinI. This matrixAgis preciselythe
matrixA * definedabove and theidentityelementofNis theprojectorAA' = A #A.
The groupinversehas been thesubjectofstudyin [3], [6] and [7]. The reader
whois notfamiliarwiththesubjectofgroupinversescan obtainall ofthenecessary
backgroundinformation by consultingany oftheabove threearticles.
DEFINITION. For A E Cmxx , the Moore-Penroseinverse forA is thematrixAt
whichsatisfiestheconditions
AAtA = A AtAAt = At (AAt)* = AAt and (AtA)* = AtA.

If A - is a matrixwhich satisfiesonly the firstcondition, i.e., AA - A = A, then A - is

said to be a (1)-inverseforA.
 THE ROLE OF THE GROUP GENERALIZED INVERSE 445

For everymatrixA E C'mX n, At E ?;n X m alwaysexistsand is unique.In general,

(1)-inversesare not unique. There is an extensiveamount of materialavailable
concerningthe Moore-Penrose inverseand (1)-inverses.The interestedreader
can consult[11] or [12] and the references givenin thesetexts.
2. Existenceof A #; Limitsand the fixedprobability vectorin termsof A *.
The firstfundamentalfactthat we need to establishis that foreverytransition
matrixT,thematrix(I - T) #alwaysexists.This can be accomplishedbyusingthe
factthateverystochasticmatrixhas onlyfirstdegreeelementarydivisorscorre-
spondingto theeigenvalue1.(See [9,p. 84].) However,itis possibleto givea simple
and directproofby using propertiesof the group inverse.We firstneed a pre-
liminarylemma.The resultmentionedabove concerningthe elementary divisors
correspondingto theeigenvalue1 willthenfollowas a corollary.
LEMMA 2.1.1 SupposeM E Em m has theform

M 1+_
Kj_
-u
_O L_,
c-

and ifL * exists,thenM exists.

whereU and L are square.If U is nonsingular
Proof.Let
[U-I U-2C(I - LL#) - U-lCL# 1
LO L#
It is veryeasy to verifythat MXM = M, XMX = X and MX = XM. Thus
X =M#. D
TiHEOREM2.1. For everytransitionmatrixT, the matrixA* exists where
A=I- T.
Proof.The proofis dividedinto two parts.Part I is forthe case when T is
irreducible.Part II is forthecase when T is reducible.
PartI. IfT is irreducible, thenthePerron-Frobeniustheoremimpliesthatthe
eigenvalue 1 has algebraic multiplicity equal to one. Therefore,0 e- (A) with
algebraicmultiplicity 1. This impliesthattheJordanformforA can be writtenas

IA A= 0 B_
It is clearthatthisimpliesthatrank(A)= rank(A2),so that
whereB is nonsingular.
A* will exist.
Part II. If T is reducible,thenbya suitablepermutationofthestatesone can
write(where - indicatesequalityaftera suitablepermutation has beenperformed)
xl Y_
T-
whe XO
aZ_
whereX and Z are square.IfX or Z is reducible,thenby anotherpermutation
we
1 This lemmais all thatis needed forour purposes.However,the readershould realizethatit is
a special case of the more generalresultwhichstatesthatM# existsifand onlyif(U# and L# both
exist,and (I - UU#)C(I - LL#) = 0), in whichcase
FU I U#2C(I - LL#) + (I - UU#)CL#2- U#CL#]
# L#{4 I
446 CARL D. MEYER, JR.

can write

T- O C D.
JO O E
If U, C, or E is reducible,thenanotherpermutationis performed.
Continuingin
thismanner,we eventuallyget

LB11 B12 Bln]

...
T~ O B2 2 B2n

O O Bnn-

whereeach Bii is irreducible.If thereare some rowsforwhicheveryblock except

thediagonalblock is 0 (i.e.,ifthereis an i such thatBik = 0 foreveryk # i), then
performone last permutationand write

T11 T12 ... Tir Ti,r+i T1,r+2 . . .

Tin

0 T22 T2r T2,r+ 1 T2,r+2 T2n

. . I

0 0 * Trr Tr',r+ 1 Tr,r+ 2 ... Trn

T ----- -t-------------
0 0 0.. 0 Tr+1,r+i 0 0
?

O O ... 0
? Tr+2,r+2 ... 0

0 0 .. 0 0 0 *.. Tnn

Now, everyTii is irreducible.From Part I of thisproof,we know that(I -Tii)

existsforeveryi. However,foreach i = 1,2, , r, thereis at least one index .

k : i such that Tik 0. This means that p(Tij)< 1 for each i = 1,2,.., r.
Therefore, fori = 1,2, ... , r,(I - l exists.We can now concludethatthere
existsa permutation matrixP suchthatA can be written as

A = p{,l_l3

and G*2exists.
whereG11is nonsingular,

.
(I - Tr+1,r+i)# 0 T.j (

22~
{ #
 THE ROLE OF THE GROUP GENERALIZED INVERSE 447

thatA * existsand is givenby

It is now easyto use Lemma 2.1 to verify
#
= p,7 1t-' -G2(G
Lo II 2
G - G22G2)-
2G -' 2G

Notice thattheabove resultsimplythattheJordanformforeverytransition

matrixT can be writtenas
-I RO
ST -_? K-
where1 0 a(K). Otherwise,it would be impossibleforrank(A)to equal rank(A2).
These observationsproducethefollowingcorollary.
COROLLARY 2.1.For everytransition matrixT, thereexistsa nonsingular
matrix
S suchthat

T =S-' -k-4S, A ==S-{0 4s

and

A# = S-1Y 0
----s
LO i IK)i
where1 0 a(K).
One ofthefundamental factsin thetheoryoffiniteMarkovchainsis thatifT
is the transitionmatrixof eithera regularchain or an absorbingchain,thenthe
limitlim Tn exists.Furthermore, foreverytransitionmatrix,thelimit

I + T+ T2 +- + Tn
lim
n- oo n
exists.For ergodicchains,

lim O (n kn-i(l - k)T1, 0 < k < 1,

n-ooi=O i

exists.The nexttheoremgivesan expressionforeach oftheselimitsin termsofA .

THEOREM 2.2. Let T denotethetransition
matrixof an m-stateMarkovchain,
and let A = I - T. Then

I +
lim + T+ T + T foreverychain;
n-oo n
n n~~

I-AA# lim I ( )kn-i(l - k) T' ifthechainis ergodicand

n-oo i=0 i < k < 1;
lim Tn if thechainis eitherregularor
absorbing.
448 CARL D. MEYER, JR.

Proof.If T is the transitionmatrixof any ergodicchain,then 1 is a simple

eigenvalueand thereexistsa nonsingularmatrixS such that

(2.1) T= sI-1j1 j S

1 0 a(K), so thatI
Furthermore, - K is nonsingular,and it is easilyverifiedthat

(2.2) A# = S -- I )

If T is thetransitionmatrixofa regularchain,thenlimn11 Kn = 0, and it is clear

that

lim Tn= S-1{-T-S = I-AA#.

If T is thetransitionmatrixofan absorbingchain withexactlyr absorbingstates,

thenthereexistsa permutationmatrixP such that

T p{IrKLJp.

wherelimnoo Qn= 0. In thiscase,

A -P'-R I -=Qp

and

A# =' (- - - -?-Q- 2R-I - Q)

so that

I-AA# =p{____ - _--T--P= lim

To
noo
(I-Q) R IO
The factthat

I -AA# = lim I kn-i(l - k)'T', O < k < 1,

n-oo i=o i

foran ergodicchain followsdirectlyfromtheprecedingresultsforregularchains.

For 0 < k < 1,let
T= kA + T and A=I-T= (1-k)A.
Now, T is a transitionmatrixfora regularchain,and
n/1
lim .
(jkn-i(l k)iTi= lim Tn = I- A # = I-AA#.
10 0 I ni oo
The factthatthe sequence { Tn is Cesaro-summableto I - AA#also has a very
 THE ROLE OF THE GROUP GENERALIZED INVERSE 449

directproofwhichavoidstheuse ofcyclicclassesas in [4,p. 101]. If T is anytransi-

tionmatrix,thenT can be writtenin theform

-I IO
T = S- T S,

where10 ((K). Now write

(2.3) I +K +K2 + + Kn-I (I - Kn)(I - K)-

By using(2.3) it is a simplematterto verify

that

I + T+ T 2+ + T (I T Tn)A
~+ - +I A A *.
_ _ _ _ _ __ n
_ _ _ _ _ _ _ n
Since 1T = 1 forall n,it followsthat

i (I - T =)A
no n
and hence

imI+ T+ T2 + + T I-AA*. ]
n - co n
The followingresultis fundamental and has a verysimpleand directproof.
THEOREM 2.3. If T is thetransition
matrixof an m-stateergodicchainand if
A = I - T, then the rows of W = I - AA* are all equal to the unique fixed
probability
vector,w',of T so that

w' = e' - rA for each i = 1, 2, , m,

whereei is thei-throwofI and r' is thei-throwofA .

Proof. The facts ej(I - AA#)A = 0 and e'(I - AA#)j = e'j = 1 for each
i = 1, 2, , m,together with the Perron-Frobenius theorem, imply that w'
= e'(I - AA#) is the unique fixed probability vector for T. [
The followingtheoremwillbe usedin thenextsectionto givean interpretation
foreach oftheentriesofA # in thecase ofregularchains.
THEOREM 2.4. For everytransition matrixT,

1 n - k
A#= lim (Tk_W)
noc k=O n

If T is thetransition
matrixof a regularchain,then

A* Z (Tk - W).
k= 0
450 CARL D. MEYER, JR.

Proof.WriteT as in Corollary2.1 to obtain

n-i nI- k _
;E (Tk - W) =sI f
l in k Js
~
=-On(-K
k~~ 0-IK
0 KK ~~~~0 HS

0 (I K
- (I - 'KK !Kk1
-; k S0-T n

From Theorem2.2,we knowthat

kz I
This impliesthat
in-i
K -+0.
Kk
nk =O

Thus
ni- IV ~ 2 A
k__W) _+S- _
k - - - - - - #
(T
kO n Eo I (I K)- -
If T is the transitionmatrixof a regularchain,thenwriteT in the form(2.1).
Sincethechainis regular,itfollowsthatlimnOOK' = O, so that
0 0 0~ p 1
Z (Tk _ W) = S- 1 E S = S- S A#.
I k=O

3. A #as thefundamentalmatrix.If T is the transitionmatrixof an ergodic

chain,thenmost of thequestionsconcerningthechain can be answeredby com-
putingthematrix
(3.1) Z= (I-(T- W))-i,
which Kemenyand Snell [4] call the "fundamentalmatrix".In this section,it
will be demonstratedthat althoughZ : A #,the matrixA# may be used in a
mannersimilarto thewayZ is used.In mostcases,Z maybe replaceddirectlyby
A #.This is quite an advantage,because in orderto computeZ, directly,foran m-
statechain,one mustobtainw',whichinvolvessolvinga homogeneoussystemofm
equationssubjectto some constraints, and thenperforman m x m matrixinver-
sion. It willbe shownin ? 4 thatA #is mucheasierto obtainthanZ because it is
only necessaryto performone (m - 1) x (m - 1) matrix inversion,and no
knowledgeofw' is required.If w' is explicitlydesired,then,as shownin Theorem
2.3,w' is easilyobtainedonce anyrowofA #has been calculated.
The followingtheoremshowshow thematrixZ in (3.1) is relatedto A#.
THEOREM 3.1.Let T be thetransitionmatrixofan ergodicchain,letA = I -T,
and let W = jw', wherew' is theuniquefixedprobability vectorof T. The matrix
 THE ROLE OF THE GROUP GENERALIZED INVERSE 451

Z = (I - (T - W))- 1 is given by
Z = A* + W = I + TA#.

Proof.2 From Theorem2.3, W = I - AA#,so thatone maywrite

(3.2) Z = (A + (I - AA#))-' = (A + (I - AA#))#.
If E and F are square matricessuch that FE = 0 and EF = 0, then (E + F)#
= E# + F #,providedthatE # and F # bothexist.To see thatthisis true,noticethat
FE = 0 implies that F#E = 0 and FE# = 0, while EF = 0 implies that E#F = 0
and EF# = 0. Let X = E# + F#, and verifythat (E + F)X(E + F) = E + F,
X(E + F)X = X and (E + F)X = X(E + F). Applyingthis resultto (3.2) and
using the fact that (I - AA#)# = I - AA, one obtains Z = A# + I - AA#
I + TA#. O
Theorem3.1 allows all expressionsinvolvingthefundamental matrixZ to be
replaced by expressionsinvolvingA*. However,the theorycan be developed
directlyin termsofA * withoutevenintroducing thematrixZ. We shall notmake
use ofany resultwhichinvolvesZ in whatfollows.
For a regularchain,thenexttheoremprovidesan interpretation foreach of
theentriesofA #.
matrixof a regularchain,and let N("
THEOREM 3.2. Let T be the transition
denotethematrixwhose(i,j)-thentryis
-i) = theexpectednumber oftimestheprocess
is in stateS. in thefirstn stages(the
initialplus(n - 1) stages)whentheprocess
was initiallyinstateSi.
In thiscase,
A = lim (N() - nW).
n -xo

Proof.It is wellknownthatN(n) = n-I Tk, so that

n- 1

N() -nW= (Tk-W) --A#

k= 0

by Theorem2.4. 0
Thus, the (i,j)th entry,A of A# has the followingmeaningfora regular
chain.For largen,theexpectednumberoftimesin stateSj, wheninitiallyin state
fromnwjby approximatelyA , wherewj is thejth componentof the
Si, differs
fixedprobabilityvector.Furthermore, forlarge n, one can comparetwo starting
statesin termsoftheelementsofA * since
lim (NM')- N W) = A*i-A*.
n - o

2 Once one is giventheexpressionI + TA#,itis notdifficult that(A + W)(I + TA#) = I

to verify
to show that(A + W)-' = I + TA#. Althoughsuch a proofis slightlymore efficient than the one
givenhere,it is conceptuallyunappealingbecause it tendsto givethe impressionthattheproblemof
findingtheinverseof(I - (T - W)) is solvedby trialand error,whenin factit is not.
452 CARL D. MEYER, JR.

For an initialprobabilityvectorp',thejthcomponent,(p'N("))j,ofp'NI) gives

theexpectednumberoftimesinstateSi innstages.The following corollaryprovides
a comparisonof (pN (n))jand (p'2N(n))jfortwo different
initialprobabilityvectors.
COROLLARY 3.1. Let p' be an initialprobabilityvector,and let w' be thefixed
vectorforT. Then
probability
lim (p'N(n) - nw') = p'A ,
-- o
n

andfor twoinitialvectorsp'1and P'2

lim (p'1N(n)
-- o
- p'2N W) (p' - p'2)A .
n

Furthermore,
tr(A#) = lim E [N() - (p'N())k] = lim( (N))) (n)j)
n- k n-

independentoftheinitialvectorp'.
Proof.The firsttwo limitsare immediate.The thirdlimitfollowsfromthe
second and the factthatA #j= 0 since
Z [N(n) - (p'NWM)k]= [(e - p')Nek]
In Z [eA eek- p A ek]
k k k

= tr(A#)p'A#j = tr(A#). O

For an ergodicchainwithtransitionmatrixT, letM denotethematrixwhose

(i,j)th entryis givenby
Mij = theexpectednumber ofstepsbeforeentering
stateSjfor thefirsttimeaftertheinitial
stateSi.
M is called the meanfirstpassage matrixby Kemenyand Snell [4], wherethey
show thatM is the unique solutionof the equation
(3.3) AX = J - TXdg.

theyshow how to writeM in termsofZ by

Furthermore,

(3.4) M = (I - Z + JZdg)D,

whereD is the diagonal matrixwithdiagonal elementsdii = 1/wi.

In the followingtheorem,it will be shown how the solutionof (3.3) can be
givenin termsofA #,and it willbe demonstratedhow to use A #to proveunique-
ness.Furthermore, thematrixZ need not be introduced.
THEOREM 3.3. For an ergodicchain,theuniquesolutionof theequation

(3.5) AX = J - TXdg

is givenby

(3.6) M = (I - A# + JA #)D,

and hencethematrixM in (3.6) is themeanfirst

passage matrix.
 THE ROLE OF THE GROUP GENERALIZED INVERSE 453

Proof.To see thatM is a solutionof AX = J - TXdg, note thatMdg = D

and use Theorem2.3 to obtain
AM = A(I - A# + JA#)D = (A - AA#)D = (A - I + W)D

= (W- T)D = J - TD = J - TMdg.

One may prove uniquenessby usingA# as follows.Multiplyboth sides of (3.5)
on the leftby w' and therebydeduce thatXdg = D forall solutionsX. If X and
Y are both solutionsof (3.5), thenA(X - Y) = 0, so thatX - Y mustbe of the
form (X - Y) = (I - AA#)H for some matrix H. By Theorem 2.3, I - AA#
= jw' so that X - Y = j(w'H). Since the diagonal elements of X - Y are all
zero, it follows that X - Y = 0. 0
Notice thatin thiscase the expressions(3.4) and (3.6) are identicalexceptZ
has been replacedby A #.
For an initialprobabilityvectorp',(p'M)kgivestheexpectednumberofsteps
beforeenteringstateSk forthe firsttime; (w'M)k is the expectednumberof steps
beforeentering timeaftertheprocessis observedinequilibrium.
stateSk forthefirst
COROLLARY 3.2. For an ergodicchain,

(w'M)k= 1 + Ak.
Wk

Proof. w'M = w'D - wAA#D + w'JAd#D j'(I + Ad#gD)because w'A#=0. D

The term (pM - p'2M)k = ((p'l - p2)M)k provides a comparison of the
expectednumberof steps beforeenteringstate Sk fortwo different initialprob-
abilityvectors.
COROLLARY 3.3. For an ergodicchain andfor two initialprobability
vectors
p'1and p2,

(p' - p')M = (p' - p') (I -A #)D.

Proof.This followsbecause p'1J= p'2J= i'. U
Kemeny and Snell show that (p'1 - p'2)M = (p'1 - p'2)(I - Z)D. Therefore
Corollary3.3 is anotherexample wherethe matrixZ may be directlyreplaced
by A#.
For an ergodicchain,let V be thematrixwhose(i,j)thentryis
Vij= varianceofthenumber ofstepsrequiredto reach
stateSjfor thefirsttimeaftertheinitialstateSi.
Kemenyand Snell show that
Vij = Bij - (Mij)2

wherethe Bij's are elementsof a matrixB whichis the unique solutionof the
equation
(3.7) AX = J - TXdg + 2T(M - Mdg),

or, by writing M - Mdg = M - D = -(A# - JA#g)D,

AX = J - TXdg -2T(A# -JAd)D

454 CARL D. MEYER, JR.

THEOREM 3.4. The uniquesolutionof

AX J
J-TXdg - 2T(A# - JAd)D

is givenby
B = M(2Ad#gD+ I) + 2(A#M - J(A#M)dg)
Proof.Note firstthatsince Mdg = D,
Bdg = 2DAd#gD
+ D.

Now, use thefactsthatAJ = O,AM = J - TD, AA#M = A#TD and TJ = J

to get
AB = (J - TD)(2AdD + I) - 2A#TD = J - TBg -2T(A# - JAd)D.

To proveuniqueness,one mayproceedin thesame manneras in Theorem3.3. O

In termsofthematrixZ,
B = M(2ZdgD - I) + 2(ZM - J(ZM)dg),
and onceagain one sees thatZ maybe directlyreplacedbyA , whenone algebraic
signis reversed.
It is easy to show thatforan ergodicchain and fortwo functionsdefinedon
the statesof the chain,the limitingcovarianceas givenin [4, 4.6.1,p. 84] can be
writtenin termsoftheelementsof A *. In fact,thenumberscij whichare defined
in [4,p. 85] can be writtenas

cij= wiA/ + wjAi + wiwj -wiij,

where6ij is the Kroneckerdelta. Other resultsinvolvingA * in place of Z are
possible.Theirderivationsand proofs,alongwiththeproofconcerningthelimiting
covariancesare leftto theinterestedreader.
4. The diagonalelementsof A '. For an ergodicchain,all of the elementsof
A * havebeen shownto be offundamental importance.However,therewerecases
whenonlythe diagonal elementsof A* were involved.In thissectionit will be
provedthatthe diagonal elementsmustalways be positive.
THEOREM 4.1. If T is the transition
matrixof an m-stateergodicchainand if
A = I- T, thenforeach i = 1,2, ,m, A > 0.
Proof.The entriesof the mean firstpassage matrixare positive,i.e.,
M = D - A#D + JAdD > 0.

Since Mdg = D, it followsthatfori : k,

0 and [(JAd -A#)D]dg

[(JA* -A A)D]ik > = 0.
Also,D is a diagonal matrixwithDii > 0, so thatfori : k,
(JA#- A#)ik> 0 and (JAd#-A#)dg = 0.
Therefore
(A*,,A*2, A*) =j'A* = w'JA* = w'(JA* -A#) > 0

because w'A# = 0 and w' > 0. 0

 THE ROLE OF THE GROUP GENERALIZED INVERSE 455

Thus thefollowingcorollaryis immediate.

COROLLARY 4.1. For an ergodicchain,tr(A #)> 0.
Kemenyand Snell [4, pp. 81, 102] show thatan ergodicchain is completely
determined by the nonsingular matrix M = M - D = (JAg - A#)D.
These resultsand some otherinterestingrelationshipsinvolvingthediagonal
elementsof A #can be obtainedas corollariesof the nexttheorem.
THEOREM 4.2. If T is thetransition
matrixofan m-stateergodicchain,thenthe
matrices

L=JAd#- A# and K = JA#-A

and
are nonsingular,
(4.1) L-' = cI + (cxL - I)A

and
(4.2) K = acI + (cxK -I)A

where a = 1/tr(A#).
Proof.3From Corollary4.1, it followsthattr(A#) 0, 0 so thata is defined.
Let X denotetheright-hand sidesof(4.1), so thatXL = cL + xLAL - AL. Now,
AL = -AA#, so thatLAL = -LAA# = JA#AA#-A#, and XL = cxJAd#W
+ AA#.Since cJAd# W = W = I - AA #,itfollowsthatXL = I, so thatX = L.
A similarargumentcan be used to prove(4.2). U
The factthatM is nonsingularis an immediateconsequence.Furthermore,
Theorem4.2 providesan explicitformforM 1, bymeansofwhichone can prove
thefollowing.
COROLLARY 4.2. For an ergodicchain,w' is givenby

w= tr(A#)(Mlj)
Furthermore,

(4.3) tr(A) (M- )ik =j'M- }.

i,k

Proof.From (4.1), M- lj = D- 1L-'j = aD- lj = aw. Equation (4.3) follows

because j'w = 1. U
COROLLARY 4.3. For an m-stateergodicchain,w' is givenby

w'= a'(JA #-A)

wherea' = (A #1,A2,
A's) , -
Proof. w'K = a'. U
It should be pointedout thateven ifone knewthe diagonal elementsofA#,
usingCorollary4.3 to obtainw' would involvemuchmoreworkthanis necessary
(see thenextsectionon computation).

3 Here again, the expressions

forL-' and K-' were not arrivedat by trialand error.The first
partofTheorem7 in [8] providedthe results.
456 CARL D. MEYER, JR.

5. Calculationof A * and w' foran ergodicchain.As has been demonstrated

in the previoussections,the matrixA * may be used in place of the matrixZ
throughout theentiretheoryoffiniteergodicchains.One ofthemajoradvantages
in usingA * ratherthan Z is the factthatA * is easier to calculatethan Z. This
should not be surprisingsince the matrixZ = (A + W)-1 is the inverseof a
matrixwhichinvolvesthe transitionmatrixtogetherwiththe fixedprobability
vector,whichis generallynotknownat theoutsetand mustitselffirst be calculated.
On theotherhand,A * is just theinverse(in a generalizedsense)ofa matrixthat
involvesonlythetransitionmatrix.Moreover,we willshowthatthecomputation
of A * involves,at most,only one (m - 1) x (m - 1) matrixinversion,whereas
thecomputationof Z alwaysinvolvesan m x m matrixinversion.
The known methodswhich are at all practical(thereare veryfewknown
methods,practicalor otherwise)forcalculatingthegroupinverseG ' ofa general
matrixG (provided,of course,that G * exists)involve determining a maximal
setoflinearlyindependent rowsand a maximalsetoflinearlyindependent columns
of G. This,in general,can pose numericalproblems.However,the nexttheorem
shows thatforergodicchains,thisproblemis not presentwhencalculatingA *,
and itis thekeyresultwhichallows one to computeA ' withoutan undueamount
ofdifficulty.
THEOREM 5.1. Let T be thetransition matrixof an m-stateergodicchain,and
let A = I - T. Every principalsubmatrixof A ofsize k x k, k = 1, 2, 3, , m - 1,
is nonsingular.Furthermore,theinverseof each principalsubmatrixof A is a non-
negativematrix.
Proof. Let A be a k x k principalsubmatrixof A, where 1 ? k ? m -1.
Then 4 = I - T, where 'T is a k x k principalsubmatrixof T. Let P be the
permutation matrixsuch that

T ,4X Y]p
R
Let

T, = P (R---T4-P.

T1is thetransitionmatrixofan absorbingchain.For sucha chainit is wellknown

that the matrix A = I-T is nonsingular and (I - T)' _ 0. []
It is now possibleto givea formulaforA #.
THEOREM 5.2. Let T be thetransitionmatrixof an m-stateergodicchain,and
let A = I - T. WriteA as

U Ic
A=E-_d'-d

whereU is (m - 1) x (m - 1) (fromTheorem5. 1, U - exists)and adoptthefollow-

ingnotation:
h' = d'U , 6 =-hU-j,

=i - h",
j-1- F U -I.
.6
 THE ROLE OF THE GROUP GENERALIZED INVERSE 457

Thescalars6 and : are each nonzero(infact,6 > 0 and > 1), and A# is givenby
(5.1) A#{_ U 'jh'U Fjh'F Fj

_ 4~~~~2
Proof.To show c3> 0 and / > 1, observethat d' ? 0. From Theorem5.1,
it followsthat U- ?> 0 and hence h' < 0. However,h' is not the zero vector.
Otherwise,ifh' = 0, thend' = 0, because U is nonsingular, and thiswould imply
that T is reducible,whichis impossible.Thereforeh' < 0, and it now followsthat
3 > 0 and , > 1.To showthevalidityof(5.1),firstnotethat(I -jh')- = I + jh'/I.
Let H = (I - jh')- 1U- '(I - jh')-', so that
U -'jh' jh'U 6jh'
(5.2) H = U-' + + : A

By expandingit out and using(5.2), it is easy to see that

U-'jh'U-' Fjh'F
u __ _ - H .

Likewise,use (5.2) to show that

h'F Fh'H ] Hj and /2 -h'Hj.

Thereforethe matrixon the right-hand

side of (5.1) can be writtenas

(5.3) ]i---j
( ) th'~~~~~~~H
-h'Hj3
Since rank(A)= rank(U) m - 1, it is well known(and can easily be verified)
thatA can be writtenas
UA Uk
Lh'U h'UkJ
and thatA * is givenby

(5.4) A hHk}

wherek = U- 'c (see [6]). Because the row sums of A are all zero,it followsthat
Uj + c = 0 and hence j -U'c = k, so that the matrices in (5.3) and (5.4) are
equal. 0
For an ergodicchain,ifone desiresto computethe fixedprobabilityvector
w',one does not needto knowtheentirematrixA *. As demonstrated in Theorem
2.3, knowledgeof any single row of A ' is sufficient.
Theorem 5.2 providesan
easily obtainable row, namelythe last one, and therebyprovides one with a
relativelysimpleway ofcomputingw'.
458 CARL D. MEYER, JR.

THEOREM 5.3. If T is the transitionmatrixof an m-stateergodicchain and

A= I - T is partitioned as

A = =-
pd' [xd'a3
-
j=J

vectorof T is givenby
whereU is (m - 1) x (m - 1), thenthefixedprobability

w' = -[-h' 1] - -[-d'U-1 I 1]

where = - hj = 1 - d'U-'j.
Proof.From Theorem2.3, w' = em- rmA,wherer' is the last row of A#.
From (5.1),

rm @h'F :

so that

rmA= [h' h'].

Therefore

W'e= e- r'A = fh' 1]. [D

From a computationalstandpointit is importantto point out that when

Theorem5.3 is used to computew',it is not necessaryto calculate U- , explicitly,
calculateC- 1 in orderto solvea nonsingular
just as itis notnecessaryto explicitly
systemof linearequations Cx = b. Indeed,one may considerthe vectorh to be
thesolutionof the nonsingularsystemU'x = d and proceedwiththesolutionof
the systemby conventionalmethods.
COROLLARY 5.1. For an ergodicchain,

I - AA#

and

AA# =

Proof.The firstresultfollowssince W = I - AA ; the second followsfrom

thefirst. O
As pointedout above, in orderto computethe fixedprobabilityvector,w',
or even U- .
foran ergodicchain,it is not necessaryto computeA* explicitly,
 THE ROLE OF THE GROUP GENERALIZED INVERSE 459

Assumenow thatone has alreadyobtainedw',eitherby theoreticalconsider-

ationsor bypreviouscalculation.Then,as shownin Corollary5.1,one has essen-
tiallyobtainedthematrixAA*. If one would wishto computeA * and he already
has knowledgeof w', and hence AA*, it may be desirableto use this known
information in obtainingA * ratherthan startingfromscratch.The following
resultshowshow thiscan be done.
THEOREM 5.4. For an m-stateergodicchain withtransition
matrixT, write
A = I - Tas
ul c
A=E-4-id'

whereU is (m - 1) x (m - 1). The matrixA# is givenby

(5.5) A# = AA#-[0-T 03AA

Proof.If A - is any (1)-inverseforA, then

AA#A-AA# = A#AA-AA# = A#AA#=A#.

Thereforeit is onlynecessaryto show that

X = ----

is a (1)-inverseforA. This followsdirectlybecause a = d'U- lc. 0

Ifone has knowledgeofw',and henceof W,thenthenexttheoremis another
way to obtainA *. This is essentiallycomputingZ.
THEOREM 5.5. For an m-stateergodicchain,A * is givenby

(5.6) A* = (A + W)-1 - W(= Z-W).

Proof.From Theorem3.1,we knowthatZ = (A + W)' = I + TA#. Thus

(A + W)-1 - W= I + TA# - W= I + TA# - I + AA# = A#. O

We now computeoperationalcountsfor(5.5) and (5.6). Assumethatone is

usingan algorithmwhichrequiresf(n) multiplications (and divisions)in orderto
perform a completen x n matrixinversion.Then it takesf(m - 1) multiplications
to computeU- 1. WriteU-1 as

U = [V1 ' V2 Vm-]

and writeAA' as

Ae'A w']
AA# I-= e2'- [el wlj e2 W2j em-wmV].
460 CARL D. MEYER, JR.

By writing(5.5) as

(AA # - - AA*#
*
I II I I

y em
ll:- [o---iT-T3!j[el
wl wjl
itcan be seenthattheproductcan be formedbyusingnotmorethan2m2- 2m + 1
multiplications. Therefore,using (5.5) (assumingthat w' is known) requiresa
total of not more thanf(m - 1) + 2m2- 2m + 1 multiplications, whereas(5.6)
requiresf (m)multiplications.
For thesake ofillustration, suppose thatone is usingan inversionalgorithm
suchas Gaussian elimination wheref(n)is about n3.Thenonlyaboutm3 -_m2 + m
multiplicationsare needed to use (5.5), whereas about m3 multiplicationsare
neededwhen(5.6) is used. Thus (5.5) would involveslightlyfewermultiplications
than(5.6) in thiscase. Noticethat(5.5) also has theadvantagethatonlyan (m - 1)
x (m - 1) matrixinversionmustbe performed, whereas(5.6) requiresan m x m
matrixinversion.
Table 1 is presentedto give a roughcomparisonof variousmethodsof this
paper forobtainingA* and w'. For example if,f(n) = n3, then the right-hand
columnin theTable 1 is shownin Table 2.

TABLE 1
For an m-stateergodicchain

Quantitiescomputed Formula used Multiplicationcountt

1. A * only (5.1) f(m -1) + 5m2- 6m + 4

2. A#and thenw' (5.1) and FPV* (using U-' to obtain h') f(m - 1) + 5m2 - 5m + 3

3. w' and thenA# FPV (using U- 1to obtain h') and (5.5) f(m - 1) + 3m2- 3m + 1

4. w' only Gaussian eliminationto obtainh' and thenFPV 'm3 + 4m - 2

5. w' and thenA# No. 4 and then(5.6) f(m) + Im3 + 4m - 2

f(n) = requiredto invertan n x n matrix.

numberofmultiplications

FPV refersto theformula

w' = -[-II' 1]

multiplication
togetherwithdivisionsA singlemethodcan giveriseto different
t Multiplicationsare takento meanmultiplications
countsbecause of theorderin whichthevariousoperationsare carriedout Thus thesecountsshould onlybe used to geta roughidea
about theamountofworkrequired
 THE ROLE OF THE GROUP GENERALIZED INVERSE 461

TABLE 2

Quantities for f(n) = n'

1. m3+ 2m2 - 3m+ 3

2. m3+ 2m2 - 2m+ 2
3. m3
4. im3+ 2m-2
5. 4m3+ 2m-2

Ifbothw' and A #are to be computed,withf (n) - n3, thenTable 2 indicatesthat

(5.5) is themostefficient, countsgo.
as faras multiplication
By usingthefactthatZ = I + TA ', one mightbe temptedto say thatcom-
putingZ isjust about as easyas computingA '. However,one mustremember that
to
ittakesm3 multiplications formtheproduct TA '. to
Referring Table 2, one can
see thatiff(n) n3, thenit would
= take twiceas many to
multiplications compute
Z as is requiredto computeA ' whentheformulaZ = I + TA ' is used.Whenthe
formulaZ = (A + W)'- is used,withf(n) = n3, Table 2 showsthatit takesmore
than4 as manymultiplications to computeZ as is requiredto obtainA .
This sectionis concluded with an example.Considertheregularchainwhose
transitionmatrixis given by

0 2 2 O-
12 0 2 0
T=4 2 1 0 1

so that

4 -2 -2 O

1 2 4 -2 0
A=-
4 -2 -1 4 -1
1 I 1 31

First,thefixedprobabilityvectorw' is computed.Ifonlyw' is desired,and not

A#, thenh is computedas thesolutionofthesystemU'x = d,where

4 -2 -2
U - -2 4 -2 and d' = (-1,- I,-1).
2 2
L:- and4d' 1
462 CARL D. MEYER, JR.

If A is desired,as is the case in this example,compute U' and obtain h' as

h' = d'U- . In our case,
-7 5 6-
LJi=1=26 6 6 and h' = --(6,5,6).
3 ~~2'
-5 4 6-
Now calculate ,B= 1-hj= 1+ 1 = -9. The fixed probabilityvector w' is
givenby

w =-[-h' 1]= (6,5, 6,2).

/3 ~~19'
Therefore

~w' F13 -5 -6 -2

AA# =1I- | 1 -6 14 -6- 2

tWT 19H6 -5 13 -2
tw-lj L-6 -5-6 17]
If A is desired,it may be computedeitherfrom(5.1), (5.5), or (5.6). For this
example,it was moreconvenientto use (5.5) to obtain
265 - 61 - 96 - 108
= 2 -96 300 -96 -108
1083 - 115 - 137 246 6
-210 - 156 -210 576
The matrixD is

6 0 0

D= 19 O 0 0
? ? 6?
00
Lo 0 ?

and themean firstpassage matrixis

5415 4332 3420 20520

1 3610 6498 3420 20520
Jdg) 1710 3800 5244 5415 17100
4750 5472 4560 16245]
 THE ROLE OF THE GROUP GENERALIZED INVERSE 463

6. Concludingremarks.At the outset,it was mentionedthat the Moore-

Penrose generalizedinversewas not usefulin developingthe theoryof finite
Markovchains.However,thereare matricesforwhichA * = At. It is wellknown
thatthissituationoccursifand onlyifN(A) = N(A'). (Matriceswhichsatisfythis
conditionare called EP matrices.)In termsofan ergodicchain,thissayssomething
veryspecial.
THEOREM 6.1. If T is thetransitionmatrixof an m-stateergodicchainand if
A = I - T, thenA * = At ifand onlyifT is a doublystochasticmatrix.
Proof.IfA * = At,thenN(A) = N(A'). Sincej E N(A), itfollowsthatj E N(A'),
whichimpliesthatthe columnsums of T are all equal to 1. Conversely,suppose
thatthecolumnsumsof T are all equal to 1,so thatj E N(A'). It is alwaystruethat
je N(A). It is well known(and easily seen fromTheorem5.1) thatrank(A)= m
- 1 = rank(A'),so thatdimN(A) = 1 = dim N(A'). Therefore{j} is a basis for
both N(A) and N(A'), and hence N(A) = N(A') so thatA# = At. L
One can now see thatthe cases whereAt = A#are onlythe trivialcases as
faras theMarkovchainis concerned,because foranydoublystochastictransition
matrixofan m-stateergodicchain,thefixedprobabilityvectorw' is alwaysgiven
by simplyw' = (l/m)j',so thatAA# = AAt = I - (l/m)Jand

A* = At = -J,gU [

In thecases whenA# : At, one can still"force"At or any other(1)-inverse

of A into the theoryof Markov chains because of the equation solvingproperty
thatall (1)-inversespossess. For example,one could use any (1)-inverseof A to
ofA as suggested
solve(3.5) and (3.7), and w' could be givenin termsof(1)-inverses
in [1] and [5]. However,thisjust leads to cumbersomeexpressionswhichdo little
or nothingto enhance the theoryand providesno computationalor practical
advantage.
On the otherhand,the groupinversedoes not need to be "forced"intothe
theory.The conceptof A seemsto intertwine naturallywiththetheoryof finite
Markov chainsas ifit wereespeciallydesignedto do so. The introduction of A *
not onlytendsto unifythe theoryof finiteMarkov chains,but it also providesa
practicaland a computationaladvantage.

REFERENCES
[1] H. P. DECELL, JR. AND P. L. ODELL, On thefixedpointprobabilityvectorof regularor ergodic
transitionmatrices,J. Amer.Statist.Assoc., 62 (1967), pp. 600-602.
[2] M. P. DRAZIN, Pseudoinversesin associativeringsand semigroups,Amer. Math. Monthly,65
(1958), pp. 506-514.
[3] I. ERDELYI, On thematrixequationAx = ABx,J. Math. Anal. Appl., 17 (1967), pp. 119-132.
[4] J. G. KEMENY AND J. L. SNELL, FiniteMarkovChains,Van Nostrand,New York, 1960.
[5] P. L. ODELL AND H. P. DECELL, On computing thefixedpointprobability vectorofergodictransition
matrices,J. Assoc. Comput. Mach., 14 (1967), pp. 765-768.
[6] P. ROBERT, On thegroup inverseof a linear transtormation, J. Math. Anal. Appl., 22 (1968),
pp. 658-669.
464 CARL D. MEYER, JR.

[7] M. J. ENGLEFIELD, The commuting inversesofa squarematrix,Proc. CambridgePhilos. Soc., 62

(1966), pp. 667-671.
[8] C. D. MEYER, Generalizedinversionof modifiedmatrices,SIAM J. Appl. Math., 24 (1973),
pp. 315-323.
[9] F. R. GANTMACHER,Matrix Theory,vol. II, Chelsea,New York.
[10] L. MIRSKY, An Introductionto LinearAlgebra,Oxfordat the ClarendonPress,London, 1955.
[11] C. R. RAO AND S. K. MITRA, GeneralizedInverseMatricesand its Applications,JohnWiley,
New York, 1971.
[12] T. L. BOULLION AND P. L. ODELL, GeneralizedInverseMatrices,JohnWiley,New York, 1971.