Numer. Math.

35, 421-442 (1980)

9 by Springer-Verlag 1980

Numerische Mathematik

Characterizations of Dominant and Dominated Solutions of Linear Recursions

R.M.M. Mattheij
Mathematisch Instituut Katholieke Universiteit, Nijmegen, The Netherlands

Summary. Dominated solutions of a linear recursion (i.e. solutions which are outgrown by other ones) cannot be computed in a stable way by forward recursion. We analyze this dominance phenomenon more closely and give practically significant characterizations for dominated and dominant solutions. For a dominated solution, in particular, this leads to a stable computational method.

Subject Classifications: AMS(MOS): 65Q05, 65F99, 39A10; CR: 5.14.

1. Introduction In this paper we consider the n dimensional linear space. Let 112~{Ai}~o be a sequence of matrices (e~" x tE"). If a sequence of vectors x ~ {xi}i~ o, satisfies the matrix vector recursion (MVR) Xi+ 1=Aixi, i>O, (1.1) then x is called a solution of the recursion; we indicate this MVR by {Ai}i~ o or [A]. We shall investigate MVRs having solutions with different growth behaviour, i.e. for which there exist solutions x and y say, such that lira ]lxill 0

such a solution x is said to be dominated by y (cf. [13]). Since the computation of dominated solutions by means of (1.1) is unstable (cf. [4; 5]), other ways have to be explored. A well known way to find approximations to the "most dominated solution", is Miller's algorithm (cf. [4; 9], see also w an interesting alternative for the summation of such a solution in the scalar second order case can be found in [3]). Quite a new approach was given in [11] (see also [12; 10]), where it was indicated how a dominated solution could be determined in a stable way using an approximating boundary value problem. Although this has in part solved the problem of computing "intermediate solutions" for the scalar case, there is still need for a more general approach; moreover these results do not lend themselves to a straightforward generalization to MVRs (these MVRs seem particularly interesting from the point of view of discretized differential equations).

0029-599X/80/0035/0421/$04.40

422

R.M.M. Mattheij

In this paper we therefore consider the question of developing techniques for computing a solution f of [A], which is dominated by a certain number, k say, of independent solutions, and which is not dominated by ( n - k - l ) other independent solutions (the last set of solutions then is also dominated by the first mentioned set). We start with investigating the dominance of solutions more closely. It turns out that not only the magnitudes of the iterands of a solution are important quantities for having a practically useful kind of dominance but also their directions (93)- The latter fact (which makes the problem really more difficult than e.g. in the scalar second order case) suggests to us a geometrical approach, which is described in a special section (94). With the concepts of dominance, and our geometrical tools, we show in 99 5, 6 how a fairly general solution can be decomposed into a component having the characteristics of solutions that dominate f and a component having the characteristics of dominated solutions. This leads to a stable computational method for f (see 97). In this paper we do not give computational details of this method; these can be found in a subsequent paper [8]. Our intention here is to describe a general approach for ruling out instabilities caused by dominant solutions. This generality is such that it also allows for a geometrical interpretation for some other existing methods, like computing special functions or solving boundary value problems. Again, this will not be worked out here. For an example we refer to [7, 919], where it is shown how Olver's algorithm can be looked upon in our way.

2. Some Conventions

2.1 7he Partitioning

Let k be a fixed integer (1 <kNn). For a vector x = ( ~ 1..... ~.)r we write

where x I =(~1 .... , ~k)T and x 2 =(~k+ t,---, ~,)r. Let A be a matrix; then this partitioning induces a partitioning for matrices for which we use the following notations:

{A 11 A12~
A = \A21

A22]=(A1]A2),

(2.2)

where A"tl and A 1 have k columns and A 11 is square.

2.2 Sums and Products of Matrices

Let {Ai} be a set of matrices. Then

i=pAi = {A, +...0 + Aq,, ~

if q __> p if q<p

(2.3)

Dominant and Dominated Solutions of Linear Recursions

423

If ViA~ is square, then

f i Ai={Aq..;Ap,
i=p
*

if if

q>p q<p

(2.4)

2.3 k-Matrices and k-Solutions

F o r vectors v,, ...,VkeqY denote by (vlt...lvk), (2.5)

the matrix having v t .... , vk as respective columns. Generally a matrix having k columns (each being an element of (17") is called a k-matrix. F o r a matrix V (whose columns are elements of ') we denote the subspace of C", spanned by its columns, by span (V). Let X = {X~}i~=o denote a sequence of k-matrices. If the columns of the X i are
D

the iterands of respective solutions of the M V R [A] and m o r e o v e r if X 0 has rank k then X is called a k-solution of [A]. (Note that the sequences of respective columns of a k-solution are independent solutions.) Especially, an nsolution is often called a fundamental solution. F o r a sequence X as above and fixed matrices M and N of suitable dimensions, the sequence {MXiN}i ~ o is denoted by MXN. For each i let X i be given by

X, = (x,(1)l ... ]x,(k))

(2.6)

then span(X) denotes the linear space generated by the sequences {xi(l)}~=o, l = 1 .... ,k. If x e s p a n ( X ) is not the sequence {0,0,...} then we indicate this by x4=0.

2.4 The Norm

I17" will be provided with the 2-norm II-II. F o r any xetE" this n o r m induces norms for x 1 and x 2 (see w2.1). As is well k n o w n the associated operator norm, denoted by II.ll or lub(.), has the property that for any A and unitary U 1 and U 2 - of proper dimensions - li U1A U 2 il = IIAIt (unitary invariance). We shall also use the notion glb(A), defined by

glb(A)= min [IAxl[.

Ilxll= 1

(2.7)

N o t e that glb(.) is also unitarily invariant. If for matrices L and M both glb(L)=glb(M) and lub(L)=lub(M) then we shall say that L and M have the same magnitude and likewise if these quantities are of the same order in some context, then L and M are said to have magnitudes of the same order.

424

R.M.M. Mattheij

3. Dominance
In w1 we introduced a notion of dominance for solutions of [A]. If a solution g dominates a solution f then ag will also dominate flf for any scalar a and fl (a~0), i.e. any g~span(g), g + 0 , will dominate any f e s p a n ( f ) . For matrix solutions we therefore have the following possible generalization.

Definition 3.1. Let G be a k-solution of [A] and F an/-solution of [A]. Then G is said to dominate F if any gEspan(G), g:t=0, dominates any f e s p a n ( F ) . In fact 3.1 defines dominance for solution subspaces and G and F are merely used to indicate bases for these subspaces. The existence of such subspaces is reported e.g. in [7; 13] assuming some very mild conditions. Especially for the case l = n - k one may ask whether they are unique, and if so, how one can find numerically appropriate representations for them, which will then allow a stable computation of solutions. The second of these questions - the main topic of this paper - will be dealt with in w5 ff. We first consider the unicity problem. For the second order scalar case with n-- 2 and k = 1 it has been shown in [4, p. 25] that any dominated solution is a multiple of a certain f say, i.e. span(F) is unique, whereas any linear combination of some f~span(F), f 4=0, with a gespan(G), g4:0, defines a dominant linear subspace, so span(G) is not unique anyway. We shall prove that span(F) is also unique in our more general situation. For this we need the following:
L e m m a 3.2. Let G and F be as in 3.1. 7hen (G]F) is a (k +l)-solution of [A].

Proof: Suppose for some (k + 1) dimensional vector v 4:0 we have: Zo=(GoIFo)v=O. D

Define Vlzi=(GiIF3v. Partition v as = 0 we have

?t

V2 '

where v 1 has k coordinates. Since V~zi

ViGiv I = - V / v 2
If v ~ is zero, then Fo v2 =0=~v 2 =0. This contradicts the assumption v 4:0.

(1)

If v I is nonzero, then lim IJFiv21[ =0, which contradicts (1). Hence there

i~

l[Givl II

exists no such v + 0 , i.e. (GofFo) has rank (k+l).

[]

Corollary 3.3. Let G and F be as in 3.1 and l = n - k , then (GIF) is a fundamental solution.
Theorem 3.4. Let G and F be as in 3.1 and l = n - k .

7hen span(F) is unique, i.e. any other dominated (n-k)-solution P equals FP, where P is a nonsingular ( n - k ) x (n - k) matrix (which depends on the choice of F). Proof: Let (~ be a k-solution of [A] that dominates an (n-k)-solution te of [A]. Then for some nonsingular matrix M there holds (d[F)=(G[F)M (cf. 3.3). Write [M 11 M12~ M = \M21 M22] (M 11 k-th order).

Dominant and Dominated Solutionsof Linear Recursions

425

(a) Suppose M 11 is singular. Let the k dimensional vector v I # 0 be such that and define g=(~v 1. Then ~,=FM21v1. Since gespan((~), ~ must dominate F. On the other hand we also have gespan(F), so G dominates ~. From 3.2 it now follows that the rank of (go]fro) equals ( n - k + l ) , and that the rank of (Gol~o fif0) equals (n+ 1), which is absurd. Hence M 11 is nonsingular and, consequently, G dominates F. (b) Suppose M12#0. Then there exists an f~span(P), which has a nonzero component in span(G). This, however, contradicts the result of part (a). So M 12 =0.

Mllv1=O

(C) Since M is nonsingular, M 22 must be nonsingular. Take P = M 22.

[]

A well known method for approximating a solution f of [A], which is dominated by some (n-1)-solution of [A], is Miller's algorithm [9]. We briefly sketch it here: Let ViAi be nonsingular and let, for some (suitably large) N, a N be N o a vector which is not orthogonal to fN. Then a sequence {f~( )}i=N, obeying fN(N) = aNand JI(N)= A F lf~+I(N) ' i = N - 1, ..., 0 is computed. Thinking of it as a generalization of inverse iteration, one may expect that f~(N) becomes directionally close to f~, the better the larger (N-i). Therefore, an approximation {~}~'=o to {fi}f=o (P suitably smaller than N) may be found by normalizing the sequence {f~(N)}f=o, say using the initial condition fo; define e.g. f~(N) =fi(N)Ilfoll/Nfo(N)l[. For a certain set {as} (e.g. all equal to some fixed vector), we can say that the algorithm is convergent if lira ~(N)=f~, for i fixed.
N~oo

However, dominance according to 3.1 may not be sufficient to ensure convergence; we demonstrate this phenomenon by an example (cf. also [9, w3]).

Example 3.5. Define /(i+2~ z - i - 2 \i!l]

Ai= i+1 i+2 0 i+2----2 1 ) i + 1 (i+1) 2 1 , i+2 1

i>=0

Let

Gi=

(i+1,i2;)1 +
0 ~
and 0 0
-1=

f / = Fi =

i>_O.

Then G is a 2-solution and f = F is a 1-solution of [A]. Obviously G dominates f. It can be verified that
N--1

N~1)2 ~ 0

N N+Io

(j~=oAs)

N+IN1
1) 1N

426 Define
[N-1 \--1

R.M.M. Mattheij

and

Thinking of inverse iteration, a reasonable requirement for convergence would seem: VuauCspan(Gu), i.e. 7+0. However, writing out fo(N) explicitly, we find: fo(N) =

fl-~)N+74(N+l)~,(fl-7)N+fl, 7

Hence we only have conver-

gence if fl=7 +0. It is clear that the latter requirement is a lot more restrictive than the former. An other point is that the choice fl = ~ is unsatisfactory from a stability point of view, being very sensitive to rounding errors. [] In order to understand the ill conditioning in 3.5 one should observe that the small angle between GN ((N +01)- 2) and GN ( ( N +0) _ I ) (both vectors having l about unit length) may cause cancellation; i.e. a N has large multiples of each of these vectors compared to the component in the fN direction. Apparently, the (]~[N-1 Aj )-1 dominance does not ensure that ~-~_ a N has a comparably large com\j=o_

ponent in the f0 direction. This can also be put in the following way: Define

g(N)=G((No

1)-2 ) and

h(N)=G(-1N+l).

Then gN(N)and

hN(N)

form an

orthonormal basis for span(GN). Generally therefore a N will contain multiples of gN(N), hu(N ) and fn (which also does not make a small angle with the other two vectors) which are of comparable size. However, 1 ]]go(U)[l=(N+l)2 and llfoll=]/3 but ]tho(U)[] = ] / 2 N z + 2 N + l ;

so the component in the dominant space has grown about N times as fast as the one in the desired direction (viz of fo). [] In order to exclude situations like above, we introduce a stronger dominance concept:

Definition 3.6. Let G be a k-solution of [A] said to dominate F uniformly if

lim sup
f~span(F), fo :i: 0
i~oo gEspan(G),gff:O

and F an/-solution of [A]. Then G is

IIf01t/tIIg0HJ

Itfi[[ /[][gitt ] = 0

Remark 3.7. Clearly, uniform

dominance implies dominance.

In 4.6 we shall show that an appropriate separation of the directions of the columns of the G i together with dominance of sequences of these columns with respect to similar sequences of span(F) implies uniform dominance. For practical use of 3.6 the following property is very helpful.

Property3.8. G dominates

F uniformly ifflim

lub(Fi) O. i ~ glb(Gi)

Dominant and Dominated Solutionsof Linear Recursions

427

Proof." If: Let gespan(G) and fespan(F). Then there exist vectors v and w (with k and l coordinates respectively) such that g = Gv and f =Fw. We obtain: []fll /[Ngi[[ ] < lub(F~) [lw[[ lub(Go) [[vl[ Ilfol[/LilgoIl/=glb(Fo) Ilwii glb(Gi) llvtl' from which the assertion follows using a limit argument. Only if: Let w(i), with }tw(i)[I= 1, be such that lub(F~)= llF~w(i)[I and v(i), with ]lv(i)l[ = 1, be such that glb(Gi)= IIGiv(i)]l. Then

lub(Fi) ]]riw(i)ll<lub(fo) ]IFiw(i)[I / [ IlG~v(i)[[ ] glb(Gi)- [[a~ v(i)l[ = glb(ao lifo w(i)ll L ~ v~(_l"
Since we can find for each i solutions Gv(i) and Fw(i) (i.e. sequences {Gjr(i)} and {F~w(i)} respectively) of which the i-th iterands satisfy this inequality, the proof again follows from a limit argument. []

Remark 3.9. We like to show that requiring 3.6 comes up to our expectations
regarding the convergence of Miller's algorithm. Let a N be a starting vector as before, and write aN=GNaN+fsa^2 (here G is a (n-1)-solution which dominates uA1 f uniformly). We assume that aN4=0 has a not too small component in the direction of fN, i.e. we assume that there is some e>0, such that VN[IGNfi~I[ <e}l/Nfi~lI (note that his was not sufficient in 3.5, and is very mild, cf. [9,w Then

Ilal l[<e lub(fN) [la2{t, = glb(GN)

Hence, for any fixed i we see that

i.e.

span(f~(N))=span ((G~lf~) [a~'l \~])--*span(f~).

[]

It is hoped that the preceding examples have convinced the reader that the stronger dominance concept 3.6 is a more satisfactory one than 3.1. Therefore it is reasonable to reformulate the problem-setting of w1, by requiring that the desired solution f is uniformly dominated by some k-solution G, whereas f is not dominated by ( n - k + 1) other independent solutions (which - together with f constitute a uniformly dominated (n-k)-solution). This will be our new starting point. (At some places we shall also give a justification of the stronger requirements.)

4. Angles Between Subspaces

For having uniform dominance of a k-solution G with respect to an/-solution F, say, it appeared in w3 that not only the growth of basis solutions of span(G) and span(F) was important, but that also the directions of these solutions might be important. A first aim of this section is to show this more precisely by introducing a notion of angle between subspaces of 112"(see 4.6). A second aim is

428

R.M.M. Manheij

to use this notion for a geometrical interpretation of transformations that will play an important role furtheron. We have (cf. [2, p. 9])

Definition 4.1. Let X be a k-matrix and Y be a n / - m a t r i x . Define

cos0= max
x~sp.n(X)
yespan(Y)

Ix*yl
IIxll IIYll "

Then 0, with 0 < , 9 < 2 is called the angle between X and Y..

Remark 4.2. Obviously ~9 in 4.1 does not depend on any particular basis for
span(X) and span(Y). Since 4.1 is based on o r t h o g o n a l projections we can translate cos 0 into a m o r e accessible quantity which uses orthogonal bases for span(X) and span(Y). Therefore, define for any /-matrix M by M a matrix with o r t h o n o r m a l column vectors and (possibly) null vectors, such that for some upper triangular l-th order matrix U there holds M =MU (4.3)

( M U can e.g. be thought of as obtained by a G r a m - S c h m i d t process, cf. [14, p. 242 ff ]). We now have:

Property 4.4. Let 0, X and Y be as in 4.1. Then cos 0 = ItX*u Proof." F o r any v and w of suitable dimensions there holds:

IlXv,l=llvll
So IIX*YI[= max v,w,O tv*X*Ywl IIv[I[Iwl[

and

IIYwll=llwll.
[]

Iv*X*Ywl - max -cos& x v , o IlXvll IIYwll

Yw4:0

Property4.5. Let 0, X and Y be as in 4.1. Let Z be an /-matrix such that span(X[ Y) = span(XIZ), and Z* X = 0. Then sin 0 = g lb(Z* Y). Proof: We can write out Y in c o m p o n e n t s of X and Z : Y = X X * Y + Z Z * Y .
Hence for any 1 dimensional vector x we obtain from Pythagoras' theorem: tlX*Yxll2 + IIZ*Yxll2 = [IYxll= = llxll 2. So for x4=0 IIZ*YxII ~ 1 / 1 - cos2 0 = sin o lix[I (cf. 4.4). If x is such that ilX*Yx[I = IIX*YII Ilxll, then the equality sign holds, i.e.

glb(Z*Y)=sinO.
1

[]

The superscript * denotes the conjugate transpose

Dominant and Dominated Solutions of Linear Recursions

429

The next theorem now gives a sufficient condition for uniform d o m i n a n c e in terms of basis solutions and angles, as announced above: T h e o r e m 4.6. Let G and F be a k- and l-solution respectively of [A]. Write Gi=(gi(1)[...[gi(k)) and Fi=(fi(l)[...Ifi(1)). Let O~(j) be the angle between span(giU)) and span(g,(1)l...l&(j- 1)]gi(j+ 1)[...Igi(k)). I f lim i~
t=

m a x lIf/(s)H s=l ..... z -0, min (llgi(t)]l sinOi(t))

1, . . . , k

then G dominates F uniformly. Proof: Let h~espan(Gi) be such that h~(g~(1)]...lgi(k-1))=O. Since for any k
dimensional x

IIGi xtl 2 = II(gi(1)l...Igi(k- 1))*Gi x[I 2 + IIh*G, xl[ 2

we obtain

g lb(G,) >=Ilh~gi(k)II ~ g lb(h*g~(k))ITg,(k)H = sin O,(k)Ilg,(k)II,

cf. 4.5. Since we can p e r m u t e the columns of G i resulting in a matrix GiP say, where glb(Gi)=glb(GiP), we thus find

glb(G,)>= min
t= 1..... k

IIg~(t)llsinO,(t).

(1)

M o r e o v e r we have

lub(Fi)= ttFilI < l~l]Fill~ =n m a x tlf/(s)tl 2

s=l,...,l

(2)

In view of 3.8, c o m b i n a t i o n of (1) and (2) and a limit a r g u m e n t prove the assertion. []

Remark 4.7. Obviously, the condition in 4.6 is not fulfilled in example 3.5, where
we have m a x IIf/(s) ll min(llg,(t) It sin 0,(t)) = (i + 1) 1/3. An other application of 4.1 is a m o r e or less quantitative interpretation of the skewness of a transformation (a matrix T is called skew if for some directionally well separated vectors their images under T m a k e a small angle). Indeed, let T be a nonsingular transformation and 1/ be the smallest angle r/ between the images of two o r t h o g o n a l linear subspaces, then c o t g ~ = tl Tll I1T - i l l . This result follows from the well k n o w n result in [1], where it was shown for vectors (note that our definition applies to spaces). The quantity 2 I1"I[1 denotes the linear norm

430

R.M.M. Mattheij

iI TIT IIT - 111 is called the condition number. In the sequel we shall use sequences of transformations (for the solutions of [A]), which should not be too skew in order to preserve dominance properties (cf. e.g. 5.9, 5.11). This motivates the following:
Definition 4.8. Let {T/} be a set of nonsingular matrices. If 3~>1 ~ IIT~ll IITi-~ II _-<~, then {T~} is said to be of bounded condition. Especially for partitioned matrices T~=(T~I[T~2) we thus have: Property4.9. Let {T~} be a set of nonsingular matrices. Let for some ~:, and all i, []Ti]] ]1Ti-1 l] < ~c. Let 0 i be the angle between T/a and T/2. Then
t~2--1 ViCOSOi <= 2 + 1 9

Proof: Let xespan(Ti 1) and yespan(T~2), x,y:l:O. Then there exist v, wcR, with
v w, such that x = T/v and y = T/w. Hence, with ti as above (viz ~c=cotg ~), w e xZ_l have cos 0 i< cos r/= x2 + 1" []

Remark 4.10. The estimate in 4.9 does not pretend to give a sharp estimate, but only to indicate that bounded condition implies that the angle between T~1 and T~ is uniformly bounded away from zero. This property wilt be used e.g. in 5.17, 2 5.18.

5. Decomposition of Fundamental Solutions

If a k-solution G of [A] dominates an (n-k)-solution F of [A] uniformly, then forward recursion of (1.1) is unstable for any fespan(F). To overcome this instability we would like to have a representation from which the dominant solutions are "removed", i.e. a (recursive) formula that is only satisfied by elements of span(F), or, at least, which is such that the influence of elements of span(G) is hardly felt. Such a formula is given in w and its computational aspects are considered in [8]. It is based on transforming the matrices A i into blocktriangular form, which - as will appear - is equivalent to computing a certain suitable fundamental solution. In order to appreciate the usefulness of this approach, we show in w6 how this triangularization gives rise to characterizations of dominated solutions, from which elements of span(Fi) for any i can then be approximated. An important point in this is this is the way in which G and F, i.e. (GIF), are transformed. It is the aim of this section to give a geometrical interpretation of transforming (GIF) and its implications with respect to dominance. Let L i be a nonsingular matrix such that K~Li(Gi[F~) is block upper triangular. This can also be rewritten as a decomposition for (G~]F~),viz (GiIF~)= L71K~, which is a more convenient form to interpret. First consider the situation where

Dominant and Dominated Solutions of Linear Recursions L i is unitary. Specifically, let H i be such that (Gi[Hi) is unitary and (GilF~)=(GilHi) p22],P1 of order k.

431

(5.1)

Clearly p12 and p22 represent projections of F~ into G i and H i respectively. If the angle between G i and F~ is not small, then the projection of F~ into G i will not be large with respect to the projection of F~ into Hi; so the magnitudes of F~ and H * F i are expected to be of the same order. We work this out in more detail for the case that (GilF~) has full rank; note that it follows from 4.5 that glb(H*F/)>0, so [ H * F i ] 1 exists.

Property5.2. Let (Gill:i) have full rank. Let 0 i be the angle between Gi and F/.
Then I u b(G* Fi[H* Fi] - 1 = cotg 0J-

Proof: From 4.5 it follows that g / b ( H * Fi)-sin 3 i. Hence, using 4.4,

I u b(G* F~[H* F/] - ~) __<l u b (G* F~)/g 1b (H* Fi) = cotg 0 i (1)

Let x be such that HG*FixN=cosOiI[x[I. Define yl=G*Fix and y2=H*Fix. Then ]]yl ]]2 + [[y2][2 = [[X[[2. S o

IIG.Fi[H.E]-ly2
[]y2 ii

[[ylll --ily2[[- cotg 0i" []

Therefore the equality sign in (1) holds.

Obviously, if (Gi[Fi) has full rank we can define

Oi=

(G* Hi) [ H * F~] -1

(5.3)

N o w write (Gi,Fi)=(Gi[H~)(Iq50/i)(lq5 and Hi.qSiFi) (5.4)

for

some

~/22 then combination of (5.1), (5.4) and (5.5) yields (p/~l

e?22 ] = ( i 0 0 i ) ( ~ 1 p/2

0 H.F.~22).

(5.6)

432

R.M.M. Mattheij

If the angle between G~ and F~ is not large, i.e. if [lOll[ is not large (see 5.2) then ~22, p/22 (see (5.6)) and hence p z2 and F i (see (5.5)) have indeed magnitudes of the same order. If we drop the requirement that (G~[F~) has full rank, a similar interpretation of the decomposition (5.1) can be given. We shall not carry this out explicitly, but consider - more generally - decompositions with not skew (hence nonsingular) transformation matrices T~ (T~=L] 1 above). So assume we can decompose (G~IFi) as (GiIF~)= T~

V22 ], V~~ of order k.

(5.7)

Remark 5.8. By assumption T~~ has full rank, so Vii1~ is nonsingular and span(G~)
= span(Til). The next theorem shows how the magnitudes of G~ and F~ are related to those of V~11 and Vii22 respectively. Theorem 5.9. Let T~ be as in (5.7). Let O~denote the angle between F~ and G~. 7hen 1 <glb(V/l') IIV/111[<

[IZ~[I= glb(G,)' HG~ -=}[r~-lll'

sin 8i

glb(F~)<=glb(V~2Z)<=IIT~- ~[I g/b(F~)

and sin ,9i ]Irijl IIF/II<IIViZ2II<IlTcltl IIFill.

Proof." We omit the indices i. Write (U 11 [ T - 1 ] , =(~-II~2)=(TIITz ) \ 0

where ~1 and U 11 have k columns. Hence

(1)

l[ U221t = [l't2 ue2l[ < lIT- z II.

Moreover lIEU123- I[I __<11 TIt, so

(2)

glb(U=~) -- i/liEUt']-~
Since V 2 = T - tF, we find using (1)
v 22

[I > 1/tl TI[.

(3)
(4)

= E~']*F--(u~)*ET~]*F

Since ~-2 _L T ~, application of 4.5 yields g I b (['1"2]* F) = sin&

(5)

Dominant and Dominated Solutions of Linear Recursions From (4) we find the following inequalities

433

g l b(U 22) g I b([]723 * F)IIV II< ]1V22 II --< IIU 22 II II[i~2] * F II IlF II

(6) (7)

glb(V22)glb([~F2]*F)glb(F)<glb(V22)<IIU221111[TE]*Fllglb(F)

Using (2), (3), (5) and the estimate ]l[~'2]*FH----< 1, the second part of the assertion then follows from (6) and (7). The first part is analogous, but somewhat simpler: Indeed, from G = T 1 V 11 we can show similar estimates as (6) and (7) to hold, by interchanging the roles of T and T-1. []

Remark 5.10. The estimates in 5.9 clearly depend on the angle 0 i, the norm of T~ and of the conditionnumber IlTil[ ]lT~-iill, i.e. of the skewness of Ti. If ]lT/]l and I[Ti[[ I[T/-1H are of the order of unity then Vili and G i on one hand and V/22 and F~ on the other hand have respective magnitudes of the same order, Vii and F~ 22 with the additional assumption that 0 i should not be small. As a consequence of 5.9 and Property 3.8 we have:
Corollary 5.11. Let Ti be as in (5.7) and let {Ti} be of bounded condition. If G dominates F uniformly there holds lim lub(Vi22) i ~ g/b(V/'1) 0.

In order to find a characterization for span(T~-lF~) we would like to have a quantity like O~ in (5.3). Therefore we introduce:

Definition 5.12. Let S denote the ( n - k) dimensional space on which Viz2 (cf. (5.7)) is defined. Then the direction (2~ is defined as a mapping on Vi22S by

Vs~s Vi12 S= ~'~iWi22S.

Combination of (5.7) and (5.12) yields an analogue of (5.6): (GiIFi)--Ti ( I 0 ~i)(V01' ~z2) (5.13)

If T~=(G~IH~) as in (5.4) i.e. if T~ is unitary, then [tf2~ll equals the cotg of the angle between G~ and F~ (see 5.2). Also more generally, O~ indicates something like a direction of T~-IFi with respect to T~-1G~, which explains the name in 5.12 (see also 5.15).

Remark 5.14. Since F~= T~ (Oi~) Vi22, it follows from 3.4, that f2~ is unique for a
given T~, i.e. for any f, dominated by some k-solution, there holds (T/-lf~)~ =s ~- ~f~)2. For such an f we therefore call f2 i the direction of T~-lfl. F r o m 5.14 we see that the direction f2~ induces a subspace cII~" of i-th iterands of dominated solutions. This important characterization will be used further on for their approximation.

434

R.M.M. Mattheij

The next theorem gives a more or less quantitative geometrical interpretation for the direction. Theorem 5.15. Let Ti be as in (5.7). Let rli be the angle between Ti 1 and Ti 2 and let
0 i be the angle between Fi and G i. Then

cotg 0 i IIT~II ilT,-'il

cos rh < IlQill < IITill liT~- ill (cotgOi+cosrli).

Proof: (a) We omit the indices. Let H G and write

(G{F)=(G,H) (I 0 If r a n k ( F ) = n - k ,

G'F\

then A = O , see 5.2 and (5.3). Similar to 5.2 we can prove: ]IA II= cotg 0 (2)

(b) Let F = F P 2z. Then we derive from (1) F = ( G I H ) (~) H * F P 22 (3)

As in the proof of 5.9 we define T - l = [-~x]~.2],; so we find from (3) and (5.13)
(m 22 (IT1] *~ I)V =\

[~']*n~ [,,2],H]
and

( A ) H * F P 22

(4)

For any s e S (see 5.12) define

l) 2 = V 2 2 5

02 = H * F p 2 2 s

(5)

Since both span(T 2) and span(H) are the orthogonal complement of span(G), we have span(7 "2) = span(H), so [,~-2], H is nonsingular. From (4) and (5) it now follows that v2= [,T2]*H/32. Hence: tIQII = m a x 11(2v211- m a x II{[~r~]* GA + [TI]*H}/321I 11/321[ v2.o IIv21~-v2.o 1t/3211 II[T2]*H/3211 We have the following estimates: II[~r~]*Gl[ < 11T - 1 II,
1 1 1 - - >-g l b ( [ ' T 1 ] * G ) = t1([7"1]*G)-111 IIG*Tlll --IITll

(6)

(7)
(8)

Dominant and Dominated Solutions of Linear Recursions

435

(note that 7"*(GIH) is block upper triangular, and [ G I H ] - 1 = [GIH]*).

I1([~] * H)-

:11 = IIH* T 211< II T tl

1

(9)
1

g 1b(([~.2], H ) - 1 ) _

IIEW2q*all = 11T-1 II

>

(10)

Finally, since ~1

2 we

have from Pythagoras' theorem and 4.5 (11)

II[~]*HII ~ II[~?a]*HII I1~'111= ~ 1 - II[Te]*HII= II~a I1~ cos r/ll T - : II Application of (7), (9) and (11) to (6) yields Ill211--<-- T - all Ilzl II + IIT - all cos t/} I1T]I {ll and application of (8), (10) and (11) to (6) yields

(12)

11~2tl

>={1

i 1 ~ IIA II- cos r/II T-111 liT_ ~11

}1
[]

(13)

(since for matrices K, L and M of suitable dimensions there holds

NKL + MII ~ g l b ( K ) l u b ( L ) - lub(M)).

The assertion now follows from (2), (12) and (13).

Remark 5.16. The upper and lower bounds in 5.15 are sharp in the sense that

they are equal for unitary T~. Apparently, the lower bound might be negative, i.e. useless if T~ is too skew. Anyway, we can draw the conclusion that a skew T~ may be the cause of a fairly large llf2ill, even if 8~ is not unduly small. A large llf2ill is an indication that the norm of the first part of the vector T~- aft, i.e. ll(T~- :f~)l II (fGspan(F)) might be large with respect to IIf~ll ; this in turn is undesirable if the magnitude of Vii has to be of the same order as that of F~ (cf. 5.10), as appears 22 to be useful further on (viz w7). We can give an analogue of 5.9 which estimates f2iV~22 in terms of T~ and F~: Theorem 5.17. Let Ti be as in (5.7). Let tli be the angle between Ti: and Ti 2 and 0 i be the angle between F1 and G i. Then

-IIBII {cos 0 , - c o s ~,11T~-: II IIT~II}~ IIO, V~22 II ~ IIGII IIT~- '11 {cos ~i+ cos ~i} IIT, ll
Proof." We omit the indices i. Since T I ~-2 (for 7" see proof of 5.9) we have

~rl ___ [ ~ 2 ] , ~rl + T l [ T ' ] * T'. ~-=

436

R.M.M. Mattheij

The estimates can now be derived in a similar way as in the proof of 5.9 and 5.15, from the observation that y2V22 = [~el]* F = ([~v13"I"2) ([~v2]*F) + ([~l]*TX) (IT*]* F) and

lub(K + L) <= lub(K) + lub(L)

and

lub(KL + MN) >= glb(K) lub(L)- lub(M) lub(N)

for suitable K, L, M and N. []

Theorem 5.17 confirms what has already been observed above, viz. that a skew transformation matrix may give relatively large first components of a transformed dominated solution. As a consequence of 5.17 we have (cf. 5.11)

Corollary 5.18. Let T~ be as in (5.7) and let {T~} be of bounded condition. If G

dominates F uniformly there holds

l u b ( ~ i [7//22)

!im glb(V,~)=0. l~o0

6. Finding a Dominated Solution from a Fundamental Solution; Consistency

The result of w5 does not seem very practicable, in as far as explicit knowledge of (GtF) is assumed. However, since any fundamental solution, K say, can be written as K=(GIF)L, for some nonsingular matrix L, we can try to transfer the characterizations - as found in w for the transformed G and F - onto transformed matrix solutions derived from K. This will be done in this section; more specifically, we shall show how we can derive approximations to f2i (as defined in w from a fairly general K, without explicit knowledge of the matrix L (and of (GIF) of course). This means that we can find approximations to span(F3 (since span(F3 c span (T~ ( ~ i ) ) , see 5.14) and this is the basis of the app roximation method of w7. For K we assume a requirement as given in the following

Definition 6.1. Let (~ be a k-solution of [A] and P an (n-k)-solution of [A] such

that ((~]_F) is a fundamental solution. Then a fundamental solution K is said to

be-k-consistent with ((~]P) if the k-th order principal submatrix L11 of

L ~ ((~0[Po)- 1K0 is nonsingular. We have

Property 6.2. Let K, (~ and P be as in 6.1. Then K is k-consistent with ((~[P) iff
span(K~) c~span(F0) =0.

Proof. Suppose L11 (see 6.1) is nonsingular. Then for any wespan(K~) there exists a k dimensional v such that w=K~v=Go Lll V+PoL2~v. If wEspan(Fo)
then v must be zero.

Dominant and Dominated Solutions of Linear Recursions

437

Suppose on the other hand that L 11 is singular, then for some non null v we have d o L 1 1 v = 0 . Define w=K~v. Clearly w=ffoL21v. Moreover, since rank(KX)=k, w must be nonzero, i.e. span(K~)c~span(/eo)4=0. [] We now presuppose (as before) the existence of a k-solution G which dominates an ( n - k ) - s o l u t i o n F uniformly and also assume k-consistency of some fundamental solution K with (GIF). We obtain: Theorem 6.3. Let (GIF) be a fundamental solution and let G dominate F uniformly. Let K be k-consistent with (G]F). Then K 1 dominates F uniformly and there exists

a k x (n - k ) matrix M 12 and a nonsingular ( n - k ) x (n - k ) matrix M 2z such that K

= ( K t l K 1 MlZ+FM22).

Proof. Let L = ( G o l F o ) - I K o . Since L 11 is nonsingular (cf 6.1) and (Gi[Fi)L=Ki, we find glb(K~) > glb(L 11) glb(Gi)_ lub(Fi) lub(I~l). Hence
lira glb(K~) i~o~ lub(Fi)

cO,

and K 1 dominates F uniformly by 3.8. For any ( n - k ) dimensional vector y2 define yl = _ [L 1 i ] - 1 L12 y2. Then

0
L y = L ( ~ ) = ( _ L 2 1 [ L l l ] - I L12 + L22) y2

If ~I22~-L22-L21[Llt]-IL12
D

were singular, then L were singular, which it is

not; so ~r 22 is nonsingular. Since K I ( = G U 1 + F L 21) and F constitute a fundamental solution we can d e c o m p o s e K 2( = GL 12 + FL 22) as K 2 = K 1M 12 + F M 22, for some matrices M t2 and M 22. Writing out gives M 1 2 = [ L l l ] - I L 12 and M 22

=~22.

[]

Remark 6.4. Since span(F) is unique (cf 3.4), the k-consistency property does not
depend on the particular choice of F neither on that of G, cf 6.2. Therefore we shall simply call K k-consistent if it obeys 6.2.

Remark 6.5. One m a y ask whether a k-consistency requirement for K is very

restrictive or not. Considering this question theoretically, one might answer regarding 6.1 - that the set of all nonsingular k-th order matrices is dense with respect to the set of all k-th order matrices. Practically, this means that for a r a n d o m choice of L tl this matrix is very unlikely to be singular. In fact it is a similar requirement that is needed for a successful application of m e t h o d s based on power iteration, such as the QR algorithm [14, ch 8]. In [8] we return to this question (cf also 6.13). Since it follows from 6.3 that K 1 dominates F (so F M 22) uniformly, whilst M 22 is nonsingular (so span ( F ) = span(FM22)) it is not restrictive to identify K 1 with G and F M 22 with F respectively as used in w5. Hence let (I

~12~
I ]'
(6.6)

K-

438

R.M.M. Mattheij

for some /~lz. Now let T~ be a transformation matrix as in (5.7) (i.e. one which brings (GilFi) into block upper triangular form), then W/(0)~ T~-IKi will have a block upper triangular form. We obtain, using 5.13: K , = T~ (W~lf (0) W/12(0)~

W/22(0)] =T/(0I ~i)(V~ 11 V/022)(I0 M/12).

(6.7,

Remark 6.8. Since V~11= Will(0) and V~Z2= W/22(0), we see that the magnitudes of W/11(0) and W/22(0) are of the same order as those of G i and F~ respectively, if the angle between G i and Fi is not unduly small (cf 5.9 and 5.10). In order to show how the O~ can be approximated, we introduce matrices Wi(l) as follows: Definition 6.9. For any i and all l < i, let W~(1)be a block upper triangular matrix (i.e. with W~21(l)=0) such that W~(0)= W~(l) W~(0). From 6.9 we derive the following relations:

(a) W~I(0)= W~l(1) Wz*l(0)

(b) W~22(0)= W~22(/) Wt2/(O) (6.10)

(C) W/12(0)= Will(l) W/12(0)-~ W/12(l) W/22(0).

Since Will(0) is nonsingular (cf 5.8, and Wi11(0)= Vii11), we see from (6.10) that Wi11(1) is nonsingular. We can now prove the following result: Theorem 6.11. Let {Ti} be of bounded condition and let for all i span(Til) = span(Gi) (G a uniformly dominant k-solution). Then for any fixed l we have (2~= - lim [W~I l(l)] * W~*2(1).
T M

i~oo

Proof: From (6.10) (c) it follows that

[W/II(1)] -I W/12(I) W/22(0):[W/I I(1)]- 1 W/12(0)- W/12(0)

(I)

Since W/11(1) = V/I1 IV/11]-i, we obtain for the first term on the right in (1), using the explicit expression for W~12(0) in (6.7):

[WI i(l)]-i W12(0)= V11 /Q12+ [I/i i]-i O, Vii22

For I fixed and i--* oo, we obtain from (2) using 5.18:

(2)
(3) (4)

[w?,(0]- 1 w? 2(0)~ v? 1 ~12

Moreover from (6.7) with i = l we note that

w,12(0)= v, 'I ~ r ~ 2 + o , v, 2~
Since WlZZ(0)= Vl2z we therefore obtain from (1), (3) and (4) [W/11(l)] -1 W/12(1) Vl22--+ --~, Vt22, 1 fixed, i--~oo. []

Dominant and Dominated Solutions of Linear Recursions

439

Remark 6.12. In the proof of 6.11 we used a result of 5.18 which is based on uniform dominance. If G should be only dominant then 6.11 is not necessarily true. This can be demonstrated using the M V R in 3.5. Let K=(G[F) and V~T~ = I. Then
[_W//ll(0)]_I=

i~i/ Obviously i~|

1ti'1 1
=

i+1) 2 0

1 ; W/~2(0)=

i+1

[W~11(0)]- ~ W~12(0) d~

n~ exist' whereas f2~ (i)"

[]

Remark 6.13. In the formulation (6.6) one important problem is camouflaged: the actual choice of K o. It may be that the angle between K~ and F o is small which means that IIf~011 is large, cf 5.15 (this is likely to happen if L11 is nearly singular, i.e. if 11[Lll]- 111 is large, cf proof of 6.3). It is therefore very likely that also the angle between K~ and Fi then is fairly small, i.e. Ill, IT fairly large. As remarked in 5.16 this may imply that the magnitudes of V~ and F i are not of 22 the same order, which might affect the effectivity of the algorithm proposed in w (since the latter makes use of an appropriate dominance of glb(Vi 11) over lub(Vi22)). Therefore it may be important to choose such a K that the angle between K o and F 0 is not too small. Obviously, a more careful choice of K o does only make sense if there exists at least some dominant K-solution (7 of l-A] such that d e and F~ do not make small angles. On the other hand, if there exists a dominant k-solution d of which the iterands outgrow F very rapidly, K i will fairly rapidly become directionally close to Gi, i.e. after a few steps Itf2ill will be expected to be moderate anyway. Remark 6.14. The matrix Vii is not necessarily nonsingular, because (Gi[F~) may 22 not have full rank, so that rank(Fi) may be smaller than n - k . This also implies that the matrices W~22(/) are not necessarily nonsingular; in the singular case they even are not unique. In (7.6) we indicate a useful choice for them.

7. The Triangularization Algorithm

In order to apply the preceding results and particularly theorem 6.11 to a given k-consistent fundamental solution K (of which we presupposed existence in w6), we need a set of triangularizing matrices {T~} of bounded condition. Below we give a stable way for determining such a set. Rather than also computing matrices like W~(l) (see 6.9) explicitly, for finding the desired dominated solution, we use the previous results to give a representation for computing the iterates of this solution without computing the W~(1). This is an important fact since direct computation of any matrix solution (or something that resembles this, like {[W/(I)] 11} or {[W~(l)]12}) is unstable if any of its components is dominant; moreover it is unnecessarily costly. First, consider the question of how to find a set {T~}. Define for K as above To = K o (7.1)

440

R.M.M. Mattheij

Then determine recursivety nonsingular {Ti} and block upper triangular (Ui} (i.e. with Ui21 = 0) by A~T~=T~+ 1U,, Since K i = ('17i--1Aj) T O we obtain from (7.2) ,,*_
\j=O--

i>O

(7.2)

K i = T~

. .s = 0

(7.3)

from which we see that the T~ decompose the matrix Ki (which is not computed actually) in the desired way. A most recommendable way for finding the above indicated set {T~}~>__, is based on a Q - U decomposition by the methods of Givens or Householder (cf [14, pp. 240, 236]), in which case the T~ are unitary, so {T~} is of bounded condition. The triangularization then runs as follows: At step i multiply the recursively found matrix T~ on the left by A~ and decompose A~ T~ into a unitary and an upper triangular matrix (as is known the matrix T~+, is not necessarily derived in its explicit form). Since unitary matrices are very stable it is hoped that the upper matrices U~ are fairly accurate; it can indeed be shown that the error is bounded in norm by a quantity of the order of 2 - ' ]IAiHF (where 2 - ' is the usual symbol for the relative computer accuracy, cf [7, p. 148 ff] and II. 1l2 is the sum of squares of the elements). An other way for finding sequences {T~} and {U~} might be based on some kind of L - U decomposition, i.e. where the matrices T~ are lower triangular (with a possible exception for To). There are two main disadvantages of such a L - U decomposition in general. First, it is not directly clear, whether dominance and k-consistency are sufficient conditions for existence of a set {T~} without pivoting (at least without interchanging one of the first k columns on the one hand and one of the last ( n - k ) columns on the other hand during a decomposition). Second, the bound for {llr~tl IIr~-~ll} may be fairly large, if it exists, which may cause larger errors in the computed U~ (and also in transforming vectors like f~ or their approximants by T~ or T~- 1). We now come to the approximation of a transformed dominated solution f , say. Define Bi 0 Ci~, E i] B i of k-th order (7.4)

Ui=

note that B i is nonsingular, since Moreover define f2p. q = -

Bj

is nonsingular, cf 5.8 and (7.3) .

l=p

Bj

CI 1~ Ej . j=p

(7.5)

i 1 Then we can identify W/(1) of 6.9 with 1-I ui, so j=l

Dominant and Dominated Solutions of Linear Recursions

i-I

441

(a) W~I'(I)= H B~ j=l

i--I

(7.6a)
(7.6b)

(b) W/22(1)= H Ej
j=l i-1

From 6.11 we obtain

Property 7.7. Let {Ti} be of bounded condition, then Yl lim ~?t,i=Ov

i~oo

Now introduce for the solution f: J~= T~- 1f~, i>0. (7.8)

Then we find from f/+ l=Aifi, (7.2) and (7.4):

~+ l = Bi ~ l q- Ci fi 2 9~2 1 = Ei fi2

(7.9a) (7.9b)

It is seen that the triangularization results in a decoupled recursion for {J~/}. In our problem setting we assumed that f was not dominated by any other element of span(F), which means that it is not unrealistic to expect Ilf~lI to be of the order of

llf~ll,

i.e. to be of the order of

cf 5.10, and note [ ] E j = Vii 22

0

unless the angle between G~ and Fi is unduly small (cf 5.9). Since j~l =Q~j~2 (cf 5.14), it then follows from 5.17 that IIJ~ll will not be large with respect to IIf~tl, i.e. IU~211 is of the order of IIf~ll. Therefore our conclusion is that (7.9b) is expected to be stable in forward direction. Once {~2} has been found, (7.9a) can be used to determine {j~l} (of course, computation of f l via an explicit approximation of ~?~ is very inefficient, if at all stable). Note that considering V~ 2 as a known vector, (7.9a) is an inhomogeneous recursion for {~1}. Its particular solution {~1} is uniformly dominated by the solutions of its homogeneous part (see e.g. 5.18). Hence we can expect an analogue of Miller's algorithm (see w3) to give arbitrarily accurate approximations to ~1, i.e. we have to use backward recursion. Details as well as error analyses of such an algorithm can be found in [8].

Remark 7.10. Since we only need a block upper triangular Ui, it is not necessary to perform the triangular decompositions completely. So e.g. in Householder's method, which requires the computation of ( n - 1 ) elementary hermitians, we can restrict ourselves to only computing the first k. This gives a factorization of the form:

442

R.M.M. Mattheij

Remark 7.11. Instead

of triangularizing at each step i, we may also triangularize subsequent products of the A i. So e.g. if T~ has been found, we can form the product Ai+lAi+t_ 1... Ai Ti, where l is some integer, and decompose this matrix.
i

Of course this may cause substantially larger errors (cf Householder's method, where the expected error in the upper triangular matrix will now be of the order
i+l \

2-t [I_,Aj FJ'_ For

some applications this larger error may not be dramatic

whereas such a strategy is more efficient. The computation of a solution via upper triangular matrices, as sketched above (cf (7.9)) wiI1 only give the values of the thus found iterands at indices i for which also the matrices T~ have been found; values at intermediate indices may be found using [A] in forward direction. A method along these lines can be regarded as an analogue to the shooting method, which is used for boundary value problems in ODE.

References
1. Bauer, F.L., Householder, A.S.: Some inequalities involving the euclidean condition of a matrix. Numer. Math. 2, 308-311 (1960) 2. Davis, C., Kahan, W.M.: The rotation of eigenvectors by a perturbation III. SIAM J. Numer. Anal. 7, 1-45 (i970) 3. Deuflhard, P.: A summation technique for minimal solutions of linear homogeneous difference equations. Computing 18, 1-13 (1977) 4. Gautschi, W.: Computational aspects of three-term recurrence relations. SIAM Review 9, 24-82 (1967) 5. Gautschi, W.: Zur Numerik rekurrenter Relationen. Computing 9, 107-127 (1972) 6. Mattheij, R.M.M.: Accurate estimates of solutions of second order recursions. Linear Algebra and Appl. 12, 29-54 (1975) 7. Mattheij, R.M.M.: Estimating and determining solutions of Matrix Vector Recursions. Thesis, Utrecht (1977) 8. Mattheij, R.M.M.: Stable computation of dominated solutions of linear recursions. Report 7803, Mathematisch Instituut, Nijmegen (1978) 9. Mattheij, R.M.M., van der Sluis, A.: Error estimates for Miller's algorithm. Numer. Math. 26, 61-78 (1976) 10. Oliver, J.: The numerical solution of linear recurrence relations. Numer. Math. 11, 349-360 (1968) 11. Olver, F.WJ.: Numerical solution of second order linear difference equations. J. Res. Nat. Bur. Standards 71 B, 111-129 (1967) 12. Olver, F.WJ., Sookne, DJ.: Note on backward recurrence algorithms. Math. Comput. 26, 941947 (1972) 13. van der Sluis, A.: Estimating the solutions of slowly varying recursions. SIAM J. Math. Anal. 7, 662-695 (1976) 14. Wilkinson, J.H.: The algebraic eigenvalue problem, Oxford: Clarendon Press 1965

Received March 20, 1979/February 18, 1980/July 15, 1980