The Principal Axis Theorem We sketch a proof of the following important theorem. THE PRINCIPAL AXIS THEOREM.

Every n n symmetric matrix has real eigenvalues and there is an orthonormal basis of eigenvectors. First we present an argument that every symmetric matrix has a real eigenvalue. This does not involve the introduction of a complex bilinear form (an general form of the dot product). THEOREM. Let A be an n n symmetric matrix. Then A has a real eigenvalue. PROOF. Instead of the complex bilinear form, we need some a fact from calculus: every continuous real valued function on a closed and bounded set assumes its maximum value on the set. In our case, the continuous function will be f(x) = |<Ax, x>| = |!i,j xiaijxj| and the closed and bounded set will be the unit sphere in "n x2 + + x2 = 1}. Sn-1 = S = {x # "n $ ||x|| = 1 n By this theorem from calculus we conclude that f assumes its maximum % at a point x0 on the unit sphere S, i.e., % = max {f(x) $ x # S} = max{|<Ax, x>| $ ||x|| = 1} = |<Ax0, x0>|. By replacing A by -A if necessary, we may assume that <Ax0, x0> = max{|<Ax, x>| $ ||x|| = 1} = %. We show that x0 is an eigenvector for the eigenvalue %. (If we replaced A by -A, then -% will be the eigenvalue and x0will still be the eigenvector.) First we have that x0 & 0 since ||x0|| = 1. We also have that <(A - %')x0, (A - %')x0> = <Ax0, Ax0> - 2%<Ax0, x0> + %2<x0, x0> = <Ax0, Ax0> - 2%<Ax0, x0> + %2<x0, x0> = <Ax0, Ax0> - 2%2 + %2 = <Ax0, Ax0> - %2. We now complete the proof by showing that <Ax0, Ax0> - %2 ( 0. This will mean that 0 ( <(A - %')x0, (A - %')x0> = <Ax0, Ax0> - %2 ( 0 or that <(A - %')x0, (A - %')x0> = 0, and consequently, that (A - %')x0 = 0. We just calculate this out using the parallelogram law. First we need to notice that x x <Ax, x> = A , ||x||2 ( l||x||2 ||x|| ||x|| for all x & 0. We even get

The Principal Axis Theorem page 1

|<Ax, x>| ( %<x, x> for x = 0 since in this case the inequality is just 0 ( 0. Now for ||x|| = ||y|| = 1, we have x +y x +y x -y x -y <Ax, y> = A , - A , 2 2 2 2 ( A x +y x +y , 2 2
2

x -y x -y , 2 2

(% because x +y 2
2 2

x +y 2

+%

x -y 2

(%

x -y 2

1 4

<x + y, x + y> + <x - y, x - y> =

1 4

2||x||2 + 2||y||2 ( 1.

So we have that |<Ax, y>| ( %||x|| ||y|| for every x and y since as before we can write x y <Ax, y> = A , ||x||||y|| ( l||x||||y||. ||x|| ||y|| So we have that ||Ax0||2 = <Ax0, Ax0> ( %||x0|| ||Ax0|| = %||Ax0|| or that ||Ax 0|| ( %. This shows that ||Ax0||2 - %2 ( 0 as was desired. So x0 is an eigenvector for A corresponding to %. Now we have an induction step. THEOREM. Let A be a symmetric n n matrix and let q1 be an eigenvector corresponding to the real eigenvalue %1. Suppose that ||q1|| = 1 and that q2, , qn are vectors so that q1, ,
qn is an orthonormal basis for "n. q1, , qn respectively. Then

Q.E.D.

Let Q = Q1 be the orthogonal matrix whose columns are %1 0 0 B

QTAQ =

where B is a symmetric (n - 1) (n - 1) matrix. PROOF. We have that AQ = [Aq1, , Aqn] and

(columns)

The Principal Axis Theorem page 2

QTAQ = (qiTAqj)ij. But q1TAqj = <Aqj, q1>. = %1<qj, ATq1> = %1<qj, Aq1> = %1)1j while qjTAq1 = <Aq1, qj>. = %1<q1, qj> = %1)1j. This proves that the first row and column of the product QTAQ is 0 except for the 1,1 entry which is %1. So QTAQ has the given form for some (n - 1) (n - 1) matrix B. The matrix B is symmetric simply because QTAQ is symmetric. Q.E.D. Since B is an (n - 1) (n - 1) symmetric matrix, we can put together an induction argument to get the Principal Axis Theorem. Given a symmetric matrix A, we find an orthogonal matrix Q such that QTAQ = diag{%1, , %n} where the %i are real. Here Q is real the product of the orthogonal matrices Q = Q1Q2Qn where Q1 found in the preceding theorem. Then the eigenvalues of A are precisely the %i since the eigenvalues of diag{%1, , %n} are %1, , %n and QTAQ and A have the same characteristic equation. We also have that QTAQei = diag{%1, , %n}ei = %iqi. so that Aqi =AQei = Q(QTAQ)ei = Q(%iei) = %iQei = %iqi. Q.E.D. 4 -1 0 EXAMPLE. Let A = -1 5 -1 . The eigenvalues are 3, 4, 6 with 0 -1 4 respective eigenvectors -1 1 1 1 , 0 , -2 . 1 1 1 These are orthogonal but not orthonormal. We just need to normalize these vectors and make a matrix Q as follows: 1/ 3 1/ 3 1/ 3 -1/ 2 -1/ 2 -1/ 2 1/ 6 1/ 6 1/ 6 4 -1 0 -1 5 -1 0 -1 4 1/ 3 -1/ 2 1/ 6 300 1/ 3 0 -2/ 6 = 0 4 0 006 1/ 3 1/ 2 1/ 6

022 EXAMPLE. Let A = 2 0 2 . The characteristic polynomial is 220

The Principal Axis Theorem page 3

p(%) = (% + 2)2(% * 4). To find the eigenvectors for % = -2, we consider the null space of A + 2'3, i.e., the homogeneous equations with coefficient matrix 222 222 . 222 The solutions are obtained by Gaussian elimination as usual, viz., first go to upper triangular form 222 111 222 + 000 222 000 and get two vectors x1 and x2 first by letting x13 = 1 and x12 = 0 to get -1 x1 = 0 1 and then by letting x23 = 0 and x22 = 1 to get -1 x2 = 1 . 0 The set S(-2)= {x $ Ax = -2x} is a subspace and so we have that the possibility of applying the GramSchmidt process to x1 and x2 and remaining in S(-2). We get the orthonormal vectors q1 = 1 -1 0 , 2 1 -1 -1 1 1 , 0 2 1 0 -1 -1 1 1 0 = 2 , q2 = 2 -1 6 2 1 -1 2 . -1

-1 q ,2 = 1 0

in S(-2). For the remaining eigenvalue % = 4, we get the eigenvector by solving the homogeneous equations *1 0 1 -4 2 2 -2 1 1 -2 1 1 2 -4 2 + 0 -3 3 + 0 -1 1 + 0 *1 1 . 2 2 -4 0 3 -3 0 0 0 0 0 0 So an eigenvector is( 1, 1, 1)T which becomes

The Principal Axis Theorem page 4

q3 =

1 3

1 1 . 1

Then we have that the matrix Q = [q1, q2, q3] gives -1/ 2 QTAQ 0 1/ 2 = -1/ 6 2/ 6 -1/ 6 1/ 3 1/ 3 1/ 3 022 202 220 -1/ 2 -1/ 6 1/ 3 0 2/ 6 1/ 3 = 1/ 2 -1/ 6 1/ 3 -2 0 0 0 -2 0 . 0 0 4

So the Principal Axis Theorem gives the geometric mapping properties of a symmetric matrix: with regard to a certain orthonormal basis, a symmetric matrix is a stretching. EXAMPLE. THE SQUARE ROOT OF A SYMMETRIC MATRIX WITH NON NEGATIVE EIGENVALUES. Let A be the symmetric matrix 5 1 -1 A= 1 5 1 -1 1 5 The characteristic polynomial is 5 -% 1 p(%) = 1 -1
by

-1 = 5 -%

5 -% 1 1 0

-1

5 -% 1 1

5 -% 1 6 -% 6 -% + (-1)1+2(1) 1 -1

=
cofactors

(-1)1+1(5- %)

5 -% 1 6 -% 6 -%

6 -% 6 -%

= (5 - %){(5 - %)(6 - %) - (6 - %)} - {(6 - %) + (6 - %) = (6 - %)(%2 - 10% + 25 - (5 - %) - 2) = (6 - %)(%2 - 9% + 18) = (6 - %)(% - 6)(% - 3). So the eigenvalues are % = 3, 6, 6. EIGENVECTOR FOR % = 3: 2 1 -1 -1 1 2 1 2 1 + 0 3 3 -1 1 2 0 3 3 + 1 -1 1 2 0 1 1 , x1 = -1 ; and 1 0 0 0 1 1 , x2 = 0 , x3 = 1 . -1 0

EIGENVECTORS FOR % = 6: -1 1 -1 1 -1 1 1 -1 1 + 0 0 0 -1 1 -1 0 0 0

We have to apply the GramSchmidt to the eigenvectors for % = 6:

The Principal Axis Theorem page 5

q2 =

1 1 x2 = ||x2|| 2 1 1 1 , 2 0

1 0 -1 1 0 -1 1 2 1 1 0 = 2 -1 1 2 1

1 q ,3 = 1 0

1 1 1 q3 = q, = 2 . ||q, 3|| 3 6 1
We need to normalize x1 to form an orthonormal system. Then for 1/ 3 1/ 2 1/ 6 Q = -1/ 3 we get QTAQ Setting 3 B= we get that B2 = QTAQ. So we have that QBQT(QBQT) = QB2QT = Q(QTAQ)QT= A, i.e., 1/ 3 -1/ 3 1/ 3 QBQT = 1/ 2 0 -1/ 2 1/ 6 2/ 6 1/ 6 3 0 0 0 6 0 0 0 6 1/ 3 1/ 2 1/ 6 -1/ 3 0 2/ 6 . 1/ 3 -1/ 2 1/ 6 0 0 0 6 0 0 0 6 , 300 = 060 . 006 0 2/ 6 1/ 3 -1/ 2 1/ 6

The Principal Axis Theorem page 6