Eigen Value Decomposition

Usman Shazad (2017-PhD-Phy-01)

Shafa Ullah (2017-PhD-Phy-02)

15-01-2018
Contents

I Introduction 2
0.1 Eigen Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . 4
0.2 Formulism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
0.2.1 Positive (semi-)de…nite matrices . . . . . . . . . . . . . . 5
0.3 Why is eigendecomposition useful? . . . . . . . . . . . . . . . . . 6
0.4 An Application of Eigen decomposition ( Quantum Mechanics) . 6
0.4.1 Eigenvectors and eigenvalues . . . . . . . . . . . . . . . . 6
0.5 Pauli matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
0.6 Diagonalization of observables: . . . . . . . . . . . . . . . . . . . 7

1
 Part I

Introduction

2
 we have discussed the Eigen value decomposition of a matrix .We make use of
Eigen value decomposition (EVD) prominently in the analysis of linear transfor-
mations.The pre…x eigen- is adopted from the German word eigen for "proper",
"characteristic".[4] Originally utilized to study principal axes of the rotational
motion of rigid bodies, eigenvalues and eigenvectors have a wide range of ap-
plications, for example in stability analysis, vibration analysis, atomic orbitals,
facial recognition, and matrix diagonalization.
De…nition:An eigenvector v of a linear transformation T is a non-zero
vector that, when T is applied to it, does not change direction. Applying T
to the eigenvector only scales the eigenvector by the scalar value , called an
eigenvalue. This condition can be written as the equation

T (v ) = v (1)
known as eigenvalue equation or eigenequation. is generally any scalar, it
may be real or complex.Linear transformations can take many di¤erent forms,
mapping vectors in a variety of vector spaces, so the eigenvectors can also take
many forms.For example, the linear transformation could be a di¤erential oper-
d
ator like , dx or r in which case the eigenvectors are functions called eigenfunc-
tions that are scaled by that di¤erential operator, such as
d x x
e = e (2)
dx

r B= B (3)
The last equation is famous Beltrami Condition , where B is some …eld such
as magnetic …eld whose curl is prallel to itself.
The linear transformation could take the form of an n n matrix, in which
case the eigenvectors are column matrices that are also referred to as eigenvec-
tors.Let A be an n n matrix representing a linear transformation acting upon
an eigen vector v , then the Eigen value equation can be written as

Av = v (4)
where the eigenvector v is an n by 1 matrix. For a matrix, eigenvalues and
eigenvectors can be used to decompose the matrix, for example by diagonalizing
it.
Eigenvalues and eigenvectors give rise to many closely related mathematical
concepts, and the pre…x eigen- is applied liberally when naming them:

The set of all eigenvectors of a linear transformation, each paired with its
corresponding eigenvalue, is called the eigensystem of that transformation.

The set of all eigenvectors of T corresponding to the same eigenvalue,

together with the zero vector, is called an eigenspace or characteristic
space of T.

3
 If the set of eigenvectors of T form a basis of the domain of T, then this
basis is called an eigenbasis.

0.1 Eigen Decomposition

Eigenvectors and eigenvalues are numbers and vectors associated to square ma-
trices, and together they provide the eigen-decomposition of a matrix which
analyzes the structure of this matrix.Even though the eigen-decomposition does
not exist for all square matrices, it has a particularly simple expression for a class
of matrices often used in multivariate analysis such as correlation, covariance,
or cross-product matrices. The eigen-decomposition of this type of matrices is
important in statistics because it is used to …nd the maximum (or minimum)
of functions involving these matrices.This decomposition generally goes under
the name "matrix diagonalization." But this is just a formal name given to it,
since the process being described is really the decomposition of a matrix into a
product of three other matrices, only one of which is diagonal, and also because
all other standard types of matrix decomposition use the term "decomposition"
in their names, e.g., Cholesky decomposition, Hessenberg decomposition, and
so on.As a result, the decomposition of a matrix into matrices composed of its
eigenvectors and eigenvalues is called eigen decomposition in this work.

0.2 Formulism
Suppose a square matrix A has non degenerate eigen values 1 , 2 ; 3 ,..., k and
corresponding linearly independent eigenvectors X1 ,X2 ,...,Xk which can be de-
noted
2 3 2 3 2 3
x11 x12 x1k
6x21 7 6x22 7 6x2k 7
6 7 6 7 6 7
6 : 7 ; 6 : 7 :::::::::::::: 6 : 7
6 7 6 7 6 7
4 : 5 4 : 5 4 : 5
xk1 xk2 xkk
De…ne
2 the matrices composed 3 of eigenvectors
x11 x12 : : x1k
6x21 x22 : : x2k 7
6 7
P=6 6 : : : : : 77
4 : : : : : 5
xk1 xk2 : : xkk
and2Eigen values as 3
1 0 : : 0
60 0 : 07
6 2 7
6
D=6 0 0 : 07
3 7
4: : : : :0 5
0 0 0 0 1
Where D is a diagonal matrix.then we can write

4
 AP = A X1 X2 : : Xk (5)
= AX1 AX2 : : AXk (6)
= 1 X1 2 X2 : : k Xk (7)
2 3
1 x11 2 x12 : : k x1k
6 1 x21 2 x22 : : k x2k 7
7
6
= 6
6 : : : : : 7
7 (8)
4 : : : : : 5
1 xk1 2 xk2 : : k xkk
2 32 3
x11 x12 : : x1k 1 0 : : 0
6x21 x22 : : x2k 7 60 0 : 07
6 76 2 7
= 6
6 : : : : : 7 6
76 0 0 3 : 07 7 (9)
4 : : : : : 4:
5 : : : :0 5
xk1 xk2 : : xkk 0 0 0 0 1

Hence

AP = P D (10)
giving the amazing decomposition of A into a similarity transformation in-
volving P and D.
we can write from above equation as
1
A = P DP (11)
The fact that this decomposition is always possible for a square matrix A as
long as P is a square matrix is known in this work as the eigen decomposition
theorem.
Furthermore, squaring both sides of equation (11) gives

A2 = P DP 1
P DP 1
(12)
1 1
= PD P P DP
2 1
= PD P
By induction, it follows that for general positive integer powers,

An = P Dn P (13)

0.2.1 Positive (semi-)de…nite matrices

A matrix is said to be positive semi-de…nite when it can be obtained as the
product of a matrix by its transpose.This implies that a positive semi-de…nite
matrix is always symmetric. So, formally, the matrix A is positive semi-de…nite
if it can be obtained as:
A = XX t (14)

5
for a certain matrix X (containing real numbers). Positive semide…nite matri-
ces of special relevance for multivariate analysis positive semi-de…nite matrices
include correlation matrices. covariance, and, cross-product matrices.
The important properties of a positive semi-de…nite matrix is that its eigen-
values are always positive or null, and that its eigenvectors are pairwise orthog-
onal when their eigenvalues are di¤erent.Because eigenvectors corresponding to
di¤erent eigenvalues are orthogonal, it is possible to store all the eigenvectors
in an orthogonal matrix (recall that a matrix is orthogonal when the product
of this matrix by its transpose is a diagonal matrix). This implies the following
equality:
1
P = Pt (15)
We can, therefore, express the positive semi-de…nite matrix A as:

A = P DP t (16)

0.3 Why is eigendecomposition useful?

Eigendecomposition decomposes a matrix A into a multiplication of a matrix of
eigenvectors V and a diagonal matrix of eigenvalues D. This can only be done
if a matrix is diagonalizable:

0.4 An Application of Eigen decomposition ( Quan-

tum Mechanics)
0.4.1 Eigenvectors and eigenvalues
When an operator A b acts on some vector j i, one usually obtains a new vector
0
j i.But one can easily imagine the special situation where the result is the
same vector or
a vector proportional to the initial vector j i. Then we have

b i = aj i
Aj (17)

where the number a in general is complex. We then say that j i is an

b and a is the corresponding eigenvalue.
eigenvector for the operator A

0.5 Pauli matrices

The Pauli matrices are a set of three 2 2complexmatriceswhichareHermitianandunitary:U suallyindicatedby

6
 0 1
x = (18)
1 0
0 i
y = (19)
i 0
1 0
z = (20)
0 1

The Pauli matrices form a basis of all Hermitian 2 2matrices : Inotherwords; everyHermitian2
2matrixAcanbewrittenas

A= x+ y+ z+ 1 (21)
with real ; ; ;

0.6 Diagonalization of observables:

Every (…nite-dimensional) observable A can be written as

A = P DP t (22)
which is same as equation (11) , a de…nition of Eigen value decomposition.
Here , D is a diagonal matrix and P satis…es P P 1 = P P t = I