Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Topics
z z z z z z z z z
Rank of a Matrix Homogeneous and Non-Homogeneous Cramers Rule Inverse Method QR-Decomposition Gaussian Elimination Lower-Upper (LU) Decomposition Cholesky Factorization Eigenvalues and Eigenvectors
ECE-MATH321 engradc (c)2011
Rank of a Matrix, r
z
It is the greatest value of r for which there exists an r x r submatrix of A with non vanishing determinants. The rank of an m x n matrix can at most be equal to the smaller of the numbers m and n but may be less.
r m < n OR r n < m
ECE-MATH321 engradc (c)2011
Rank of a Matrix, r
z
r < n if A = 0
z
Full rank can be achieved if and only if A is a square and non-singular matrix.
r = n if A 0
ECE-MATH321 engradc (c)2011
Example
z
4 3 2 1 a.) A = 6 4 2 8
2 4 3 6 b.) A = 3 6 5 9 1 2 1 3
ECE-MATH321 engradc (c)2011
a11 x1 + a12 x2 + ... + a1n xn = b1 a21 x1 + a22 x2 + ... + a2 n xn = b2 a31 x1 + a32 x2 + ... + a3n xn = b3 ...... am1 x1 + am 2 x2 + ... + amn xn = bm
z
General Form
z
Ax = b
ECE-MATH321 engradc (c)2011
Ax=0
thus, The given linear equations are homogeneous
z
If vector b is not a null vector thus, the given linear equations are non-homogeneous
ECE-MATH321 engradc (c)2011
If the system of linear equations is homogeneous i.e. b = 0, and A is non-singular square matrix (full-rank), therefore the system has only trivial solution i.e.
x1 = x2 = x3 = = xn = 0
ECE-MATH321 engradc (c)2011
If the system of linear equations is homogeneous i.e. b = 0, and A is not a full rank matrix whose r < m, consider only r equations whose coefficient matrix has a rank r and omitting the other mr equations. There will be an infinite solution to the system
ECE-MATH321 engradc (c)2011
Examples
z
4y = 0, x + y = 0 b. x + y + z = 0 2x y 5z = 0 x y 5z = 0
ECE-MATH321 engradc (c)2011
Examples
z
a.
x + y + z = 0, 2x y 7z = 0, x y 5z = 0 b. 2x + y + z = 0 4x + 3y z = 0
ECE-MATH321 engradc (c)2011
If the system of linear equations is non-homogeneous i.e. b 0, and A is not full-rank matrix: Let
a12 a 22 : am2 ... a1n a11 a 21 ... a 2 n , B= : : : ... a mn a m1 a12 a 22 : am2 ... a1n ... a 2 n : : ... a mn b1 b2 : bn
a11 a 21 A= : a m1
b.
c.
Cramers Rule
b1 b2 D( x1 ) = : : bn a12 a 22 : : an2 .. .. a1n .. .. a 2 n : : : : : :
a11 a 21 D( x 2 ) = : : a n1
b1 b2 : : bn
.. .. a1n .. .. a 2 n : : : : : : .. .. a nn
.. .. a nn
a11 a 21 D( x n ) = : : a n1
a12 a 22 : : an2
.. .. b1 .. .. b2 : : : : : :
ECE-MATH321 engradc (c)2011
D ( xn ) D( x1 ) D ( x2 ) x1 = , x2 = , ..., xn = A A A
.. .. bn
Examples
z
2x 5y + 13z = 24 3x + y 6z = 2 x y + z = 4, x + y z = 0 2x 3y + z = 4, 3x + 2y 3z = 1
ECE-MATH321 engradc (c)2011
b.
[a1
a2
... an ] = [q1 q 2
Where:
ak column vectors of A qk orthonormal column vectors R upper triangular matrix (also known as projection matrix)
ECE-MATH321 engradc (c)2011
QR-Decomposition Steps
Consider a 4x3 matrix A
[a1
Where:
a 2 a3 ] = [q1 q 2
r11 r12 q3 ] 0 r 22 0 0
ak 4x1 column vectors of A qk 4x1 orthonormal vectors R 3x3 upper triangular matrix
ECE-MATH321 engradc (c)2011
QR-Decomposition Steps
1 , r11 = q r12 = q 1 a 2 , 2 , r22 = q r13 = q 1 a 3 , r23 = q 2 a 3 , 3 , r33 = q 3 q q3 = 3 q
ECE-MATH321 engradc (c)2011
1 = a1 q 1 q q1 = 1 q
2 = I q 1q 1T a 2 q 2 q q2 = 2 q
T T q 3 = I q 1q 1 I q 2 q 2 a 3
)(
( )( = (I q q q q
)(
... q k 1q
T k 1
)a
QR-Decomposition
Case 1: Overdetermined System of Linear Equations The Least-Square Method
QTQ = I
Rx = Q b
T
QR-Decomposition
Case 2: Underdetermined System of Linear Equations The Least-Norm Method
AT = QR R y =b x = Qy
T
Examples
z
2x 5y + 13z = 24 3x + y 6z = 2 x y + z = 4, x + y z = 0 2x 3y + z = 4, 3x + 2y 3z = 1
ECE-MATH321 engradc (c)2011
b.
Rules: If rank(A) rank(B), the system has no exact solution and the system can be solved by estimation if m > n (more equations than unknowns) Typical solution to best estimate the unknowns is Method of Least Squares using QRDecomposition method
ECE-MATH321 engradc (c)2011
Since the solution is not exact, The residual error (vector), e is the distance vector of the actual vs. estimate and the loss function, (scalar) is the square norm of the error:
0 e = b Ax min = e e
T
ECE-MATH321 engradc (c)2011
QR-Decomposition Example
Obtain the best solution given an overdetermined system of equations
1 1 4 1 1 4 2 x1 0 x2 = 1 4 2 0 x3 1 1 0 0
ECE-MATH321 engradc (c)2011
If the system of linear equations is non-homogeneous i.e. b 0, and A is a full-rank matrix r = n Can be solved by:
z z z z z
a a a ... 11 12 1 n Inverse method Gaussian elimination a a a ... 21 22 2n A= LU-decomposition : : : : QR-Decomposition Cholesky Factorization an1 an 2 ... ann
ECE-MATH321 engradc (c)2011
Inverse Method
z
Since Ax = b
Premultiply both sides of the equation by A-1
A Ax = A b
-1 -1
Ix = A b
-1
x=A b
-1
ECE-MATH321 engradc (c)2011
Example
1.
Solve for the unknowns, x and y using Inverse method and compare it to the solution using Cramers rule
2x + 3y = 5 3x y = 4
Gaussian Elimination
z
This method involves replacing equations by obtaining its triangular matrix form
Example
z
3x + 4y + 2z w = 2 4x + 2y 3z + 2w = 4 5x + 3y 4z + 3w = 4 3x + y + z + 2w = 2
ECE-MATH321 engradc (c)2011
Example
z
3x + 4y + 2z w = 2 4x + 2y 3z + 2w = 4 5x + 3y 4z + 3w = 4 3x + y + z + 2w = 2
ECE-MATH321 engradc (c)2011
Cholesky-Decomposition
If A is real symmetric positive definite square matrix
A = LLT
l11 0 0 0 l11 l21 l31 l l22 0 0 0 l22 l32 21 A= l31 l32 l33 0 0 0 l33 l41 l42 l43 l44 0 0 0
A is positive definite square matrix iff its eigenvalues are real > 0
ECE-MATH321 engradc (c)2011
Cholesky-Decomposition
The elements of L can be computed easily using the Cholesky algorithm shown below
j 1
l j , j = a j , j l j ,k
k =1
j 1 1 li , j = ai , j li ,k l j ,k , l j, j k =1
for i > j
Cholesky-Decomposition
Example: Obtain the Cholesky Decomposition of A
Given a square matrix A, a non-trivial vector v and a scalar that satisfy the relationship
Av = v
z
Where
Matrix A has n eigenvalues Generally, eigenvalues are not always real and distinct If A is real symmetric therefore, its eigenvalues are real For each eigenvalue, there is a corresponding eigenvector.
ECE-MATH321 engradc (c)2011
z z
Av v = 0 Iv = v Av Iv = 0 (A I) v = 0
det (A I) = 0
z
This leads to a polynomial equation f() = 0 of degree equal to rank of A The roots of f() are the eigenvalues of A
ECE-MATH321 engradc (c)2011
Eigenvectors
z
(A iI) v(i) = 0
z
Eigenvectors
z
The eigenvector v(i), of the corresponding eigenvalue i is not unique, thus a normalization factor is needed to have a common eigenvector
(i ) 2
+ v2 + ... + vn
(i ) 2
(i ) 2
(i )
(i )
Eigen-Decomposition
A = QQ1
A = [v1 v2 11 0 0 22 ... v n ] : 0 .. 0 0 0 0 [v 1 .. : 0 nn .. v2 ... v n ]
1
Where:
Q eigenvector matrix arrange in column vectors diagonal matrix whose elements are the eigenvalues of A
ECE-MATH321 engradc (c)2011
Example
z
Solve for the eigenvalues and the eigenvector of the greatest positive eigenvalue. Decompose A = QQ1
5 3 2 A= 2 6 3 3 2 4
ECE-MATH321 engradc (c)2011