2/18/2010

A. Matrix: Definition
B. Special Types of Matrices
C. Equality of Matrices
Lecture on Matrices D. Elementary Operations on Matrices
1. Matrix Addition
2. Scalar Multiplication
3. Matrix Subtraction
4. Matrix Multiplication
5. Transposition of a Matrix

CE 26

E. Determinant of a Square Matrix

F. Adjoint of a Matrix
G. Inverse of a Matrix
Solution to System of Linear Equations
H.
(Direct Methods) Matrix
1. Inverse Method
2. Cramer’s Rule
3. LU Factorization
4. Gaussian Elimination
5. Gauss-Jordan Reduction Method

A matrix is a rectangular array of numbers or
functions arranged in rows and columns
usually designated by a capital letter and
enclosed by brackets, parentheses or double
bars. A matrix may be denoted by:

Unless stated, we assume that all our
matrices are composed of real numbers.

The horizontal groups of elements are called
the rows of the matrix. The ith row of A is

 a11 a12 ... a1n 

a [a a i2 ... a in ] (1 ≤ i ≤ m )
a 22 ... a2n 
i1

A =  21 
 : : : 
 
am1 am 2 ... amn 

1
 2/18/2010

The vertical groups of elements are called the
The size of a matrix is denoted by “m x n” (m
columns of the matrix. The jth column of A is by n) where m is the number of rows and n is
the number of columns.
 a1j 
a 
We refer to aij as the entry or the element in
 2j  (1 ≤ j ≤ n) the ith row and jth column of the matrix.
 : 
  We may often write a given matrix as
a 3 j 


A = [aij].

Columns

Example:

5 − 3 2 5 
A = 0 1 1 4
Rows
Special Types of Matrices
 
7 9 1 8  3 x 4

“9” is element a32 located at

3rd row and 2nd column of Size of Matrix A
matrix A.

C. Square Matrix – is a matrix in which the no.

A. Row Matrix or Row Vector – is a matrix of rows equals the no. of columns.
consisting of only one row.
2 0 3 
B = [b1 b2 . . . bj . . . bn]
A = 1 1 4
 
 c1  0 5 7  3 x 3
B. Column Matrix or Column c 
Vector – is a matrix consisting  .2  Order of a Square Matrix – is the number of
of only one column.  .
C= .  rows or columns of the matrix. Thus, we can
c just refer to a 3x3 matrix as a square matrix
 .i 
 ..  of order 3.
c m 

2
 2/18/2010

C. Square Matrix – is a matrix in which the no. C. Square Matrix – is a matrix in which the no.
of rows equals the no. of columns. of rows equals the no. of columns.

2 0 3  2 0 3 
A = 1 1 4 A = 1 1 4
   
0 5 7  3 x 3 0 5 7  3 x 3
Principal Diagonal or Main Diagonal of a The Trace of a Square Matrix – is the sum
Square Matrix – consists of the elements a11, of the elements on the main diagonal of the
a22, a33, … ann. matrix. In the above matrix, trace = 2 + 1
+ 7 = 10.

D. Upper Triangular Matrix – a square matrix E. Lower Triangular Matrix – a square matrix
all elements of which below the principal all elements of which above the principal
diagonal are zero (aij = 0 for i>j). diagonal are zero (aij = 0 for i<j).

u11 u12 u13  l11 0 0

U =  0 u 22 u23  L = l21 l22 0
   
 0 0 u33  l31 l32 l33 

F. Diagonal Matrix – is a square matrix that is G. Scalar Matrix – is a diagonal matrix whose
an upper triangular and lower triangular at diagonal elements are equal.
the same time. The only non-zero elements
are the elements on the principal diagonal.
(aij = 0 for i ≠ j) s 0 0
d11 0 0 S = 0 s 0
 
D =  0 d22 0  0 0 s 
 
 0 0 d33 

3
 2/18/2010

H. Identity Matrix – represented by In, it is a I. Null Matrix – represented by Ο is a matrix in

diagonal matrix where all the elements which all the elements are zero.
along the main diagonal are equal to 1 or
unity. 0 0 ... 0
0 0 ... 0
1 0 0
Ο= 
I3 =  0 1 0 : : :
   
 0 0 1  0 0 ... 0

J. Symmetric Matrix – a square matrix whose K. Skew Symmetric Matrix – a square matrix
element aij = aji. whose element aij = -aji

1 2 4   0 2 − 4
S = 2 2 − 5 T = − 2 0 − 5
   
 4 −5 3   4 5 0 

Two matrices A = (aij) and B = (bij) are equal

if and only if the following conditions are
satisfied:
a) They have equal number of rows.
Equality of Matrices b) They have equal number of columns.
c) All elements in A agree with the
elements in B. (aij=bij, for all i and j.)

4
 2/18/2010

Example: The matrices

x 1 c − 2  2 1 −1 
A = a y + 1 2  and B =  x + 1 b − 2 2  Elementary Operations on
   
3 b z   3 4 c + 1 Matrices
are equal if and only if x = 2, y = 1, z = 2, a =
3, b = 4 and c = 1.

A. MATRIX ADDITION Example: Given two 2x3 matrices A and B

 1 − 2 4 3 2 − 2
A=   B= 
If A = (aij) and B = (bij) are matrices of the − 3 2 1 4 − 1 1 
same size m x n, then the sum A + B is
another m x n matrix C = [cij] where cij = aij  1 − 2 4 3 2 − 2
+ bij for i = 1 to m and j = 1 to n. A +B=  + 4 − 1 1 
 − 3 2 1   
 1+ 3 − 2 + 2 4 + ( −2)
Matrix addition is accomplished by adding =
algebraically corresponding elements in A  − 3 + 4 2 + ( −1) 1 + 1 
and B.  4 0 2
= 
 1 1 2

B. SCALAR MULTIPLICATION Examples:

If A = (aij) is an m x n matrix and k is a real 2 0 − 1 3   − 6 0 3 − 9

number (or a scalar), then the scalar multiple − 3  =  
4 − 3 2 − 5 − 12 9 − 6 15 
of A by k is the m x n matrix C = [cij] where cij
= k*aij for all i and j.
 2 − 1 3  8 − 4 12
In other words, the matrix C is obtained by 4  5 2 1 =  20 8 4 
   
multiplying each element of the matrix by the − 2 4 7 − 8 16 28
scalar k.

5
 2/18/2010

C. MATRIX SUBTRACTION Example: 3 4 − 2   2 − 1 4

A=   B= 
5 7 − 4  − 3 8 3
If A and B are m x n matrices, the difference 3 4 − 2   2 − 1 4
between A and B denoted as A – B is obtained A −B=  + − 3 8 3 
5 7 − 4   
from the addition of A and (-1)B.
 3−2 4 − ( −1) − 2 − 4 
=
5 − ( −3) 7 − 8 − 4 − 3 
A – B = A + (-1)B
1 5 − 6 
= 
Matrix subtraction is accomplished by 8 − 1 − 7 
subtracting from the elements of the first
matrix the elements of the second matrix
Note: We can only add or subtract matrices
correspondingly. with the same number of rows and columns.

D. MATRIX MULTIPLICATION The formula tells us that in order to get the

element cij of the matrix C, get the elements
If A = (aij) is an m x n matrix and B = (bij) is of the ith row of A (the pre-multiplier) and
an n x p matrix, then the product of A and B, the elements on the jth column of B (the post
AB = C = [cij] is an m x p matrix where multiplier). Afterwards, obtain the sum of the
products of corresponding elements on the
n
two vectors.
c ij = ∑k =1
a ik b kj

for i = 1 to m and j = 1 to p

Note: Examples:
 1 2 2 − 4   1(2) + 2(3) 1( −4 ) + 2( −1) 
A=  B=  AB =  
The product is defined only if the number of 3 4 3 − 1 3(2) + 4(3 ) 3( −4 ) + 4( −1)
columns of the first factor A (pre-multiplier) is 8 −6
equal to the number of rows of the second AB =  
18 − 16
factor B (post-multiplier). If this is satisfied,
we say that the matrices are conformable in 2(1) + ( −4 )(3 ) 2(2) + ( −4)(4 )
the order AB. Note: Although AB and BA =  
BA are defined it is not  3(1) + ( −1)(3 ) 3( 2) + ( −1)(4 ) 
necessary that AB = BA. − 10 − 12
BA = 
The formula An will be defined as A∗ A∗ A…∗ A  0 2 

6
 2/18/2010

If A = [aij] is an m x n matrix, then the

 2 − 1 transpose of A, denoted by AT=[a’ij] is an n x
 6 2 − 3 1 3 
A= 5 3 B=  C=  m matrix defined by a’ij=aji.
  − 1 4 5  2 − 1
− 4 6 
The transpose of A is obtained by
a) A x B is a 3 x 3 matrix while B x A is a 2 interchanging the rows and the columns of A.
x 2 matrix.  1 2 − 2
Example: − 1 3  1 −1 2 5
b) A x C is a 3 x 2 matrix but C x A is not 4
If A =   then AT =  2 3 −1 3 
 2 −1 3   
defined.    − 2 4 3 − 2
c) B x C is not defined but C x B is defined (2  5 3 − 2
x 3). Note: The transpose of a symmetric matrix is
equal to itself.

A. MATRIX ADDITION B. SCALAR MULTIPLICATION

A+O=A Existence of Additive Identity 0xA=O

A + (-A) = O Existence of Additive Inverse

1x A=O
A+B=B+A Commutative Property
k l (A) = k (l A) = l (k A)
(A + B) + C = A + (B + C) Associative
Property k(A + B) = k A + k B

C. MATRIX MULTIPLICATION D. MATRIX TRANSPOSITION

A(BC) = (AB)C Associative Property
A(B + C) = AB + AC Left Distributive (AT)T = A
Property (A + B)T = AT + BT
(A + B)C = AC + BC Right Distributive (k A)T = k AT
Property (AB)T = BTAT
AI = IA = A Existence of
Multiplicative Identity In general
kl (AB) = (k A)(l B) = (l A)(k B) (A1 A2 A3…An-1 An)T= AnTAn-1T…A3TA2TA1T
Note: In general, Matrix Multiplication is not
commutative. That is, AB ≠ BA.

7

Determinant
By the Fundamental Principle of Counting we
can put any one of the n elements of S in the
first position, any one of the remaining (n-1)
elements in the second position, any one of
the remaining (n-2) elements in the third
position, and so on until the nth position.
Thus there are n(n-1)(n-2)…3*2*1 = n!
permutations of S. We refer to the set of all
permutations of S by Sn.
Examples: ODD and EVEN Permutation
S1 has only one permutation; that is 1, which is
even since there are no inversions.
In the permutation 35241, 3 precedes 2 and 1,
5 precedes 2, 4 and 1, 2 precedes 1 and 4
precedes 1. There is a total of 7 inversions,
thus the permutation is odd.
2/18/2010
Another very important number associated
with a square matrix A is the determinant of
A which we will now define. This unique
number associated to a matrix A is useful in
the solutions of linear equation.
Permutation:
Let S={1, 2, 3, … , n} be the set of integers
from 1 to n, arranged in increasing order. A
rearrangement a1a2a3…an of the elements in S
is called a permutation of S.
Examples:
If S = {1, 2, 3} then S3 = {123, 132, 213, 231,
312, 321}
If S = {1, 2, 3, 4} then there are 4! = 24
elements of S4.
Odd and Even Permutations
A permutation a1a2a3…an is said to have an
inversion if a larger number precedes a
smaller one. If the total number of inversion
in the permutation is even, then we say that
the permutation is even, otherwise it is odd.
S3 has 3! = 6 permutations: 123, 231and 312
are even while 132, 213, and 321 are odd.
S4 has 4! = 24 permutations: 1234, 1243,
1324, 1342, 1423, 1432, 2134, 2143, 2314,
2341, 2413, 2431, 3124, 3142, 3214, 3241,
3412, 3421, 4123, 4132, 4213, 4231, 4312,
4321.
For any Sn, where n>1 it contains n!/2 even
permutations and n!/2 odd permutations.
8
 2/18/2010

DEFINITION: DETERMINANT Examples:

Let A = [aij] be a square matrix of order n. The If A = [a11] is a 1 x 1 matrix then det(A) or
determinant of A denoted by det(A) or A is A= a11.
defined by
If A =  11
a a12 
det (A ) = A = ∑ (± )a1j1a 2 j2 ...anjn a
 21 a 
22 

where the summation is over all permutations
j1j2…jn of the set S = {1,2,…,n}. The sign is
taken as (+) if the permutation is even and (–)
if the permutation is odd.
then to get Awe write down the terms a1-a2-
b and replace the dashes with the all-possible
permutations of S = {1, 2}, namely 12 (even)
and 21 (odd). Thus A= a11a22 - a12a21.

a11 a12 a13 

If A = a 21 a 22 a 23  If A is a square matrix of order n, there will be
  n! terms in the determinant of A with n!/2
a 31 a 32 a 33 
positive terms and n!/2 negative terms.
then to compute the A we write down the six
terms a1-a2-a3-, a1-a2-a3-, a1-a2-a3-, a1-a2-a3-, Example:
a1-a2-a3-, a1-a2-a3-. Replace the dashes with 2 1 3
3 2 1 = (2)( 2)(2) + (3)(1)(3) + (0)(1)(1) − (0)(2)(3) − ( 2)(1)(1) − (3)(1)(2)
all the elements of S3, affix a (+) or (-) sign
0 1 2
and get the sum of the six terms.
=8+9+0−0−2−6
=9

METHODS IN GETTING THE DETERMINANT METHODS IN GETTING THE DETERMINANT

A. DIAGONAL METHOD A. DIAGONAL METHOD
This method is applicable to matrices with This method is applicable to matrices with
size less than or equal to 3. size less than or equal to 3.
2. 3 x 3 Matrix
1. 2 x 2 Matrix
a 11 a 12 a 13 a 11 a 12
a 11 a 12 a 21 a a 23 a 21 a 22
= a 11 a 22 − a 21 a 12
22

a 21 a 22 a 31
a 32
a 33
a 31
a 32

= a 11 a 22
a 33
+ a 12 a 23
a 31
+ a 13 a 21
a 32

−a 31 a 22 a 13 − a 32 a 23 a 11 − a 33 a 21 a 12

9

METHODS IN GETTING THE DETERMINANT
B. COFACTOR METHOD
Complementary Minor, det( Mij) or Mij
det(Mij) Mij

The complementary minor or simply minor of
an element aij of the matrix A is that
determinant of the sub-matrix Mij obtained
after eliminating the ith row and jth column
of A
METHODS IN GETTING THE DETERMINANT
B. COFACTOR METHOD

The determinant of a square matrix maybe
obtained using expansion about a row or
expansion about a column. The following
formulas maybe used in getting the
determinant:
n n
det( A ) = ai1A i1 + ai2 A i2 + ... + ain A in = ∑ aik A ik
k =1
(expansion about the ith row)
METHODS IN GETTING THE DETERMINANT
B. COFACTOR METHOD

Note: We may choose any row or any column
in getting the determinant of a given matrix.
2/18/2010
METHODS IN GETTING THE DETERMINANT
B. COFACTOR METHOD
Algebraic Complement or Cofactor, Aij

The algebraic complement or cofactor of an
element aij of the matrix A is that signed
minor obtained from the formula (-1)i+j Mij
METHODS IN GETTING THE DETERMINANT
B. COFACTOR METHOD

The determinant of a square matrix maybe
obtained using expansion about a row or
expansion about a column. The following
formulas maybe used in getting the
determinant:
det( A ) = a1j A 1j + a 2 j A 2 j + ... + anj A nj = ∑ akj A kj
k =1
(expansion about the jth column)
Example: Evaluate the given matrix
1 0 0 = a14 A 14 + a 24 A 24 + a 34 A 34 + a 44 A 44
3
2 1 − 1 = a 24 A 24 = a 24 ( −1)2+ 4 M24
4
3 2 4 0 1 0 3
0 3 − 1 0 = (− 1)(− 1) 3 2 4 = (− 1)(− 2 + 0 + 27 − 0 − 12 − 0)
6
0 3 −1
= − 13
It is best to expand about the fourth row because it has the most
numbers of zeros. The optimal course of action is to expand
about the row or column that has the largest number of zeros,
because in that case the cofactors Aij of those aij which are zero
need not be evaluated since the product of aijAij = (0)Aij = 0.
10
 2/18/2010

The Adjoint of a square matrix A=[aij] of order n is

that square matrix with the same order n denoted
by adj(A)=[Aji] where Aij is the cofactor of the
element aij of matrix A. The adjoint of a matrix is
Adjoint the transpose of the matrix of cofactors of the
elements of A.

Input: Square Matrix

Output: Square Matrix (with the same size as the
original matrix)

Notation: adj A, adj B…

Step 1: Get the cofactors of all the elements
Step 2: Set up the adjoint matrix by taking
in the original matrix. the transpose of the matrix of cofactors.

adj A = [A ij ]
T

Recall: the cofactor of an element aij can be
denoted as Aij and is defined by:
Example:
A ij = ( −1)i+ j Mij a b  d − b
If A =   then adj(A) = 
 c d − c a 

The inverse of a square matrix A = [aij] of

order n is that matrix B = [bij] of the same
order n such that AB = BA = In. We denote
the inverse matrix of A by A-1. Thus, we
Inverse define the inverse of A as that matrix A-1
such that
A(A-1) = (A-1)A = In.

11
 2/18/2010

Not all matrices has its inverse. However, if

Matrix Inversion applies only to square
the inverse of a matrix exists, it is unique.
matrices and can be produced using the
adjoint matrix and the determinant.

If the inverse of a matrix exists, we say that
the matrix is invertible or non-singular.
adjA
Otherwise, we say that the matrix is non- A −1 =
invertible or singular. A

Notation: A-1, B-1

From the above formula for inverse, it is
1 Find A-1 if

Example 1: 1 1 1 
highly suggested that the determinant be A = 2 5 − 2
 
computed first. If it so happened that the  1 7 − 7 
matrix is singular (i.e., the determinant is
Compute first the determinant using the
zero), then the inverse of the matrix is said to diagonal method
be non-existent.
A = − 35 − 2 + 14 − 5 + 14 + 14 = 0

Note that it is a waste of effort to still

Since matrix A is singular, as evidenced by its
produce the adjoint if the matrix is singular.
zero determinant, it can thus be concluded
Therefore, it is advised that you first check
that the Inverse of A (or A-1) does not exist.
for singularity.

 1 − 1 1

Example 2: Set up the inverse of the given matrix
Getting the cofactors of all the A =  1 2 3
 
 1 − 1 1 elements in the original matrix. 4 − 2 3
A =  1 2 3 A ij = ( −1)i+ j Mij
 
4 − 2 3 A 11 = ( −1)2
2 3
= 12 A 12 = ( −1)3
1 3
=9 A 13 = ( −1)4
1 2
= −10
−2 3 4 3 4 −2

Compute first the determinant using the diagonal
−1 1 1 1 1 −1
method A 21 = ( −1)3
=1 A 22 = ( −1) 4
= −1 A 23 = ( −1) 5
= −2
A = 6 − 12 − 2 − 8 + 6 + 3 = −7 −2 3 4 3 4 −2
−1 1 1 1 1 −1
A 31 = ( −1)4 = −5 A 32 = ( −1)5 = −2 A 33 = ( −1)6 =3

Since the determinant is not zero, then matrix A 2 3 1 3 1 2
is said to be non-singular. In this case, the
inverse exists and there is a need to set up the  12 9 − 10 
adjoint. Thus, the cofactor
matrix Aij is [A ] =  1
ij
−1 − 2 

− 5 − 2 3 

12
 2/18/2010

Cofactor Matrix from the previous slide
Consequently, the inverse of A is

 12 9 − 10   12 1 − 5
adj A 1 
[A ij ] =  1 − 1 − 2  A −1 = A −1 = ∗ 9 − 1 − 2
A −7  
− 5 − 2 3  − 10 − 2 3 

Therefore the adjoint of A is
To check if A-1 is correct, use A −1 ⋅ A = In

1 − 5
(or A ⋅ A −1 = In )
 12
[ ]
adj A = A ij
T adj A =  9

− 1 − 2
  12
1 
1 − 5  1 − 1 1 1 0 0
− 10 − 2 3  A −1 ⋅ A = ∗ 9 − 1 − 2 ⋅  1 2 3 = 0 1 0 = I3
−7      
− 10 − 2 3  4 − 2 3 0 0 1

In general, we can think of a system of linear

equations as a set of “m” equations that
Solution to System of contains “n” unknowns. There are several
forms by which a system of equations can be
Linear Equations written.
(Direct Methods) A. Equation Form
B. Matrix Form

We can have the equation form
Or we can transform that to the matrix form:
a11x 1 + a12 x 2 + a13 x 3 + L + a1n x n = b1
 a11 a12 a13 L a1n   x1   b1 
a 21x1 + a 22 x 2 + a 23 x 3 + L + a2n x n = b 2 a a 22 a 23 L a 2n   x 2   b 2 
a 31x 1 + a32 x 2 + a 33 x 3 + L + a3 n x n = b 3  21    
 a31 a32 a33 L a 3n   x 3  =  b 3 
M M M M M     
am1x1 + am2 x 2 + am3 x 3 + L + amn x n = bm  M M M M M  M   M 
am1 am 2 am 3 L amn   x n  bm 

where aij are constant coefficients of the
unknowns xj and bj are the constants

13
 2/18/2010

Referring to the matrix form, we can actually

Solution to System of
rewrite the system of equations as a compact Linear Equations
matrix operation:
(Direct Methods)
AX=B
Where: A. Inverse Method
A
Coefficient Matrix B. Cramer’s Rule
X
Column Matrix of
Unknowns/Variables C. LU Factorization
B
Column Matrix of Constants D. Gaussian Elimination
E. Gauss-Jordan Reduction

The Inverse Method maybe applied only to a

system of linear equations in which the number
of independent equations is equal to the
number of unknowns. If the number of
Inverse Method equations is equal to the number of unknowns,
the equation AX = B will have a matrix of
coefficients that is square.

If the matrix of coefficients A is non-singular,
the solution to the system is unique. On the
other hand, if A is singular, either the system
has a non-unique solution or no solution at all.

Derivation of the Solution for xi’s :
Example: Determine the x1 − x 2 + x 3 = 1

values of x1, x2 and x3 in
x 1 + 2x 2 + 3 x 3 = 6
Take note that the following system of
AX = B equations. 4 x1 − 2x 2 + 3x 3 = 5
the derivation
A −1 ∗ AX = A −1 * B assumes that A-1
Solution:
exists. If A-1 does
( A −1A ) * X = A −1 * B not exist, we can  1 − 1 1  x1   1
The above system of
I * X = A −1B not find the  1 2 3   x  = 6 
equations can be written   2   
solution to the
X = A −1B system AX = B.
in matrix form AX = B: 4 − 2 3 x 3  5 

A X B

14
 2/18/2010

 1 − 1 1
Performing the operation  x1  1

Consider matrix A
A =  1 2 3 X =  x 2  = 1


A-1B will yield the solution
  matrix:
matrix   
4 − 2 3  x 3  1

Getting A-1  12 1 − 5
Make it a habit to check if all the computed values

1 
− 1 − 2
of the unknowns satisfy all the given equations.
A −1 = ∗ 9
Note: Refer to previous −7   Checking is done by substituting the values x1 = 1,
example for A-1 − 10 − 2 3  x2 = 1 and x3 = 1 to the original equations.

To get x1, x2 and x3 , multiply A-1 to B: Equation 1
1(1) – 1(1) + 1(1) =? 1
Satisfied
Equation 2
1(1) + 2(1) + 3(1) =? 6
Satisfied
 x1   12 1 − 5   1
X =  x 2  = A −1 B =
1 
9 − 1 − 2  * 6  Equation 3
4(1) – 2(1) + 3(1) =? 5
Satisfied
  −7    
 x 3  − 10 − 2 3  5  Since all the equations were satisfied, then (x1, x2,
x3) = (1, 1, 1) is indeed the solution to the system.

Recall that A system of equation “n”

equations in “n” unknowns can be modeled
as a matrix operation AX = B.

Cramer’s Rule a11 a12 a13 L a1n   x1  b1 

a a 22 a 23 L a 2n   x 2  b2 
 21    
a31 a32 a33 L a3n   x 3  = b3 
    
 M M M M M  M   M 
an1 an 2 an 3 L ann   x n  bn 

Notice that regardless of the variable i that is

Let:
computed, the denominator of the above
A
coefficient matrix
formula is fixed at |A|. Therefore, it is
xi
ith variable suggested that the determinant of the
B
right hand side constants coefficient matrix be the first to be computed.
Ai
matrix resulting from
replacing the ith column of A
Example: Using Cramer's
by the column vector of Rule, determine the values 1 2 1   x1  1
constants B of x1, x2 and x3 that 3 3 1  x  = 2
simultaneously satisfy the   2   
1 2 − 1 x 3  3
Ai following system of
Solution is given by: xi = equations.
A

15
 2/18/2010

 1

Solution: Compute the determinant of A first. 2 

The right hand side matrix B is  
1 2 1  3 
A = 3 3 1 
 
To set up the matrix A1, all you just have to do is
 1 2 − 1
to replace the first column of A by B. Doing what
A = −3 + 2 + 6 − 3 − 2 + 6 = 6 has just been described will result in:

Now, let us compute for the value of x1 by
1 2 1 
using the formula A1 0
A 1 = 2 3 1  x1 = = =0
  A 6
A1 3 2 − 1
x1 =
A A 1 = −3 + 6 + 4 − 9 − 2 + 4 = 0

Applying the same process to solve x2 and x3:

1 1 1 1 2 1
A2 1 A3 1
x2 = = * 3 2 1 =1 x3 = = * 3 3 2 = −1
A 6 A 6
1 3 −1 1 2 3 LU Factorization

or

Direct L-
L-U Factorization:
From matrix multiplication, we know that:
In theory any square matrix A may be factored into
a product of lower and upper triangular matrices. a11 a12 a13 a14  l11 0 0 0  1 u12 u13 u14 
a a 22 a 23 a 24  l21 l22 0 0  0 1 u23 u24 
 21 = * 
Let us take the case of a 4th order matrix: a31 a 32 a33 a34  l31 l32 l33 0  0 0 1 u34 
     
a 41 a 42 a 43 a 44  l41 l42 l43 l44  0 0 0 1
a11 a12 a13 a14  l11 0 0 0  1 u12 u13 u14 
a a 22 a 23 a 24  l21 l22 0 0  0 1 u23 u24  1. Get First Column Elements of Matrix L (1 unknown per equation)
 21 = * 
a31 a 32 a33 a34  l31 l32 l33 0  0 0 1 u34  a11 = l11(1) + 0(0) + 0(0) + 0(0) l11 = a11
     
a 41 a 42 a 43 a 44  l41 l42 l43 l44  0 0 0 1
l21 = a21
a 21 = l21(1) + l22 (0) + 0(0) + 0(0)
Notice that the diagonal elements of the upper a31 = l31(1) + l32 (0) + l33 (0) + 0(0) l31 = a 31
triangular matrix have been set to values of 1 for
reason of simplicity. (L-U Factorization is not a 41 = l41(1) + l42 (0) + l43 (0) + l44 (0) l41 = a 41
unique.)

16
 2/18/2010

or or

From matrix multiplication, we know that:
From matrix multiplication, we know that:
a11 a12 a13 a14  l11 0 0 0  1 u12 u13 u14  a11 a12 a13 a14  l11 0 0 0  1 u12 u13 u14 
a a 22 a 23 a 24  l21 l22 0 0  0 1 u23 u24  a a 22 a 23 a 24  l21 l22 0 0  0 1 u23 u24 
 21 = *   21 = * 
a31 a 32 a33 a34  l31 l32 l33 0  0 0 1 u34  a31 a 32 a33 a34  l31 l32 l33 0  0 0 1 u34 
           
a 41 a 42 a 43 a 44  l41 l42 l43 l44  0 0 0 1 a 41 a 42 a 43 a 44  l41 l42 l43 l44  0 0 0 1

2. Get First Row Elements of Matrix U (1 unknown per equation) 3. Get Second Column Elements of Matrix L (1 unknown per equation)

a12 = l11(u12 ) + 0(1) + 0(0) + 0(0) u12 = a12 / l11 a22 = l21(u12 ) + l22 (1) + 0(0) + 0(0) l22 = a 22 − l21(u12 )

a13 = l11 (u13 ) + 0(u23 ) + 0(1) + 0(0) u13 = a13 / l11 a32 = l31 (u12 ) + l32 (1) + l33 (0) + 0(0) l32 = a32 − l31(u12 )

a14 = l11(u14 ) + 0(u24 ) + 0(u34 ) + 0(1) u14 = a14 / l11 a 42 = l41(u12 ) + l42 (1) + l43 (0) + l44 (0) l42 = a42 − l41(u12 )

From matrix multiplication, we know that:
From matrix multiplication, we know that:
a11 a12 a13 a14  l11 0 0 0  1 u12 u13 u14  a11 a12 a13 a14  l11 0 0 0  1 u12 u13 u14 
a a 22 a 23 a 24  l21 l22 0 0  0 1 u23 u24  a a 22 a 23 a 24  l21 l22 0 0  0 1 u23 u24 
 21 = *   21 = * 
a31 a 32 a33 a34  l31 l32 l33 0  0 0 1 u34  a31 a 32 a33 a34  l31 l32 l33 0  0 0 1 u34 
           
a 41 a 42 a 43 a 44  l41 l42 l43 l44  0 0 0 1 a 41 a 42 a 43 a 44  l41 l42 l43 l44  0 0 0 1

4. Get Second Row Elements of Matrix U (1 unknown per equation) 5. Get Third Column Elements of Matrix L (1 unknown per equation)

a 23 = l21 (u13 ) + l22 (u 23 ) + 0(1) + 0(0 ) a 33 = l31(u13 ) + l32 (u23 ) + l33 (1) + 0(0)
u23 = [a 23 − l21(u13 )] / l22 l33 = a 33 − l31(u13 ) − l32 (u23 )

a 24 = l21(u14 ) + l22 (u24 ) + 0(u23 ) + 0(1) a 43 = l41 (u13 ) + l42 (u 23 ) + l 43 (1) + l 44 (0 )

u24 = [a 24 − l21(u14 )] / l22 l43 = a 43 − l41(u13 ) − l42 (u23 )

From matrix multiplication, we know that:
From matrix multiplication, we know that:
a11 a12 a13 a14  l11 0 0 0  1 u12 u13 u14  a11 a12 a13 a14  l11 0 0 0  1 u12 u13 u14 
a a 22 a 23 a 24  l21 l22 0 0  0 1 u23 u24  a a 22 a 23 a 24  l21 l22 0 0  0 1 u23 u24 
 21 = *   21 = * 
a31 a 32 a33 a34  l31 l32 l33 0  0 0 1 u34  a31 a 32 a33 a34  l31 l32 l33 0  0 0 1 u34 
           
a 41 a 42 a 43 a 44  l41 l42 l43 l44  0 0 0 1 a 41 a 42 a 43 a 44  l41 l42 l43 l44  0 0 0 1

6. Get Third Row Element of Matrix U (1 unknown per equation) 7. Get Fourth Column Element of Matrix L (1 unknown per equation)

a34 = l31(u14 ) + l32 (u24 ) + l33 (u34 ) + 0(1) a 44 = l41(u14 ) + l42 (u24 ) + l43 (u34 ) + l44 (1)
u34 = [a34 − l31(u14 ) − l32 (u24 )] / l33 l44 = a 44 − l41 (u14 ) − l42 (u24 ) − l43 (u34 )

17
 2/18/2010

Recall: A system of equations can be written Two-

Two-Stage Solution:
Solution:
as a compact matrix operation AX = B
Stage 1: Solve for Y in the equation LY = B

If we factor out the coefficient matrix A as using forward substitution.
L*U and substitute to AX = B, we can
Stage 2: Solve for X in the equation UX = Y
generate the equation L(UX)=B. using back substitution.

Momentarily define UX = Y which suggests
LY = B. From this transformation, we have
Example: Determine the solution X in
actually decomposed AX = B to two systems  − 4 2 1   x 1  3 
of equations.  − 3 5 − 1  x  = 4
  2   
 1 2 1   x 3  8 

Knowing that A = LU, determining L and U as
Stage 1: Forward substitution using LY = B

below y1 = −
3

− 4 2 1  − 4 0 0   y 1  3 
4
 
A = − 3 5 − 1 0   y 2  =  4
1
− 3 7
    y2 =
   2
5   y 3  8 
2
 1 2 1   1
5
2 2 
y3 = 3
− 4 0 0 1 − 1 − 1 
   2 4
Note that the computed values of yi's here
L = − 3 7 0 U = 0 1 −1 
 2 2 are not yet the solution since the original
1 5 5  0 0 1 
 2 2   system of equations is in terms of xi's.

Stage 2: Back substitution using UX = Y
If A is an m x n matrix and B is a p x n

matrix, then the augmented matrix of A and
x1 = 1
1 − 1 − 1   x  − 3  B denoted by [A : B] is the matrix formed by
 2 4 1  4 the elements of A and B separated by pipes.
0 1 − 1  x 2  =  1  x2 = 2
2    2
0 0 1   x 3   3   1 −2 5  − 2 1 
   
Example:
x3 = 3 A=7 1 − 6 B= 0 2 
   
− 3 − 3 8   7 − 4 

This time (x1, x2, x3) = (1, 2, 3) is the solution  1 −2 5 | −2 1 

to the original system of equations. A :B =  7 1 −6 | 0 2
 
− 3 − 3 8 | 7 − 4

18
 2/18/2010

The augmented matrix associated to a system
An m x n matrix A is said to be in row echelon

form if it satisfies the following properties:
of linear equation AX=B is the matrix [A : B].
For example, we can now rewrite the system 1. All rows whose elements are all zeros, if exist,
of equation: are at the bottom of the matrix.

 2 − 1 3 x  1  2 − 1 3 | 1 2. If at least one element on a row is not equal to

 1 zero, the first non-zero element is 1, and this
2 6 y  = 1  1 2 6 | 1 is called the leading entry of the row.
      
− 2 4 1 z  1 − 2 4 1 | 1
3. If two successive rows of the matrix have
leading entries, the leading entry of the row
below the other row must appear to the right of
the leading entry of the other row.

Examples:

An m x n matrix A is said to be in reduced row
echelon form if added to the first three The following matrices are not in row echelon form.
properties it satisfies a fourth property: (Why not?)

1 2 0 −1 3 2 1 0 0 0
0 1 2 −2 4 
4. If a column contains a leading entry of some 1 0 1 − 3 5 0 1 2 0 0
    1 6 − 7
row, then all the other entries must be zero. A = 0 0 0 1 9 2 B = 0 0 − 1 0 C= 
    0 0 1 8
0 0 0 0 0 0 0 0 0 1  
0 1 0 0
0 0 0 0 0 1 0 0 0 0

Did not satisfy Did not satisfy Did not satisfy

property 1 property 2 property 3

Examples:
Examples:
The following matrices are in row echelon form but not The following matrices are in reduced row echelon
in reduce row echelon form. form. (Hence, in row echelon form.)
Must all be zeroes to satisfy the 4th property.
1 0 − 3
1 0 0 0 0
1 0 8 0 
0  0 1 0 0 2  1 0 
1 2 3 4 0  
0 1 0 − 5 6 2  1 0 − 3 G=
1 0 0
 H = 0 0 1 0 − 3 J = 0 0 0 
1 2 3    
D=  E = 0 1 0 0 − 8  F = 0 0 1 0  0 0 1 0  
0 0 1 2     0 0 0 1 0  0 0 0 
0 0 1 0 0    0 0 0 1
  0 0 0 0  0 0 0 
0 0 0 1
0 0 0 0 

All properties are satisfied except property 4.

19
 2/18/2010

An elementary row (column) operation on a
Example: Let 1 0 −2 0 

matrix A is any one of the following A = − 3 1 6 − 4
 
operations:  2 − 8 2 2 
Type I: Interchanging rows 1 and 3 of A (R1↔R3)
◦ Type I. Interchange any two rows (columns). obtain 2 8 2 2 
◦ Type II. Multiply a row (column) by a non-zero
B = − 3 1 6 − 4 
constant k.  
 1 0 − 2 0 
◦ Type III. Add to elements of a row k times of the
elements of another row the Type II: Multiplying row 3 by ½ (R3’→ ½R3), we obtain
correspondingly.
 1 0 −2 0 
C = − 3 1 6 − 4
 
 1 − 4 1 1 

As a applied to the augmented matrix [A:B] as

Example: Let 1 0 −2 0  a system of equation, the three elementary row
A = − 3 1 6 − 4 operation will correspond to the following:
 
 2 − 8 2 2 
TYPE I → rearranging the order of the
Type III: Adding 3 times the elements in row 1 to the equations
elements in row 2 (R2’→R2 + 3R1), we obtain TYPE II → multiplying both side of the
equation by a constant
1 0 − 2 0  TYPE III → working with two equations
D = 0 1 0 − 4
 
From this observation, we could see that as
2 − 8 2 2 
applied to a operations does not alter the
solution of the system.

In general a system of “m” equations in “n”

unknowns may be written in matrix form:
 a11 a12 a13 L a1n   x 1   b1 
a a 22 a23 L a2n   x 2   b 2 
 21    
 a 31 a 32 L a3n   x 3  =  b 3 

 M M
a33
M M
   
M  M   M  Gaussian Elimination Method
am1 am2 am3 L amn   x n  bm 

This system may now be represented by the

augmented notation:
 a11 a12 a13 L a1n | b1 
a a 22 a 23 L a 2n | b 2 
 21 
 a 31 a32 a 33 L a3 n | b 3 
 
 M M M M M M M 
am1 am 2 a m3 L amn | bm 

20
 2/18/2010

The objective of the Gaussian Elimination
Method is to transform the augmented matrix
[A : B] to the matrix [A* : B*] in row echelon
form by applying a series of elementary row
transformations. Getting the solution of the
system [A* : B*] using back substitution will
also give the solution to the original system
[A : B].
1. Find the leftmost non-zero column.
2. If the 1st row has a zero in the column of step 1,
interchange it with one that has a non-zero entry
in the same column.
3. Obtain zeros below the leading entry by adding
suitable multiples of the top row and to the rows
below that.
4. Cover the top row and repeat the same process
starting with step 1 applied to the leftover
submatrix. Repeat this process with the rest of
the rows.
5. For each row obtain leading entry 1 by dividing
each row by their corresponding leading entry

Example: The linear system
Transform in row echelon form

x + 2y + 3z = 9 1 2 3 | 9
2x −y +z = 8 [A : B] = 2 −1 1 | 8

3x −z = 3 3 0 − 1 | 3

Has the augmented matrix associated to the system
Element in the 1st row 1st column is non-zero and already
is equal to 1 (leading entry).
 1 2 3 | 9
Transform into zeroes all elements below the leading
entry.
[A : B] = 2 − 1 1 | 8 1 2 3 9 |
3 0 − 1 | 3 R 2 ' → R 2 − 2 ⋅ R1
R 3 ' → R 3 − 3 ⋅ R1
[A * : B *] = 0 −5 | − 10 
−5

0 − 6 − 10 | − 24

Consider now the transformation of column 2
Consider now the transformation of column 3
elements 1 2 3 | 9  elements
1 2 3 | 9 
[A * : B *] = 0 − 5 − 5 | − 10 
 [A * : B *] = 0 1 1 | 2 

0 − 6 − 10 | − 24 0 0 − 4 | − 12

Element at 2nd row 2nd column is non-zero and will be

Element at 3rd row 3rd column is non-zero and will be
transformed to 1 (leading entry). Afterwards, transform transformed to 1 (leading entry).
elements below to zero.
1 2 3 | 9 
 1  1 2 3 | 9
R2 ' →  −  R2 [A * : B *] = 0 1 1 | 2 
  1
R3 ' →  −  R 3 [A * : B *] = 0 1 1 | 2
 5 
0 − 6 − 10 | − 24  4
0 0 1 | 3
1 2 3 | 9 
[A * : B *] = 0 1 1 | 2 
Augmented Matrix in Row Echelon Form!
R3 ' → R3 + 6 R 2

0 0 − 4 | − 12

21
 2/18/2010

Using back substitution

 1 2 3 | 9
[A * : B *] = 0 1 1 | 2

0 0 1 | 3 Gauss-Jordan Reduction
z=3 Method
y+z=2 ⇒ y = 2 − 3 = −1
x + 2y + 3 z = 9 ⇒ x = 9 − 2( −1) − 3(3 ) = 2

Thus, we have the solution (x, y, z) = (2, -1, 3).

On the other hand a second method called the
Gauss-
Gauss-Jordan Reduction Method gets rid of the
back substitution phase. The objective of the
Gauss-Jordan Reduction Method is to
transform the augmented matrix [A : B] to the
matrix [A* : B*] in reduced row echelon form by
applying a series of elementary row
transformations. Doing this will automatically
give the solution of the system [A* : B*] which
also provides the solution to the original
system [A : B].
S Search – search the ith column of the augmented
matrix from the ith row to the nth row for the
maximum pivot, i.e. element with the largest
absolute value.
I Interchange – assuming the maximum pivot
occurs in the jth row, interchange the ith row and
the jth row so that the maximum pivot will now
occur in the diagonal position.
N Normalize – normalize the new ith row by
dividing it by the maximum pivot on the diagonal
position.
E Eliminate – eliminate the ith column from the first
up to the nth equation, except in the ith equation
itself using the transformations.

Example: The linear system
Which can be transformed as a matrix in

x + 2y + 3z = 9 reduced row echelon form
2x −y +z = 8 1 0 0 | 2 
3x −z = 3 [A *
: B * ] = 0 1 0 | − 1
 
0 0 1 | 3 

Has the augmented matrix associated to the system

Thus, we have the solution (x, y, z) = (2, -1, 3).
1 2 3 | 9
[A : B] = 2 − 1 1 | 8

3 0 − 1 | 3

22
 2/18/2010

THE END

23