BITS Pilani

Hyderabad Campus

Lecture 13:
System of Linear Equations
Special Matrices and Gauss-Seidel Method
Special Matrices

• Certain matrices have particular structures

that can be exploited to develop efficient
solution schemes (e.g. banded, symmetric)

• A banded matrix is a square matrix that has

all elements equal to zero, with the exception
of a band centered on the main diagonal.

• Standard Gauss Elimination is inefficient in

solving banded equations because
unnecessary space and time would be
expended on the storage and manipulation of
zeros.

• There is no need to store or process the zeros

(off of the band)

CHE F242 Numerical Methods for Chemical Engineers, BITS, Pilani-Hyderabad Campus P-2
Solving Tridiagonal Systems
(Thomas Algorithm)
A tridiagonal system has a bandwidth of 3
LU COMPOSITION

for k = 2, n
 f1 g1   x1   r1 
e   x  r  ek = ek / fk-1
 2 f2 g2   2    2  fk = fk - ek gk-1
 e3 f3 g 3   x3  r3  end
    
 e4 f 4   x4  r4  Time Complexity?
O(n)
1 0 0 0  f1 g1  vs. O(n3)
e ' 1 0 0  f 2' g2 
A  L U   2 
0 e'3 1 0  f 3' g3 
  
0 0 e'4 1  f 4' 

CHE F242 Numerical Methods for Chemical Engineers, BITS, Pilani-Hyderabad Campus P-3
Solving Tridiagonal Systems
(Thomas Algorithm)
{d}
1 0 0 0  f1 g1   x1   r1 
e' 1 0 0  f 2' g2   x  r 
 2  2    2 
0 e'3 1 0  f 3' g3   x3   r3 
     
0 0 e'4 1  f 4'   x4  r4 

1 0 0 0  d1   r1   f1 g1   x1   d1 
e ' 0 d 2  r2     x  d 
 2 1 0
 f 2' g2  2    2 

0 e'3 1 0 d 3   r3   f 3' g3   x3   d 3 
         
0 0 e' 4 1 d 4  r4   f 4'   x 4  d 4 
Forward Substitution Back Substitution
d1 = r1 xn = dn /fn
for k = 2: n for k = n-1: -1:1
dk = rk - ek dk-1 xk = (dk - gk . xk+1 )/fk
end end
CHE F242 Numerical Methods for Chemical Engineers, BITS, Pilani-Hyderabad Campus P-4
Jacobi Iterative Method
Iterative methods provide an alternative to the elimination methods.

 a11 a12 a13  a11 0 0

Ax  b A  a 21 a 22 a 23  D   0 a 22 0 
a31 a32 a33   0 0 a33 

[ D  ( A  D )] x  b  Dx  b  ( A  D ) x  x  D 1[b  ( A  D ) x ]

 x1  1 / a11 0 0    b1   0 a12 a13   x1  

x    0        
 2  1 / a22 0  *  b2   a21 0 a23   x2  
 x3   0 0 1 / a33   b3  a31 a32 0   x3  
b1  a12 x2k 1  a13 x3k 1 b2  a21 x1k 1  a23 x3k 1 b3  a31 x1k 1  a32 x2k 1
x 
k
1 x 
k
2 x 
k
3
a11 a22 a33
Choose an initial guess (i.e. all zeros) and Iterate until the equality is satisfied.
No guarantee for convergence! Each iteration takes O(n2) time!

CHE F242 Numerical Methods for Chemical Engineers, BITS, Pilani-Hyderabad Campus P-5
Gauss-Seidel
• The Gauss-Seidel method is a commonly used iterative method.

• It is same as Jacobi technique except with one important difference:

A newly computed x value (say xk) is substituted in the subsequent
equations (equations k+1, k+2, …, n) in the same iteration.

Example: Consider the 3x3 system below:

b  a x old
 a x old • First, choose initial guesses for the x’s.
x1new  1 12 2 13 3
a11 • A simple way to obtain initial guesses is
to assume that they are all zero.
b  a x new
 a x old
x2new  2 21 1 23 3
• Compute new x1 using the previous
a22 iteration values.
b  a x new
 a x new
• New x1 is substituted in the equations to
x3new  3 31 1 32 2
a33 calculate x2 and x3
{ X }old  { X }new • The process is repeated for x2, x3, …
CHE F242 Numerical Methods for Chemical Engineers, BITS, Pilani-Hyderabad Campus P-6
Convergence Criterion for
Gauss-Seidel Method
• Iterations are repeated until the convergence criterion is satisfied:

xij  xij 1 For all i, where j and j-1 are

 a ,i  j
100%   s the current and previous iterations.
xi
• As any other iterative method, the Gauss-Seidel method has problems:
– It may not converge or it converges very slowly.

• If the coefficient matrix A is Diagonally Dominant Gauss-Seidel is

guaranteed to converge.
For each equation i :
Diagonally Dominant  n
aii   ai , j
j 1
j i

• Note that this is not a necessary condition, i.e. the system may still have a
chance to converge even if A is not diagonally dominant.

7
CHE F242 Numerical Methods for Chemical Engineers, BITS, Pilani-Hyderabad Campus P-7
Convergence Criterion for
Gauss-Seidel Method

CHE F242 Numerical Methods for Chemical Engineers, BITS, Pilani-Hyderabad Campus P-8