EKC 245 Mathematical Methods for Chemical Engineering

SYAMSUL RIZAL ABD SHUKOR PhD AMIChemE

chsyamrizal@eng.usm.my, ext 6413

9-2

C OURSE C ONTENT

9-3

L INEAR A LGEBRAIC E QUATION

Motivation

PART 3

In Part Two, we determined the value x that satisfied a single equation f(x)=0. Now, we deal with finding x1, x2, ,xn that simultaneously satisfy a set of equations:

9-4

L INEAR A LGEBRAIC E QUATION

Motivation

In Part Three, we deal with linear algebraic equations that are of the general form

PART 3
Where the as are constant coefficients, What do you have in mind when bs are constants and n is the number of you see these sets of equations? equations

9-5

L INEAR A LGEBRAIC E QUATION

General Study Objectives

Be able to solve problems involving linear algebraic equation Appreciate the usage of linear algebraic equations in any field of engineering Mastering several techniques and their reliability Nave Gauss elimination Gauss-Jordan elimination LU decomposition Gauss Siedel Be able to use a program to successfully solve systems of linear algebraic equations

PART 3

9-6

L INEAR A LGEBRAIC E QUATION : A N O VERVIEW

Study Objectives: Able to

Overview Pt. 3

understand matrix notation identify the following types of matrices: identity, diagonal, symmetric, triangular, and tridiagonal perform matrix multiplication assess matrix multiplication when it is feasible represent a system of linear equations in matrix form solve linear algebraic equations with left division and matrix inversion in MATLAB

9-7

L INEAR A LGEBRAIC E QUATION : A N O VERVIEW

Overview Pt. 3

A matrix consists of a rectangular array of elements represented by a single symbol (example: [A]). An individual entry of a matrix is an element (example: amn where m=2, n=3)
1xn matrices are row vectors mx1 matrices are column vectors

9-8

L INEAR A LGEBRAIC E QUATION : S PECIAL M ATRICES

Overview Pt. 3

Matrices where m=n are called square matrices. There are a number of special forms of square matrices:

9-9

L INEAR A LGEBRAIC E QUATION : M ATRIX O PERATION

Overview Pt. 3

Two matrices are considered equal if and only if every element in the first matrix is equal to every corresponding element in the second. This means the two matrices must be the same size. Matrix addition and subtraction are performed by adding or subtracting the corresponding elements. This requires that the two matrices be the same size. Scalar matrix multiplication is performed by multiplying each element by the same scalar:

9-10

L INEAR A LGEBRAIC E QUATION : M ATRIX M ULTIPLICATION

Overview Pt. 3

The elements in the matrix [C] that results from multiplying matrices [A] and [B] are calculated using:

9-11

L INEAR A LGEBRAIC E QUATION : M ATRIX I NVERSE AND T RANSPOSE

Overview Pt. 3

The inverse of a square, nonsingular matrix [A] is that matrix which, when multiplied by [A], yields the identity matrix. [A][A]-1 = [A]-1[A] = [I] The transpose of a matrix involves transforming its rows into columns and its columns into rows. (aij)T = aji

9-12

L INEAR A LGEBRAIC E QUATION : R EPRESENTING L INEAR A LGEBRA

Overview Pt. 3

Matrices provide a concise notation for representing and solving simultaneous linear equations:

9-13

L INEAR A LGEBRAIC E QUATION : S OLVING WITH MATLAB

Overview Pt. 3

RECALL what you have learnt in EKC 244

9-14

G AUSS E LIMINATION
Study Objectives: Able to solve small sets of linear equations with the graphical method and Cramers rule implement forward elimination and back substitution as in Gauss elimination count flops to evaluate the efficiency of an algorithm. understand the concepts of singularity and illcondition. understand how partial pivoting is implemented and how it differs from complete pivoting. recognize how the banded structure of a tridiagonal system can be exploited to obtain extremely efficient solutions.

CHAPTER 9

9-15

S OLVING S MALL N UMBER OF E QUATIONS

Small numbers (n<=3) of equations: Graphical method, Cramers rule, elimination of unknowns. GRAPHICAL METHOD Graphing small sets of simultaneous equations and determining the location of the intercept provides a solution. The point where the three planes (or two lines) intersect would represent the solution.

GAUSS ELIMINATION

9-16

S OLVING S MALL N UMBER OF E QUATIONS : G RAPHICAL M ETHOD

GAUSS ELIMINATION

Graphing the equations can also show systems where: a) No solution exists b) Infinite solutions exist c) System is ill-conditioned

Extremely sensitive to round off error. The point of intersection is difficult to detect visually

(a)

(b)

(c)

9-17

S OLVING S MALL N UMBER OF E QUATIONS : C RAMER S R ULE

DETERMINANTS

GAUSS ELIMINATION

The determinant D=|A| of a matrix is formed from the coefficients of [A]. Determinants for small matrices are:

Determinants for matrices larger than 3 x 3 can be very complicated.

9-18

S OLVING S MALL N UMBER OF E QUATIONS : C RAMER S R ULE

GAUSS ELIMINATION

The rule states that each unknown in a system of linear algebraic equations may be expressed as a fraction of two determinants with denominator D and with the numerator obtained from D by replacing the column of coefficients of the unknown in question by the constants b1, b2, , bn :

9-19

C RAMER S R ULE : E XAMPLE IN C LASS

Question 0.3x1 + 0.52 x 2 + x 3 = 0.01 Find x2 in the following system of equations: 0.5x + x 2 + 1.9x 3 = 0.67 1 0.1x1 + 0.3x 2 + 0.5x 3 = 0.44 Solution Find the determinant, D :

GAUSS ELIMINATION

9-20

C RAMER S R ULE : E XAMPLE IN C LASS

Question 0.3x1 + 0.52 x 2 + x 3 = 0.01 Find x2 in the following system of equations: 0.5x + x 2 + 1.9x 3 = 0.67 1 0.1x1 + 0.3x 2 + 0.5x 3 = 0.44 Solution Find the determinant, D :

GAUSS ELIMINATION

Find determinant D2 by replacing Ds second column with b

Divide

9-21

S OLVING S MALL N UMBER OF E QUATIONS : E LIMINATION OF U NKNOWNS

The simplest approach by eliminating one of the unknowns.
a11x1 + a12 x 2 = b1 a21x1 + a22 x 2 = b2
Multiply with a21 Multiply with a11

GAUSS ELIMINATION

a21a11x1 + a22 a11x 2 = b2 a11 a11a21x1 + a12 a21x 2 = b1a21

a22 a11x 2 a12 a21x 2 = b2 a11 b1a21

subtract

THEN, solve for x2

9-22

N AVE G AUSS E LIMINATION

GAUSS ELIMINATION

For larger systems, Cramers Rule can become unwieldy. Instead, a sequential process of removing unknowns from equations using forward elimination followed by backward substitution may be used - this is Gauss elimination. Nave Gauss elimination simply means the process does not check for potential problems resulting from division by zero no pivoting needed

9-23

N AVE G AUSS E LIMINATION

A systematic technique use to solve a large sets of linear algebraic equations simultaneously with two steps:
Forward elimination
the equations were manipulated to eliminate all the

GAUSS ELIMINATION

Back Substitution

elements below the main diagonal of matrix A

the elimination step result in one equation with one

unknown the equation could be solved directly and the result back-substituted into one of the original equations to solve the remaining unknown.

9-24

N AVE G AUSS E LIMINATION : P ROCEDURE

Represent the linear algebraic equations in an augmented matrix form a x + a x + a x + ... + a
11 1 12 2 13 3

GAUSS ELIMINATION

x n = b1 a21x1 + a22 x 2 + a23 x 3 + ... + a2 n x n = b2 a31x1 + a32 x 2 + a33 x 3 + ... + a3n x n = b3

1n

a11 a 21 a31 an1

a12 a13 ... a1n x1 b1 x b a22 a23 ... a2 n 2 2 a32 a33 ... a3n x 3 = b3 an 2 an 3 ... ann x n bn

an1x1 + an 2 x 2 + an 3 x 3 + ... + ann x n = bn

Label as row or equation (1.n)

a11 a21 a31 an1 a12 a13 ... a1n a22 a23 ... a2 n a32 a33 ... a3n an 2 an 3 ... ann b1 b1 b1 b1 - (1) - (2) - (3) - (n)

9-25

N AVE G AUSS E LIMINATION : P ROCEDURE

Example of Nave Gauss elimination with 3 unknowns in 3 eqns Forward Elimination Starting with the first row, add or subtract multiples of that row to eliminate the first coefficient from the second row and beyond. Continue this process with the second row to remove the second coefficient from the third row and beyond. Stop when an upper triangular matrix remains

GAUSS ELIMINATION

9-26

N AVE G AUSS E LIMINATION : P ROCEDURE

Forward Elimination To eliminate the first unknown, x1, from the second through the nth row/eqn.

GAUSS ELIMINATION

row/eqn (1) is called the pivot equation, and a11 is called pivot element
a13 a23 a33 b1 - (1) b2 - (2) b3 - (3)

a11 a 21 a31

a12 a22 a32

a11 a12 a ' 22 a '32

a13 a ' 23 a '33

b1 - (1) b' 2 - (2' ) b'3 - (3' )

eqn (1) multiply with a21/a11 eqn (1a) eqn (2) minus eqn (1a) eqn (2) eqn (1) multiply with a31/a11 eqn (1b) eqn (3) minus eqn (1b) eqn (3) The prime indicates that the elements have been modified

9-27

N AVE G AUSS E LIMINATION : P ROCEDURE

Forward Elimination To eliminate the second unknown, x2, from the third through the nth row/eqn.

GAUSS ELIMINATION

row/eqn (2) is called the pivot equation

a13 a ' 23 a '33 b1 - (1) b' 2 - (2' ) b'3 - (3' )

a11 a12 a ' 22 a '32

eqn (2) multiply with a32/a22 eqn (2a) eqn (3) minus eqn (2a) eqn (3)
a11 a12 a ' 22 a13 a ' 23 a ' '33 b1 - (1) b' 2 - (2' ) b' '3 - (3' ' )

The double prime indicates that the elements have been modified twice

9-28

N AVE G AUSS E LIMINATION : P ROCEDURE

Back Substitution Starting with the last row, solve for the unknown, then substitute that value into the next highest row. Because of the upper-triangular nature of the matrix, each row will contain only one more unknown. Then, from eqn (3) [a' '33
b' ' 33 ] a' ' 33 x 3 = c ' ' 3 x 3 = c ' ' 3 / a ' ' 33

GAUSS ELIMINATION

The result x3 can be back-substituted into eqn (2) and (1) to solve for x2 and x1

9-29

N AVE G AUSS E LIMINATION : P ROCEDURE

Ultimately,

GAUSS ELIMINATION

There is only one variable in the nth row

9-30

N AVE G AUSS E LIMINATION : E XAMPLE IN C LASS

Use the Nave Gauss elimination to find the solution for the given set of equations Solution Forward Elimination

GAUSS ELIMINATION

9-31

N AVE G AUSS E LIMINATION : E XAMPLE IN C LASS

Solution Backward substitution

GAUSS ELIMINATION

Verification

9-32

N AVE G AUSS E LIMINATION : G AUSS P ROGRAM E FFICIENCY

GAUSS ELIMINATION

The execution of Gauss elimination depends on the amount of floating-point operations (or flops). The flop count for an n x n system is:

Conclusion As the system gets larger, the computation time increases greatly. Most of the effort is incurred in the elimination step.

9-33

N AVE G AUSS E LIMINATION : PIVOTING

Problems arise with nave Gauss elimination if a coefficient along the diagonal is 0 (problem: division by 0) or close to 0 (problem: round-off error) One way to combat these issues is to determine the coefficient with the largest absolute value in the column below the pivot element. The rows can then be switched so that the largest element is the pivot element. This is called partial pivoting. If the rows to the right of the pivot element are also checked and columns switched, this is called complete pivoting.

GAUSS ELIMINATION

9-34

N AVE G AUSS E LIMINATION : PIVOTING - E XAMPLE

Use the Gaussian elimination to solve equations below:

GAUSS ELIMINATION

If no partial pivoting

9-35

N AVE G AUSS E LIMINATION : PIVOTING - E XAMPLE

GAUSS ELIMINATION

The result very sensitive to the number of significant figures carried in the computation

9-36

N AVE G AUSS E LIMINATION : PIVOTING - E XAMPLE

GAUSS ELIMINATION

WITH partial pivoting

9-37

N AVE G AUSS E LIMINATION : T UTORIAL DO question no 9.1 (page 271)

Additional: c) Solve the given set of equations using Nave Gauss elimination method.

GAUSS ELIMINATION

9-38

QUIZ
Q1. Derive the truncating error as given below for the Forward Elimination, Backward Elimination.

GAUSS ELIMINATION

Q2. Proof that the total truncating error is given by

Q3. Write down the complete Taylor series expansion and the Remainder, Rn term.

9-39

G AUSS -J ORDAN M ETHOD

GAUSS ELIMINATION

Another variation of Gauss Elimination method Major difference: When unknowns are eliminated, they are eliminated from all other equations All rows are normalized by dividing them by their pivot elements results in identity matrix No substitution is employed to obtain the solution

9-40

G AUSS -J ORDAN E LIMINATION : P ROCEDURE

Elimination of 1st unknown Change the value of a11 to 1 and eliminate the other elements in the first column
a11 a 21 a31 a12 a22 a32 a13 a23 a33 b1 - (1) b2 - (2) b3 - (3)

GAUSS ELIMINATION

1 a '12 0 a ' 22 0 a ' 32

a '13 a ' 23 a ' 33

b'1 - (1' ) b' 2 - (2' ) b' 3 - (3' )

eqn (1) multiply with 1/a11 eqn (1) eqn (1) multiply with a21 eqn (1a) eqn (2) minus eqn (1a) eqn (2) eqn (1) multiply with a31 eqn (1b) eqn (3) minus eqn (1b) eqn (3) The prime indicates that the elements have been modified

9-41

G AUSS -J ORDAN E LIMINATION : P ROCEDURE

Elimination of 2nd unknown Change the value of a22 to 1 and eliminate the other elements in the second column
1 a '12 0 a ' 22 0 a ' 32 a '13 a ' 23 a ' 33 b'1 - (1' ) b' 2 - (2' ) b' 3 - (3' )

GAUSS ELIMINATION

1 0 a ' '13 0 1 a ' ' 23 0 0 a ' ' 33

b' '1 - (1' ' ) b' ' 2 - (2' ' ) b' ' 3 - (3' ' )

eqn (2) multiply with 1/a22 eqn (2) eqn (2) multiply with a12 eqn (2a) eqn (1) minus eqn (2a) eqn (1) eqn (2) multiply with a32 eqn (2b) eqn (3) minus eqn (2b) eqn (3) The prime indicates that the elements ave been modified

9-42

G AUSS -J ORDAN E LIMINATION : P ROCEDURE

Elimination of 3rd unknown Change the value of a33 to 1 and eliminate the other elements in the third column
1 0 a ' '13 0 1 a ' ' 23 0 0 a ' ' 33 b' '1 - (1' ' ) b' ' 2 - (2' ' ) b' ' 3 - (3' ' )

GAUSS ELIMINATION

1 0 0 b' ' '1 - (1' ' ' ) 0 1 0 b' ' ' - (2' ' ' ) 2 - (3' ' ' ) 0 0 1 b' ' ' 3

eqn (3) multiply with 1/a33 eqn (2) eqn (3) multiply with a13 eqn (3a) eqn (1) minus eqn (3a) eqn (1) eqn (3) multiply with a23 eqn (3b) eqn (2) minus eqn (3b) eqn (2) The triple prime indicates that the elements have been modified three times

9-43

G AUSS -J ORDAN E LIMINATION : P ROCEDURE

Determination of the unknowns The value of the unknowns can be determined directly without the back substitution step as in the nave gauss elimination.
1 0 0 b' ' '1 - (1' ' ' ) 0 1 0 b' ' ' - (2' ' ' ) 2 - (3' ' ' ) 0 0 1 b' ' ' 3 1 0 0 x1 b' ' '1 0 1 0 x = b ' ' ' 2 2 b' ' ' 3 x3 0 0 1

GAUSS ELIMINATION

x1 b' ' '1 x 2 = b' ' ' 2 x 3 b' ' ' 3

gives

9-44

G AUSS -J ORDAN E LIMINATION : E XAMPLE

Use Gauss-Jordan method to solve the given set of equations below:
3x1 0.1x 2 0.2 x 3 = 7.85 0.1x1 + 7 x 2 0.3x 3 = 19.3 0.3x1 0.2 x 2 + 10 x 3 = 71.4
3 0.1 0.2 7.85 0.1 7 0 . 3 19 . 3 71.4 0.3 0.2 10

GAUSS ELIMINATION

Solution:

9-45

G AUSS -J ORDAN E LIMINATION : E XAMPLE - S OLUTION

3 0.1 0.2 7.85 - Eqn (1) - Eqn (2) 0.1 7 0 . 3 19 . 3 71.4 - Eqn (3) 0.3 0.2 10

GAUSS ELIMINATION

Eqn (1) x 1/3

1 0.0333 0.0667 2.6167 - Eqn (1' ) - Eqn (2) 0.1 7 0 . 3 19 . 3 0.2 10 71.4 - Eqn (3) 0.3

Eqn (2) Eqn (1) x 0.1, and Eqn (3) Eqn (1) x 0.3
2.6167 - Eqn (1' ) 1 0.0333 0.0667 0 7.0033 0.2933 19.5617 - Eqn (2' ) - Eqn (3' ) 0 0.1900 10.0200 70.6150

9-46

G AUSS -J ORDAN E LIMINATION : E XAMPLE - S OLUTION

2.6167 - Eqn (1' ) 1 0.0333 0.0667 0 7.0033 0.2933 19.5617 - Eqn (2' ) - Eqn (3' ) 0 0.1900 10.0200 70.6150

GAUSS ELIMINATION

Eqn (2) x 1/7.0033

1 0.0333 0.0667 2.6167 - Eqn (1' ) - Eqn (2' ' ) 0 1 0 . 0419 2 . 7932 - Eqn (3' ) 0 0.1900 10.0200 70.6150 1 0 0.0681 2.5236 - Eqn (1' ' ) 0 1 0.0419 2.7932 - Eqn (2' ' ) - Eqn (3' ' ) 0 0 10.0120 70.0843

Eqn (3) - Eqn (2) x -0.19, and Eqn (1) Eqn (2) x -0.0333

9-47

G AUSS -J ORDAN E LIMINATION : E XAMPLE - S OLUTION

1 0 0.0681 2.5236 - Eqn (1' ' ) 0 1 0.0419 2.7932 - Eqn (2' ' ) - Eqn (3' ' ) 0 0 10.0120 70.0843 1 0 0.0681 2.5236 - Eqn (1' ' ) 0 1 0.0419 2.7932 - Eqn (2' ' ) 1 7.0000 - Eqn (3' ' ' ) 0 0

GAUSS ELIMINATION

Eqn (3) x 1/10.120

Eqn(2) - Eqn(3) x -0.0419, and Eqn (1) Eqn (3) x -0.0681

1 0 0 3.0000 - Eqn (1' ' ' ) 0 1 0 2.5000 - Eqn (2' ' ' ) - Eqn (3' ' ' ) 0 0 1 7.0000