Sei sulla pagina 1di 14

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Extremely powerful problem-solving tools


Design your own programs to solve
practical problems without expensive
software
Efficient vehicle for learning to use
computer
Reinforce understanding of mathematics
You will see particular reasons and
applications in next slide.

Why Study Numerical Methods?

http://www.cs.aaue.dk/~ymzhang/courses/NumMethod/S2006.html

Phone: 7912 7741 Office Location: FUV 0.22


Email: ymzhang@cs.aaue.dk

Youmin Zhang

F6-2: Numerical Methods


in Signals and Systems

Lecture 1

Not to be
covered

Course
Contents

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Text: S.C. Chapra and R.P. Canale, Numerical Methods for


Engineers: With Software and Programming Applications
4th (or 5th) edition, McGraw-Hill, 2002 (or 5th, 2006)

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

A class of methods for solving a wide


variety of mathematical problems
These methods can be implemented
directly on digital computers
Are capable of handling nonlinearities,
complex geometries, and large system
of coupled equations of many real
physical situations

What Are Numerical Methods?

Lectures 1 & 2: Solving Linear and


Non-linear Algebraic Equations

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Lecture 1

Nonlinear equations

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Newtons method:

Gauss, Pivoting, LU: Linear equations

Newtons Method (6.5, 9.6)

LU-Decomposition and Gauss-Seidel Method


(10.1, 11.2)

Pivoting Strategies and Gauss-Jordan Elimination


(9.3, 9.4, 9.7)

Gauss Elimination (9.1, 9.2)

Overview (PT3.2)

Lecture 1

Note: (g) not to be covered

Course Contents (contd)

Lecture 1

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Sections PT 3.2, 9.1, 9.2

Overview and Gauss Elimination

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Lecture 5: Least-squares regression for curve


fitting and summary to the course

Lecture 4: Optimization

Lecture 3: Solving ordinary differential equations

Lecture 2: Solving linear and non-linear algebraic


equations (Part II)

Lecture 1: Solving linear and non-linear algebraic


equations (Part I)

Course Contents (contd)

Nonlinear Equations: Example

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

f1 ( x1 , x2 , L xn ) = 0

f 2 ( x1 , x2 , L xn ) = 0

M
M
f n ( x1 , x2 , L xn ) = 0

=0

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

a11 x1 + a12 x1 x2 + L + a1n ( xn ) = b1

a21 ( x1 ) 3 + a22 e x2 + L + a2 n ( x2 ) 3 / xn = b2

M
M
a x + a x + L + a x = b
nn n
n
n1 1 n 2 2

Lecture 1

Matrix/vector
representation

Solutions of
equations with
multiple variables

A set of equations:

Typical methods: Bisection method; NewtonRaphson method; Secant method; and Mllers
method (Chapters 5-7)
We will not elaborate more on this topic

Roots of a single equation (Part 2 in the Text):


Solutions of equations with one variable f ( x )

Overview

11

Lecture 1

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

5. Matrix operating rules: addition and


multiplication

4. Special types of square matrices: symmetric,


diagonal, identity, upper triangular, lower
triangular

3. Square matrices, principal or main diagonal

2. Row vectors and column vectors

1. Matrix

Matrix Notation

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

a11 x1 + a12 x2 + L + a1n xn = b1

a 21 x1 + a 22 x2 + L + a 2 n xn = b2

M
M
a n1 x1 + a n 2 x2 + L + a nn xn = bn

Linear Algebraic Equations

12

10

Lecture 1

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Principal or main diagonal: aii

Square Matrix

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Matrix and Elements

15

13

Element: aij

Matrix: [A]

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Lecture 1

1
3
7
39

Identity

Diagonal

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Lower Triangular

a11

a 21 a22

[ A] =

M
O

a n1 LL ann

[I ] =

a11

a 22

[ D] =

a nn

a11 a12 L a1n

a 22 L a 2 n

[ A] =

O
M

a nn

Upper Triangular

2 16
7 39
9 6

6 88
Symmetric

5
1
[ A] =
2

16

16

14

{C}T or {C}' is transpose of {C}

{C}T = [c1 c2 L cn ]

Special Types of Square Matrices

Lecture 1

c1
c
{C} = 2
M

c n

Column vectors:

{B} = [b1 , b2 ,L , bn ] or b = [b1 , b2 ,L , bn ]

Row vectors:

Row Vectors & Column Vectors

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

17

(E1.2)

(E1.1)

Lecture 1

3 x1 + 2 x2 = 18

8 x2 = 24

x1 = 4

x2 = 3

The Graphic Method

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

(E1.1)
(E1.2)

(E1.2)3 +(E1.1) or (E1.2)-(E1.1)(-1/3)

The Elimination of Unknowns:

3 x1 + 2 x2 = 18

x1 + 2 x2 = 2

19

If n = 2 or 3, the problem is very simple. Such problems can


be solved by Graphical Method (n = 2) or by variable
elimination

Example 1: Linear Algebraic Equation

Lecture 1

[ A]1

[ A][ A]1 = [ A]1[ A] = [ I ]

Inverse of matrix [A]:

k =1

cij = aik bkj

Multiplication: [C] = [A]*[B]

cij = aij + bij

Addition: [C] = [A] + [B]

For matrices [A] and [B]

Matrix Operating Rules

a21 x1 + a22 x2 + L + a2 n xn = b2

Lecture 1

Lecture 1

a12 L a1n x1 b1
a22 L a2 n x2 b2
=
M M M M M

an 2 L ann xn bn

or

[ A]{ X } = {B}

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

2. Consequently, this equation could be solved


directly and the result back-substituted into
one of the original equations to solve for the
remaining unknowns.

1. The equations were manipulated to


eliminate one of the unknowns from the
equations. The result of this elimination step
is that we can have one equation with one
unknown.

Two-step procedure:

Observation

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

where aij and bi are known constants. x j are the


variables to be determined to satisfy the n linear
equations simultaneously.

a11
a
21
M

an1

an1 x1 + an 2 x2 + L + ann xn = bn
or in matrix form, it can be written as

A set of linear equation

a11 x1 + a12 x2 + L + a1n xn = b1

Gauss Elimination

20

18

Forward Elimination

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

(t1.k )

(t1.k )

ak 1
a
a
) x2 + L + (akn k1 ) xn = bk - k1 b1
a11
a11
a11

ak 2 x2 + L + akn xn = bk

21

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

a11 x1 + a12 x2 + L + a1n xn = b1


a x + a x + L + a x = b
21 1 22 2
2n n
2

M
M

an1 x1 + an 2 x2 + L + ann xn = bn

23

(t1.n)

(t1.2)

(t1.1)

2. The same procedure is then repeated for the remaining


equations: for k=3,,n, calculate (t1.k)-(t1.1)(ak1/a11)

( ak 2

1. To eliminate the first unknown, x1, in kth equation, from


the 2nd through the nth equations: (t1.k)-(t1.1)(ak1/a11)

Lecture 1

The Two Phases of Gauss Elimination

a11 x1 + a12 x2 + L + a1n xn = b1


a x + a x + L + a x = b
21 1 22 2
2n n
2

M
M

an1 x1 + an 2 x2 + L + ann xn = bn

(t1.n)

(t1.2)

(t1.1)

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

(t1.n (n -1) )

(t1.3)

(t1.1)
(t1.2)

24

22

See blackboard!

(t1.n)

(t1.2)

(t1.1)

Forward Elimination (contd)

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

a11 x1 + a12 x2 + a13 x3 + L + a1n xn = b1

x2 + a23
x3 + L + a2 n xn = b2
a22

x3 + L + a3n xn = b3
a33

( n 1)

ann
xn = bn( n 1)

Lecture 1

a11 x1 + a12 x2 + L + a1n xn = b1


a x + a x + L + a x = b
21 1 22 2
2n n
2

M
M
an1 x1 + an 2 x2 + L + ann xn = bn

Naive Gauss Elimination

b
a

Lecture 1

for i = n-1, n-2 , ..., 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

j = i +1
( i 1)
ii

aiji 1 x j

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Back Substitution

Forward Elimination

Pseudocode to
Perform Gauss
Elimination

Lecture 1

xi =

bi(i 1)

2. The value of xn can be back-substituted into the (n1)th equation to solve for xn-1. This procedure repeats
for all subsequent equations.

xn =

( n 1)
n
( n 1)
nn

1. Equation (t1.n(n-1)) can be solved for xn:

a11 x1 + a12 x2 + a13 x3 + L + a1n xn = b1

x2 + a23
x3 + L + a2 n xn = b2
a22

x3 + L + a3n xn = b3
a33

( n 1)
ann
xn = bn( n 1)

Back Substitution

27

25

(t1.n (n -1) )

Lecture 1

Lecture 1

(3)'(2)'(2.6 / 2)

x3

6
-5
-8.1

6
-5
-1.6

6
7
2

2.87654321 0.135802469 -0.654320988

x2

4
-2
0

5
0
0

x1

4
-2
2.6

4
6
5

ak 1
a
a
) x2 + L + (akn k1 ) xn = bk - k1 b1
a11
a11
a11

5
0
0

5
10
3

( ak 2

Naive Gauss Elimination

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Summary

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

(3)

(2) (1) (10 / 5)


(3) (1) (3 / 5)

3 x1 + 5 x2 + 2 x3 = 8

10 x1 + 6 x2 + 7 x3 = 25 (2)

(t1.3)

(1)

5 x1 + 4 x2 + 6 x3 = 11

(t1.1)
(t1.2)

Example 2

11
3
5.3

11
3
1.4

11
25
8

28

26

(t1.k)

Pivot equation

Forward Elimination

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Pivot element

ak 1
a
a
) x2 + L + (akn k1 ) xn = bk - k1 b1
a11
a11
a11

(t1.k )

(t1.k )

29

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

31

2. The same procedure is then repeated for the remaining


Normalization
equations: (t1.k)-(t1.1)(ak1/a11)
(t1.1)/a11

xn = bk
a2 k x2 + L + akn

( a2 k

1. To eliminate the first unknown, x1, from the second


through the nth equations: (t1.k)-(t1.1)(ak1/a11)

Lecture 1

Sections 9.3, 9.4, 9.7

Pivoting Strategies and Gauss-Jordan


Elimination

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

When the determinant is close to zero.

Ill-conditioned Systems

It may happen for larger problems. Check the


solution and may consider using double-precision
variables.

Round-off Errors

The pivot element may be zero or may be very


close to zero - the problem is not well conditioned.

Division by Zero

Pitfalls of Naive Gauss Elimination

Lecture 1

32

30

Review of Naive Gauss Elimination

Lecture 1

(c ) too close

2
x2 =
3

(E2.2)

(E2.1 )

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

2.0001 3 x2 2.0001 3(2 / 3)


x1 =
=
0.0003
0.0003

Step 3: Get the solution

9999 x2 = 6666

Step 2: (E2.2)-(E2.1)

1.0000 x1 + 10,000 x2 = 6667

Step 1: (E2.1)1.0000/0.0003 yields

(E2.2)

(E2.1)

Example 2 round-off errors

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

(b) infinite solution

0.0003 x1 + 3.0000 x2 = 2.0001

1.0000 x1 + 1.0000 x2 = 1.0000

Lecture 1

(a) no solution

Singular and Ill-conditioned


Systems

35

33

Lecture 1

Lecture 1

2
3

0.6667
0.66667
0.666667
0.6666667

4
5
6
7

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Round-off errors can bring in wrong solution !

0.667

x2

2.0001 3 x2
0.0003

0.67

x1 =

Significant
Figures

x2 =

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

What can we do in this case?

Equation (t1.k): division by zero

8
{B} = -3
5

(E1.3)

(E1.2)

(E1.1)

0.333000

0.33000

0.3000

0.000

-3.00

-33.0

x1

Example 2: Observation

a11 = 0

0 2 3
[ A] = 4 6 7
2 1 6

2 x2 + 3 x3 = 8

4 x1 + 6 x2 + 7 x3 = 3
2 x + x + 6 x = 5
3
1 2

Example 1 division by zero

36

34

1.0000 x1 + 1.0000 x2 = 1.0000

0.0003 x1 + 3.0000 x2 = 2.0001

0.0003 x1 + 3.0000 x2 = 2.0001

1.0000 x1 + 1.0000 x2 = 1.0000

Lecture 1

x2 =

2
3

(E2.1 )

(E2.2)

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

2.9997 x2 = 1.9998
1.0000 x2
x1 =
1.0000

(E2.1)-(E2.2)

0.0003 x1 + 0.0003 x2 = 0.0003

(E2.1)

(E2.2)

(E2.2)

(E2.1)

Partial Pivoting: Example

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

(E2.2)0.0003/1.0000 yields

Lecture 1

Scaling

Pivoting

Use of more significant figures

39

37

Techniques for Improving Solutions

(E1.3)

(E1.2)

(E1.1)

(E2.2)

(E2.1)

Lecture 1

Lecture 1

2
3

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

0.6666667

0.666667

0.66667

0.6667

4
5

0.667

x2

1.0000 x2
1.0000

0.67

x1 =

Significant
Figures

x2 =

Observation

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

0.3333333

0.333333

0.33333

0.3333

0.333

0.33

x1

40

38

Before each row is normalized, find the largest available


coefficient in the column below the pivot element. The rows
can then be switched so that the largest element is used as
the pivot element. Partial Pivoting

0.0003 x1 + 3.0000 x2 = 2.0001

1.0000 x1 + 1.0000 x2 = 1.0000

2 x2 + 3 x3 = 8

4 x1 + 6 x2 + 7 x3 = 3
2 x + x + 6 x = 5
3
1 2

Pivoting

Complete Pivoting

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Note: complete pivoting is rarely used


because switching columns changes the
order of the xs and, consequently, adds
significant and usually unjustified
complexity to the computer program.

If columns as well as rows are searched


for the largest element and then
switched, the procedure is called
complete pivoting.

Lecture 1

Switch rows

Find the pivot element

Pivoting:
Pseudocode

43

41

(E1.3)

(E1.2)

(E1.1)

0.00002 x1 + x2 = 1

x1 + x2 = 2

2 x1 + 100,000 x2 = 100,000

x2 = 2
x1 +

Scaling

(E3.2)

(E3.1)

(E3.2)

(E3.1)

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Divide each equation by its maximum element so that the


maximum element in the equation is 1. Scaling

Lecture 1

Please verify the solution by yourself !

x3 = 2.6364 x2 = 0.0454 x1 = 5.4318

Solution:

2 x2 + 3x3 = 8

4 x1 + 6 x2 + 7 x3 = 3
2 x + x + 6 x = 5
3
1 2

Partial Pivoting: Example

44

42

(E3.2)

(E3.1)

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

x1 = x2 = 1.00

Back-substitution

x2 = 2
x1 +

99,998 x2 = 99,996

Forward elimination

x2 = 2
x1 +

2 x1 + 100,000 x2 = 100,000

0.00002 x1 + x2 = 1

x1 + x2 = 2

(E3.2)

(E3.1)

(3) Use the scaled coefficient to determine whether


pivoting is necessary. However, actually solve the
equations with the original coefficient values. 2xx ++ 100,000xx == 2100,000

Lecture 1

(E3.2)

(E3.1)

45

Note: Although x2 is correct, x1 is 100 percent in error because


of round-off. The correct answer are x2 = 1.00002, x1 = 0.99998.

x2 = 1.00 x1 = 0.00

Back substitution

2 x1 + 100, 000 x2 = 100, 000

49,999 x2 = 49,998

Forward elimination

(1) Solve this equation using Gauss elimination and a


pivoting strategy (for two significant figures).

2 x1 + 100,000 x2 = 100,000

x2 = 2
x1 +

Scaling: Example

(E3.2)

(E3.1)

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

It can be used to determine the pivoting


equation, but the original coefficient values
are retained for the actual elimination and
substitution computations.

48

(E3.2)

(E3.1)

Scaling leads to correct answer.

0.00002 x1 + x2 = 1

x1 + x2 = 2

2 x1 + 100,000 x 2 = 100,000

x2 = 2
x1 +

Scaling: Observation

x1 = x2 = 1.00

Back-substitution

x1 + x2 = 2

x2 = 1.00

Forward elimination

x1 + x2 = 2

0.00002 x1 + x2 = 1

Pivoting

0.00002 x1 + x2 = 1

x1 + x2 = 2

(2) Repeat the solution after scaling the equations

Lecture 1

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Gauss elimination is preferred.

Gauss-Jordan method involves more


operations than Gauss elimination.

Gauss-Jordan Elimination: Comment

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

A variation of Gauss
elimination
Major differences:
an unknown has to be
eliminated from all
equations
all rows are normalized
the elimination results
in an identify matrix
no need for back
substitution for the
solution
can be used to obtain
matrix inverse

Gauss-Jordan
Elimination

51

49

4
6
5
4
-2
2.6
4
-2
0
4
-2
0
4
-2
0
4
1
0
0
1
0
0
1
0

6
7
2
6
-5
-2
6
-5
-8.1
6
-5
1
0
0
1
0
0
1
0
0
1
0
0
1

Summary

5
10
3
5
0
0
5
0
0
5
0
0
5
0
0
5
0
0
5
0
0
1
0
0

11.00000
25.00000
8.00000
11.00000
3.00000
1.400000
11.00000
3.00000
5.30000
11.00000
3.00000
-0.65432
14.92593
-0.27160
-0.65432
14.92593
0.13580
-0.65432
14.38272
0.13580
-0.65432
2.87654
0.13580
-0.65432

x1
x2
x3

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

Gauss-Jordan elimination method (a variant of


Gauss elimination method)

9 Scaling

9 Pivoting

pitfalls existing in the Naive Gauss elimination method: division by


zero and round-off errors)

Two numerically improved methods (for avoiding

(Naive) Gauss elimination method

We have learnt today for solving linear algebraic


equations using:

Solution:

3 x1 + 5 x2 + 2 x3 = 8

10 x1 + 6 x2 + 7 x3 = 25

Example:
5 x1 + 4 x2 + 6 x3 = 11

Making the A matrix a diagonal matrix and find the solution.

Gauss-Jordan Elimination: Example

52

Lecture 1

Lecture Notes on Numerical Methods,


Methods, by Y. M. Zhang (AUE)

53

3. Verify your results in Problem 9.8 using the MATLAB


code you have written.

2. Write a MATLAB code for the Pseudocode given in


slide #27 to perform Gauss elimination.

1. Problems 9.1; 9.2; 9.8; 9.9; 9.10.

Exercise:

9 Pivoting Strategies and Gauss-Jordan Elimination


(9.3, 9.4, 9.7)

9 Gauss Elimination (9.1, 9.2)

Reading:

Reading and Exercise

Potrebbero piacerti anche