MNchap 1

Solving Nonlinear Equation
0
x
1
x
2
x
) (
1
x f
) (
0
x f
Root r
Numerical Methods Wen-Chieh Lin 2
Nonlinear Equations
Given function f, we find value x for which
Solution x is root of equation, or zero of
function f
So problem is known as root finding or zero
finding
0 ) ( = x f
Nonlinear Equations: two cases
Single nonlinear equation in one unknown, where
Solution is scalar x for which f(x) = 0
System of n coupled nonlinear equations in n
unknowns, where
Solution is vector x for which all components of f
are zero simultaneously, f(x) = 0
Example of 1-D nonlinear equation
for which x = 0.3604 is one approximate solution
Example of system of nonlinear equations in two
dimensions
for which x = [-1.8, 0.8] is one approximate
solution vector
1
4
2
2
2
2
1
1
= +
= +
x e
x x
x
Examples: Nonlinear Equations
0 ) sin( 3 = +
x
e x x
Multiplicity
If f(R) = f(R) = f(R) = = f
(m-1)
(R) = 0 but
f
(m)
(R) 0, then root R has multiplicity m
If m = 1 (f(R) = 0, f(R) 0 ), then R is simple
root
Bisection method begins with initial bracket and
repeatedly halves its length until solution has
been isolated as accurately as desired
http://www.cse.uiuc.edu/iem/nonlinear_eqns/Bisection/
Interval Halving (Bisection)
while |b-a| > tol,
m = a + (b-1)/2;
If f(a)*f(m) < 0,
b = m;
else
a = m;
end;
end;
a
b m
Bisection (cont.)
Simple and guaranteed to work if
f is continuous in [a, b]
[a, b] brackets a root
Needed iterations to achieve a specified accuracy is
known in advance
Error after n iterations < |b - a| / 2
n
Slow to converge
Good for initial guess for other root finding
algorithms
Finding the initial bracket may be a problem if f is not
given explicitly
Use graphing to assist root finding
Set the initial bracket
Detect multiple roots
Can we find a root in a better way?
Bisection only utilizes function values f(x)
We can find a root with fewer iterations if
other information is used
linear approximation of f(x)
Secant line secant method
Tangent line Newtons method
Polynomial approximation of f(x)
Mullers method
Secant Method
Approximate a function by a straight line
Compute the intersection of the line and x-axis
) ( ) (
) (
) (
) (
1 0
1 0
1
2 1
x f x f
x x
x f
x x
) ( ) (
) (
) (
1 0
1 0
1 1 2
x f x f
x x
x f x x
=
0
x
1
x
2
x
) (
1
x f
) (
0
x f
Root r
Secant Method (cont.)
Update endpoints
Repeat
) ( ) (
) (
) (
1
1
1
n n
n n
n n n
x f x f
x x
x f x x
+
0
x
1
x
) (
1
x f
) (
0
x f
Root r
with swap , ) ( ) ( if
1 0 1 0
x x x f x f <
Example
f(x) = 3x + sin(x) exp(x)
Find the root in [0, 1]
http://www.cse.uiuc.edu/iem/nonlinear_eqns/Secant/
0 0.5 1 1.5 2
-1
-0.5
0
0.5
1
1.5
Method of False Position
Problem of secant method
Remedy
Always bracket a root in the interval [x
0
, x
1
]
How to do this?
Newtons method
Better approximation using the first derivative
) (
0
x f
1 0
x x
0
x
1
x
1 0
0
0
) (
) ( ' ) tan(
x x
x f
x f
= =
) ( '
) (
0
0
0 1
x f
x f
x x =
) ( '
) (
1
n
n
n n
x f
x f
x x =
+
Interpretation of Newtons method
Truncated Taylor series
is a linear function of h approximating f near x
Replace nonlinear function f by this function,
whose zero is h = - f(x)/f(x)
Zeros of original function and linear
approximation are not identical, so repeat
process
) ( ' ) ( ) ( x hf x f h x f + ~ +
) ( '
) (
1
n
n
n n
x f
x f
x x =
+
Example: Newtons method
http://www.cse.uiuc.edu/iem/nonlinear_eqns/Newton/
Comparison of Secant and
Newtons methods
) ( ) (
) (
) (
1
1
1
n n
n n
n n n
x f x f
x x
x f x x
+
) ( '
) (
1
n
n
n n
x f
x f
x x =
+
Secant method
Newtons method
Pros and Cons of Newtons method
Pros
efficient
Cons
Need to know the derivative function
) ( '
) (
1
n
n
n n
x f
x f
x x =
+
When will Newtons method not converge?
x
1
=x
6
, loop
passing
maximum/minimum
Mullers method
Instead of linear approximation, Mullers
method uses quadratic approximate
Evaluation of derivatives are not required
See the textbook for details
Fixed-point Iteration Method
Rearrange f(x)=0 into an equivalent form x=g(x)
f(x) = x g(x) = 0
If r is a root of f, then f(r) = r-g(r) = 0
r=g(r)
In iterative form,
Also called function iteration
For given equation f(x)=0, there may be many
equivalent fixed-point problems x=g(x) with different
choice of g
Will the method always converge?
) (
1 n n
x g x =
+
Example: Fixed-point Iteration
0 3 2 ) (
2
= = x x x f
3 2 ) (
1
+ = x x g
2
3
) (
2
=
x
x g
Example: Fixed-point Iteration (cont.)
0 3 2 ) (
2
= = x x x f
2
3
) (
2
3
=
x
x g
D
i
v
e
r
g
e
!
Convergence Rate
For general iterative methods, define error at
iteration n by
e
n
= x
n
R
where x
n
is approximate solution and R is true
solution
For methods that maintain interval known to
contain solution, rather than specific
approximate value for solution, take error to be
length of interval containing solution
Convergence Rate (cont.)
Sequence converges with rate r if
for some finite nonzero constant C
Some cases of interest
r=1: linear (C<1)
r>1: superlinear
r=2: quadratic
C
e
e
r
n
n
n
=
+

1
lim
Convergence Rate of Bisection
Length of interval containing solution reduced
by half at each iteration
Linearly convergent
r = 1
C = 0.5
Convergence of Fixed-point Iteration
If R = g(R) and |g(R)| < 1, then there is an
interval containing R such that iteration
x
n+1
= g(x
n
)
converges to R if started within that interval
If |g(R)| > 1, then iterative scheme diverges
Proof of Convergent Condition
n n n
e g e ) ( '
1
=
+
) ( ) ( ) (
1 n n n
x g R g x g R x R = =
+
n
n
n
n
x R
x R
x g R g
x R
=
+
) ( ) (
1
) )( ( '
1 n n n
x R g x R =
+

Mean Value Theorem
) , (
n n
x R e
C
e
e
r
n
n
n
=
+

1
lim
Fixed-point iteration is linearly convergent
) ( ' R g C =
Convergence of Newtons Method
Represent Newtons method in fixed-point
iteration form
Condition for convergence
) (
) ( '
) (
1 n
n
n
n n
x g
x f
x f
x x = =
+
1
)] ( ' [
) ( " ) (
) ( '
2
< =
x f
x f x f
x g
Convergence rate of Newtons method
) ( ) (
1 n n
x g R g x R =
+
2 ) ( ) ( " ) ( ) ( ' ) ( ) (
2
n n n
x R g x R R g R g x g + + =
0 ) ( '
)] ( ' [
) ( " ) (
) ( '
0 ) (
2
= =
=
R g
x f
x f x f
x g
R f
2 ) ( ) ( " ) ( ) (
2
n n
x R g R g x g + =
2 ) ( " ) ( ) (
2
1 1 n n n n
e g x g R g x R e = = =
+ +

C
e
e
r
n
n
n
=
+

1
lim
Newtons method is
quadratically convergent!
Recall that
Question from the last class
Is the initial solution important for the
convergence of fixed-point iteration?
Answer to the convergence problem of
fixed-point Iteration
|g(R)|<1 is a necessary condition
if |g(R)|>1, the fixed-point iteration will diverge
even if the initial condition is very close to a root
since the iteration will eventually reach the region
causing divergence
Initial solution is important but less critical
The algorithm may not converge if the initial
solution is far from the true solution
Recall the conditions that Newtons method does
not converge?
Example: Newtons Method for Finding
Complex Roots
f(x) = x
3
+ 2x
2
x + 5
Start Newtons method with a complex value
i x + =1
0
i x f 5 2 ) (
0
+ = i x f 10 3 ) ( '
0
+ =
i
i
i
i
x f
x f
x x 04587 . 1 486238 . 0
10 3
5 2
1
) ( '
) (
0
0
0 1
+ =
+
+
+ = =
5 2 ) (
2 3
+ + = x x x x f 1 4 3 ) ( '
2
+ = x x x f
i
x f
x f
x x 23665 . 1 448139 . 0
) ( '
) (
1
1
1 2
+ = =
Newtons Methods for Multiple Roots
Quadratically convergent for simple root,
Linearly convergent for multiple roots as
0 ) ( '
)] ( ' [
) ( " ) (
) ( '
0 ) (
2
= =
=
R g
x f
x f x f
x g
R f
0
0
)] ( ' [
) ( " ) (
) ( '
2
= =
R f
R f R f
R g
Remedies for Multiple Roots with
Newtons method
If f(x) has a root of multiplicity k at x=R, we
can factor out (x-R)
k
from f(x) to get
With a slightly modified Newtons method
It can be proved that and
Newtons method still converges quadratically
) ( ) ( ) ( x Q R x x f
k
=
0 ) ( ' = R g
k
) (
) ( '
) (
1 n k
n
n
n n
x g
x f
x f
k x x = =
+
Remedies for Multiple Roots with
Newtons method (cont.)
In practice, we dont know k in advance
Remedies
Try and error
Deflate f(x)
f(x)/(x-s) where s is an approximate R
Be warned that an indeterminate form at x=R is
created
Systems of Nonlinear Equations
Solving systems of nonlinear equations is
much more difficult than scalar case because
Wide variety of behavior is possible, so
determining existence and number of solutions or
good starting guess is much more complex
In general, there is no simple way to guarantee
convergence to desired solution or to bracket
solution to produce absolutely safe method
Computational overhead increases rapidly with
dimension of problem
Example: Systems in Two Dimensions
From Michael T. Heath
Newtons Method
In n dimensions, Newtons method has form
where J(x) is Jacobian matrix of f
In practice, we do not explicitly invert J(x
n
),
but instead solve linear system
for Newton step s
n
, then take as next iterate
) ( ) (
1
n n n 1 n
x f x J x x

+
=
) ( ) (
n n n
x f s x J =
n n 1 n
s x x + =
+
Example: Newtons Method
0
1
4
) , (
) , (
) (
2
2
2
2
1
2 1 2
2 1 1
1
=
(
+
+
=
(
=
x e
x x
x x f
x x f
x
x f
(
=
(
(
(
(
c
c
c
c
c
c
c
c
=
1
2 2
) (
1
2 1
2
2
1
2
2
1
1
1
x
e
x x
x
f
x
f
x
f
x
f
x J
| |
T
7 . 1 1
0
= x
(
=
0183 . 0
11 . 0
) (
0
x f
(

=
1 7183 . 2
4 . 3 2
) (
0
x J
) ( ) (
0 0 0
x f s x J =
(
=
(

0183 . 0
11 . 0
1 7183 . 2
4 . 3 2
0
s
T
] 7298 . 1 0043 . 1 [
0 0
= + = s x x
1
Example: Newtons Method
0
1
4
) , (
) , (
) (
2
2
2
2
1
2 1 2
2 1 1
1
=
(
+
+
=
(
=
x e
x x
x x f
x x f
x
x f
(
=
(
(
(
(
c
c
c
c
c
c
c
c
=
1
2 2
) (
1
2 1
2
2
1
2
2
1
1
1
x
e
x x
x
f
x
f
x
f
x
f
x J
| |
T
7298 . 1 0043 . 1
1
= x
(
=
4 9651 . 1
4 2653 . 8
) (
1
e
e
x f
(

=
1 7300 . 2
4596 . 3 0086 . 2
) (
1
x J
) ( ) (
1 1 1
x f s x J =
(
=
(

4 9651 . 1
4 2653 . 8
1 7300 . 2
4596 . 3 0086 . 2
1
e
e
s
T
] 729637 . 1 004169 . 1 [
1 1 2
= + = s x x
Fixed-Point Iteration
Fixed-point problem for is to find
vector x such that
Corresponding fixed-point iteration is
Converges if
starts close enough to solution
) (x g x =
) (
1 n n
x g x =
+
n n
R R : g
1 )) ( ( < R J
A
A
R
J
matrix a of s eigenvalue the
of norm complex maximum : ) (
solution true :
matrix Jacobian :

Fixed-Point Iteration (cont.)
Convergence rate is normally linear, with
constant C given by
If then convergence rate is at least
quadratic, e.g., Newtons method
n n
R R : g
0 ) ( = R J
)) ( ( R J
Example: Fixed-point Iteration
0
1
4
) , (
) , (
) (
2
2
2
2
1
2 1 2
2 1 1
1
=
(
+
+
=
(
=
x e
x x
x x f
x x f
x
x f
(

=
(
2
1
2
2
1
4
) 1 ln(
x
x
x
x
| |
T
7 . 1 1
0
= x
1.00000000000000 -1.73205080756888
1.00505253874238 -1.72912388057290
1.00398063482304 -1.72974647995027
1.00420874039761 -1.72961406264780
1.00416023020985 -1.72964222660783
Next Monday
Introduction to Matlab/Octave
Matlab offers a student version with discounted
price
Octave is a shareware that has similar environment
as Matlab!
You are encouraged to bring your laptop with
Octave or Matlab installed next Monday

MNchap 1

Caricato da

Informazioni sul documento

Descrizione originale:

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

MNchap 1

Caricato da

Copyright:

Formati disponibili

Solving Nonlinear Equation

Numerical Methods Wen-Chieh Lin 44

Potrebbero piacerti anche