Numerical Methods

A Manual
Lektor: Youri V. Shestopalov
e-mail: youri.shestopalov@kau.se Tel. 054-7001856
Homepage: www.ingvet.kau.se\ youri
Karlstads Universitet
2000
1
Contents
1 Numerical Methods in General 3
1.1 Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Formulas for errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Error propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Loss of signicant digits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Binary number system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Solution of Nonlinear Equations 6
2.1 Fixed-point iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Convergence of xed-point iterations . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 Newtons method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Secant method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.5 Numerical solution of nonlinear equations using MATLAB . . . . . . . . . . . . 11
3 Interpolation 13
3.1 Quadratic interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.2 Estimation by the error principle . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3 Divided dierences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4 Numerical Integration and Dierentiation 29
4.1 Rectangular rule. Trapezoidal rule . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.2 Error bounds and estimate for the trapezoidal rule . . . . . . . . . . . . . . . . 30
4.3 Numerical integration using MATLAB . . . . . . . . . . . . . . . . . . . . . . . 32
4.4 Numerical dierentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5 Approximation of Functions 33
5.1 Best approximation by polynomials . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.2 Chebyshev polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6 Numerical Methods in Linear Algebra. Gauss Elimination 42
6.1 Gauss elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.2 Application of MATLAB for solving linear equation systems . . . . . . . . . . . 46
7 Numerical Methods in Linear Algebra. Solution by Iteration 49
7.1 Convergence. Matrix norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
7.2 Jacobi iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
7.3 GaussSeidel iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
7.4 Least squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
8 Numerical Methods for First-Order Dierential Equations 54
8.1 Euler method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
8.2 Improved Euler method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
8.3 RungeKutta methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
8.4 Numerical solution of ordinary dierential equations using MATLAB . . . . . . 59
2
1 Numerical Methods in General
Signicant digit (S) of a number c is any given digit of c except possibly of zeros to the left
of the rt nonzero digit:
each of the numbers 1360. 1.360, 0.001360 has 4 signicant digits.
In a xed-point system all numbers are given with a xed number of decimal places:
62.358, 0.013, 1.000.
In a oating-point system the number of signicant digits is xed and the decimal point
is oating:
0.6238 10
3
= 6.238 10
2
= 0.06238 10
4
= 623.8,
0.1714 10
13
= 17.14 10
15
= 0.01714 10
12
= 0.00000000000001714
(13 zeros after the decimal point)
0.2000 10
1
= 0.02000 10
2
= 2.000,
also written
0.6238E03 0.1714E-13, 0.2000E01.
Any number a can be represented as
a = m 10
e
, 0.1 m < 1, e integer.
On the computer m is limited to t digits (e.g., t = 8) and e is also limited:
a = m 10
e
, m = 0.d
1
d
2
. . . d
t
, d
1
> 0, |e| < M.
The fractional part m (or m) is called the mantissa and e is called the exponent.
The IEEE Standard for single precision: 38 < e < 38.
1.1 Rounding
3.45 3.4, 3.55 3.6 to 1 decimal [2S (signicant digits)]
1.2535 1.254 to 3 decimals (4S),
1.2535 1.25 to 2 decimals (3S),
1.2535 1.3 to 1 decimal (2S),
but 1.25 1.2 to 1 decimal (2S),
1.2 Formulas for errors
If a is an approximate value of a quantity whose exact value is a then
= a a
is called the error of a. Hence
a = a +
For example, if a = 10.5 is an approximation of a = 10.2, its error is = 0.3.
3
The relative error
r
of a

r
=

a
=
a a
a
=
Error
True value
(a = 0).

r


a
.
The error bound for a is a number such that
|| hence |a a| .
Similarly, for the relative error, the error bound is a number
r
such that
|
r
|
r
hence

a a
a

r
.
1.3 Error propagation
In addition and subtraction, an error bound for the results () is given by the sum of the error
bounds for the terms (
1
and
2
):
||
1
+
2
.
In multiplicaion and division, a bound for the relative error of the results (
r
) is given (ap-
proximately) by the sum of the bounds for the relative errors of the given numbers (
r1
and

r2
):
|
r
|
r1
+
r2
.
1.4 Loss of signicant digits
Consider rst the evaluation of
f(x) = x[

x + 1

x] (1)
for an increasing x using a six-digit calculator (6S computations):

x computed f(x) true f(x)

1.00000 .414210 .414214

10.0000 1.54430 1.54347
100.000 4.99000 4.98756
1000.00 15.8000 15.8074
10000.0 50.0000 49.9988
100000 100.000 158.113

In fact, in 6S computations, we have, for x = 100,

100 = 10.0000,

101 = 10.0499. (2)

4
The rst value is exact, and the second value is correctly rounded to six signicant digits (6S).
Next

x + 1

x =

101

100 = 0.04990 (3)

The true value should be .049876 (rounded to 6D). The calculation performed in (3) has a a loss-
of-signicance error: three digits of accuracy in

x + 1 =

101 were canceled by subtraction

of the corresponding digits in

x =

100.
The properly reformulated expression
f(x) = x

x + 1

x
1

x + 1 +

x + 1 +

x
=
x

x + 1 +

x
(4)
will not have loss-of-signicance error, because we have no subtraction. Completing the 6S
computation by (4) we obtain the result
f(100) = 4.98756
which is true to six digits.
1.5 Binary number system
In the decimal system (with the base 10 and allowed digits 0, 1, ..., 9), a number, for example,
342.105, means
3 10
2
+ 4 10
1
+ 2 10
0
+ 1 10
1
+ 0 10
2
+ 5 10
3
(5)
In the binary system (with the base 2 and allowed digits 0, 1), a number, for example,
1101.11 means
1 2
3
+ 1 2
2
+ 0 2
1
+ 1 2
0
+ 1 2
1
+ 1 2
2
(6)
in the decimal system. We write also
(1101.11)
2
= (13.75)
10
because
1 2
3
+ 1 2
2
+ 0 2
1
+ 1 2
0
+ 1 2
1
+ 1 2
2
= 8 + 4 + 1 + 0.5 + 0.25 = 13.75.
To perform the conversion from the decimal to binary system of a number
a
n
2
n
+ a
n1
2
n1
+ . . . + a
1
2
1
+ a
n
2
0
= x (7)
or
(a
n
a
n1
. . . a
1
a
0
)
2
= (x)
10
divide x by 2 and denote the quotient by x
1
; the reminder is a
0
. Next divide x
1
by 2 and denote
the quotient by x
2
; the reminder is a
1
. Continue this process to nd a
2
, a
3
, . . . a
n
in succession.
5
Problem 1.1
Floating-point form of the numbers
23.49, 302.867, 0.000527532, 0.25700
rounded to 4S (signicant digits):
0.2349 10
2
, 0.3029 10
3
, 0.5275 10
3
, 0.2570 10
5
also written
0.2349E02, 0.3029E03, 0.5275E-3, 0.2570E00.
Problem 1.2
Compute
a
b c
=
a(b + c)
b
2
c
2
, with a = 0.81534, b = 35.724, c = 35.596.
and dierent types of rounding.
We calculate
R =
a(b + c)
b
2
c
2
step by step with 5S:
b + c = 71.320, a(b + c) = 58.150, b
2
= 1276.2, c
2
= 1267.1, b
2
c
2
= 9.1290,
R =
58.150
9.1290
= 6.3698
Round to 4S, 3S, and 2S to obtain
R =
58.15
9.129
= 6.370, R =
58.2
9.13
= 6.37,
R =
58
9.1
= 6.4, R =
60
10
= 6.
2 Solution of Nonlinear Equations
There are some equations of nonlinear type for which there exist analytical methods leading
to a formula for the solution. The quadratic equations or trigonometric equations such as
x
2
3x + 2 = 0, sin x + cos x = 1 provide simple examples. Among them is of course a
linear equation ax + b = 0.
However, many nonlinear equations, which may be represented in the general form
f(x) = 0 (8)
where f(x) is assumed to be an arbitrary function of one variable x, cannot be solved directly
by analytical methods and a numerical method based on approximation must be used.
Each numerical method for the solution of nonlinear equation (8) has the form of an algo-
rithm which produces a sequence of numbers x
0
, x
1
, . . . approaching a root of (8). In this case
we say that the method is convergent. A method must contain a rule for the stop, when we are
close enough to the true value of the solution.
6
But usually the true solution is not known and this rule may be chosen in the form of
simultaneous fullment of two conditions
| f(x
N
) | <
f
; (9)
|x
N
x
N1
| <
x
(10)
at a certain step with number N = 1, 2, . . ., where
f
and
x
are given and they indicate the
required accuracy of the method.
The value x
0
plays a special role and it is called initial approximation.
2.1 Fixed-point iteration
Consider the equation
f(x) = 0
and assume that we can rewrite it in the equivalent form
x = g(x).
Then the xed-point iterations for nding roots of x = g(x) is dened by
x
n+1
= g(x
n
) (11)
Consider solving the equation
f(x) = x
2
5
to nd the root a =

5 = 2.2361 (5S). List four types of xed-point iterations:

i x
n+1
= g
1
(x
n
) = 5 + x
n
x
2
n
ii x
n+1
= g
2
(x
n
) =
5
x
n
iii x
n+1
= g
3
(x
n
) = 1 + x
n

1
5
x
2
n
iv x
n+1
= g
4
(x
n
) =
1
2
_
x
n
+
5
x
n
_
-
n x
n
,i x
n
,ii x
n
,iii x
n
,iv
-
0 2.5 2.5 2.5 2.5
1 1.25 2.0 2.25 2.25
2 4.6875 2.5 2.2375 2.2361
3 12.2852 2.0 2.2362 2.2361

2.2 Convergence of xed-point iterations

Example 2.1
Consider the equation
f(x) = x
2
3x + 1
7
written in the form
x = g
2
(x) = 3
1
x
.
Solve the equaiton x = g
2
(x) by the xed-point iteration process. Setting x
0
= 1 we obtain
x
1
= g
2
(x
0
) = 3
1
1
= 2, x
2
= g
2
(x
1
) = 3
1
2
= 2.5,
x
3
= g
2
(x
2
) = 3
1
2.5
= 3
2
5
=
13
5
= 2.6, x
4
= g
2
(x
3
) = 3
1
2.6
= 2.615, . . .
so that
1 = x
0
< x
2
< x
3
< . . . hence 1 =
1
x
0
>
1
x
1
>
1
x
2
. . .
and
1
x
n
K =
1
2
, n = 1, 2, . . . .
Let us perform a stepwise estimation of the dierence |x
n+1
x
n
| and then of the error |x
n+1
s|
where s = g(s) is the desired solution, taking into account that x
n+1
= g(x
n
), n = 0, 1, 2, . . .:
|x
2
x
1
| = |g
2
(x
1
) g
2
(x
0
)| =

3
1
x
1
3 +
1
x
0

1
x
0

1
x
1

=
|x
0
x
1
|
|x
0
||x
1
|
=
1
5
|x
1
x
0
|;
|x
3
x
2
| = |g
2
(x
2
) g
2
(x
1
)| =

1
x
1

1
x
2

=
|x
2
x
1
|
|x
2
||x
1
|
=
1
2.6 2.5
|x
2
x
1
| =
1
6.5
|x
2
x
1
| =
1
6.5
1
5
|x
1
x
0
| =
1
30.25
|x
1
x
0
|;
. . . ;
|x
n+1
x
n
| = p
1
p
2
. . . p
n
|x
1
x
0
|, 0 < p
i
K < 1, n = 0, 1, 2, . . . ,
in summary,
|x
n+1
x
n
| K
n
|x
1
x
0
| =
1
2
n
|x
1
x
0
|
_
=
1
2
n
, x
0
= 1
_
, n = 0, 1, 2, . . . .
On the other hand,
|x
n
s| = |g
2
(x
n1
) g
2
(s)| =

1
s

1
x
n1

=
|s x
n1
|
|s||x
n1
|
K|x
n1
s|
with K
1
2
because from the crude analysis of the considered function f(x) = x
2
3x +1 we
know that s > 2 (indeed, f(2) = 1 < 0 and f(3) = 2 > 0) and x
n
> 2. Proceeding in a
similar manner, we obtain
|x
n
s| K|x
n1
s| = K|g
2
(x
n2
) g
2
(s)| = K
2
|x
n2
s| . . . K
n
|x
0
s|.
Since K < 1, we have K
n
0 and |x
n
s| 0 as n .
The same proof can be obtained if we apply on each step above the mean value theorem of
calculus, according to which
g
2
(x) g
2
(s) = g

2
(t)(x s) =
1
t
2
(x s), t between x and s
8
and take into account that we seek for roots in the domain t k > 1 where
g

2
(t) =
1
t
2


K < 1.
The same proof is valid also in a rather general case.
Let us formulate the general statement concerning the convergence of xed-point iterations.
Let x = s be a solution of x = g(x) and suppose that g(x) has a continuous derivative in
some interval J containing s. Then if |g

(x)| K < 1 in J, the iteration process dened by

x
n+1
= g(x
n
) converges for any x
0
in J.
In the above-considered case
g(x) = g
2
(x) = 3
1
x
and one may take as J an interval x : k
1
< x < k
2
for any k
1
and k
2
such that k
1
> 1 and
k
2
> k
1
because in every such interval
g

(x) =
1
x
2
K =
1
k
2
1
< 1.
2.3 Newtons method
Consider the graph of a function y = f(x). The root a occurs where the graph crosses the
x-axis. We will usually have an estimate of a (which means that we have some preliminary
information about the position of a root), and it will be denoted by x
0
. To improve on this
estimate, consider the straight line that is tangent to the graph at the point (x
0
, f(x
0
)). If x
0
is
near a, this tangent line should be nearly coincident with the graph of y = f(x) for points
x about a. Then the root of the tangent line should nearly equal a. This root is denoted here
by x
1
.
To nd a formula for x
1
, consider the slope of the tangent line. Using the derivative f

(x) we
know from calculus that the slope of the tangent line at (x
0
, f(x
0
)) is f

(x
0
). We can also
calculate the slope using the fact that the tangent line contains the two points (x
0
, f(x
0
)) and
(x
1
, 0). This leads to the slope being equal to
f(x
0
) 0
x
0
x
1
,
namely, the dierence in the y-coordinates divided by the dierence in the x-coordinates. Equat-
ing the two dierent formulas for the slope, we obtain
f

(x
0
) =
f(x
0
)
x
0
x
1
.
This can be solved to give
x
1
= x
0

f(x
0
)
f

(x
0
)
.
9
Since x
1
should be an improvement over x
0
as an estimate of a, this entire procedure can be
repeated with x
1
as the initial approximation. This leads to the new estimate, or, in other
words, to the new step of the algorithm
x
2
= x
1

f(x
1
)
f

(x
1
)
.
Repeating this process, we obtain a sequence of numbers x
1
, x
2
, x
3
, . . . that we hope approach
the root a. These numbers may be called iterates, and they are dened by the following general
iteration formula
x
n+1
= x
n

f(x
n
)
f

(x
n
)
, n = 0, 1, 2, . . . (12)
which denes the algorithm of Newtons method for solving the equation f(x) = 0.
Note that actually Newtons method is a particular case of xed-point iterations
x
n+1
= g(x
n
), n = 0, 1, 2, . . .
with
g(x
n
) = x
n

f(x
n
)
f

(x
n
)
. (13)
Example 2.2
Let us solve the equation
f(x) = x
6
x + 1, f

(x) = 6x
5
1.
The iteration of Newtons method is given by
x
n+1
= x
n

x
6
n
x
n
1
6x
5
n
1
, n = 0, 1, 2, . . . . (14)
We use an initial approximation of x
0
= 1.5. The results are shown in the table below. The
column x
n
x
n1
is an estimate of the error a x
n1

x
n
f(x
n
) x
n
x
n1

1.5 88.9
1.30049088 25.40 0.200
1.18148042 0.538 0.119
1.13945559 0.0492 0.042
1.13477763 0.00055 .00468
1.13472415 0.000000068 0.0000535
1.13472414 0.0000000040 0.000000010

The true root is a = 1.134724138 and x

6
equals a to nine signicant digits. Note that the
change of sign in the last line of the table indicates the presence of root in the interval (x
5
, x
6
.
We see that Newtons method may converge slowly at rst; but as the iterates come closer
to the root, the speed of convergence increases, as shown in the table.
If to take, for example,
f
= 10
8
= 0.00000001 and
x
= 10
6
= 0.000001, the method
stops, according to the rule (2), at the step N = 6.
10
2.4 Secant method
Take Newtons method (12) and to approximate the derivative by the formula
f

(x)
f(x
n
) f(x
n1
)
x
n
x
n1
.
We get the formula for the secant method
x
n+1
= x
n
f(x
n
)
x
n
x
n1
f(x
n
) f(x
n1
)
(15)
=
x
n1
f(x
n
) x
n
f(x
n1
)
f(x
n
) f(x
n1
)
In the secant method the points are used in strict sequence. As each new point is found the
lowest numbered point in the sequence is discarded. In this method it is quite possible for the
sequence to diverge.
2.5 Numerical solution of nonlinear equations using MATLAB
It is very easy to write a MATLAB program for computing a root of the given nonlinear
equation
f(x) = 0
[function f(x) should be saved in an M-le, e.g., .m], written it in the form
x = g(x) (16)
using the iterations
x
n+1
= g(x
n
), n = 0, 1, 2, . . . .
Indeed, it is sucient to save the function g(x) in (16) in an M-le (e.g., gg.m), choose the
initial approximation x0 (according to the preliminary information about positions of roots),
set x = x0, and repeat (16) several times. In order to stop, one may use the rule (9) as follows:
calculate each time the dierence between the current and preceding values of x and stop when
this value becomes less than the given tolerance [a small number
x
in (9); one may choose,
e.g.,
x
= 10
4
] which species the accuracy of calculating the root.
In the case of Newtons method one should use formula (13) to create g(x), that is, to
calculate the derivative of the initial function f(x).
To nd a zero of a function MATLAB uses an appropriate method excecuted by the following
MATLAB command:
fzero(fcn,x0) returns one of the zeros of the function dened by the string fcn. The
initial approximation is x0. The relative error of approximation is the MATLAB predened
variable eps.
fzero(fcn,x0,tol) does the same with the relative error tol.
11
Example 2.3
Find the points(s) of intersection of the graphs of functions sinx and 2x 2, i.e., the point(s)
at which sin x = 2x 2.
To this end, create an M-le sinm.m for the function sinm(x) = sin x 2x + 2
function s = sinm(x)
s = sin(x) - 2.x + 2;
Estimate the position of the initial approximation by plotting functions:
fplot(sinm, [-10,10])
grid on; title(The sin(x) - 2.x + 2 function);
We see that x0 = 2 is a good initial approximation. Therefore, we may type
xzero = fzero(sinm,2)
to obtain
xzero =
1.4987
Problem 2.1
Solve the equation
f(x) = x
3
5.00x
2
+ 1.01x 1.88
by the xed-point iteration.
According to the method of xed-point iteration dened by x
n+1
= g(x
n
), the equation
f(x) = x
3
5.00x
2
+ 1.01x 1.88
is transformed to the approriate form
x = g(x) =
5.00x
2
1.01x 1.88
x
2
The iterations computed by x
n+1
= g(x
n
), n = 0, 1, 2, . . . , are
x
0
= 1, x
1
= 2.110, x
2
= 4.099, x
3
= 4.612, x
4
= 4.6952, x
5
= 4.700;
x
0
= 5, x
1
= 4.723, x
2
= 4.702, x
3
= 4.700.
Problem 2.2
Find Newtons iterations for the cube root and calculate 7
1/3
.
The desired value 7
1/3
is the root of the equation x
3
7 = 0.
The Newton iterations dened as
x
n+1
= G(x
n
), G(x
n
) = x
n

f(x
n
)
f

(x
n
)
, n = 0, 1, 2, . . .
12
give, for f(x) = x
3
7,
f

(x) = 3x
2
; x
n+1
=
2x
3
n
+ 7
3x
2
n
, n = 0, 1, 2, . . . .
The computed result is
x
4
= 1.912931.
Check the result using the fzero MATLAB command.
Problem 2.3
Use Newtons method to nd the root of the equation x
2
2 = 0 with the accuracy
x
= 0.001
and
x
= 0.00001. Compare the number of iterations. Draw the graphs. Check the result using
the fzero MATLAB command.
3 Interpolation
Introduce the basic terms:
interpolation polynomial;
Lagrange interpolation polynomial;
Newton interpolation polynomial;
interpolation points (nodes);
linear interpolation;
piecewise linear interpolation;
deviation;
interpolation error.
We shall be primarly concerned with the interpolation of a function of one variable:
given a function f(x) one chooses a function F(x) from among a certain class of functions
(frequently, but not always, the class of polynomials) such that F(x) agrees (coincides) with
f(x) at certain values of x. These values of x are often referred to as interpolation points, or
nodes, x = x
k
; k = 0, 1, . . ..
The actual interpolation always proceeds as follows: the function f(x) to be interpolated is
replaced by a function F(x) which
a) deviates as little as possible from f(x);
b) can be easily evaluated.
Assume that it is given a function f(x) dened in an interval a x b and its values
f(x
i
) = f
i
(ordinates) at n + 1 dierent nodes x
0
, x
1
, . . . , x
n
lying on [a, b] are known. We
seek to determine a polynomial P
n
(x) of the degree n which coincides with the given values of
f(x) at the interpolation points:
P
n
(x
i
) = f
i
(i = 0, 1, . . . n). (17)
The possibility to solve this problem is based on the following theorem:
there is exactly one polynomial P
n
(x) of degree less or equal n which satises the conditions
(1).
13
For n = 1 when P
1
(x) = Ax + B is a linear function, this statement is proved below. This
simplest case corresponds to the linear interpolation by the straight line through two points
(x
0
, f
0
) and (x
1
, f
1
).
Given a function f(x) dened in an interval a x b we seek to determine a linear function
F(x) such that
f(a) = F(a), f(b) = F(b). (18)
Since F(x) has the form
F(x) = Ax + B
for some constants A and B we have
F(a) = Aa + B, F(b) = Ab + B.
Solving for A and B we get
A =
f(b) f(a)
b a
, B =
bf(a) af(b)
b a
and, by (18),
F(x) = f(a) +
x a
b a
(f(b) f(a)). (19)
One can verify directly that for each x the point (x, F(x)) lies on the line joining (a, f(a))
and (b, f(b)).
We can rewrite (19) in the form
F(x) = w
0
(x)f(a) + w
1
(x)f(b),
where the weights
w
0
(x) =
b x
b a
, w
1
(x) =
x a
b a
.
If x lies in the interval a x b, then the weights are nonnegative and
w
0
(x) + w
1
(x) = 1,
hence, in this interval
0 w
i
(x) 1, i = 0, 1.
Obviously, if f(x) is a linear function, then interpolation process is exact and the function
F(x) from (19) coincides with f(x): the bold curve and the dot line on Fig. 1 between the
points (a, f(a)), (b, f(b)) coincide.
The interpolation error
(x) = f(x) F(x)
14
shows the deviation between interpolating and interpolated functions at the given point x
[a, b]. Of course,
(a) = (b) = 0
and
(x) 0, x [a, b]
if f(x) is a linear function.
In the previous example we had only two interpolation points, x = x
0
= a, x = x
1
= b.
Now let us consider the case of linear (piecewise linear) interpolation with the use of several
interpolation points spaced at equal intervals:
x
k
= a + kh; k = 0, 1, 2, . . . , M, h =
b a
M
; M = 2, 3, . . . .
For a given integer M we construct the interpolating function F(M; x) which is piecewise linear
and which agrees with f(x) at the M + 1 interpolation points.
In each subinterval [x
k
, x
k+1
] we determine F(M; x) by linear interpolation using formula
(19). Thus we have for x [x
k
, x
k+1
]
F(M; x) = f(x
k
) +
x x
k
x
k+1
x
k
(f(x
k+1
) f(x
k
)).
If we denote f
k
= f(x
k
) and use the rst forward dierence f
k
= f
k+1
f
k
we obtain
F(M; x) = f
k
+ (x x
k
)
f
k
h
, x [x
k
, x
k+1
].
One can calculate interpolation error in each subinterval and, obviously,
(x
k
) = 0, k = 0, 1, 2, . . . M.
Example 3.1 Linear Lagrange interpolation
Compute ln 9.2 from ln 9.0 = 2.1972 and ln 9.5 = 2.2513 by the linear Lagrange interpolation
and determine the error from a = ln 9.2 = 2.2192 (4D).
Solution. Given (x
0
, f
0
) and (x
1
, f
1
) we set
L
0
(x) =
x x
1
x
0
x
1
, L
1
(x) =
x x
0
x
1
x
0
,
which gives the Lagrange polynomial
p
1
(x) = L
0
(x)f
0
+ L
1
(x)f
1
=
x x
1
x
0
x
1
f
0
+
x x
0
x
1
x
0
f
1
.
In the case under consideration, x
0
= 9.0, x
1
= 9.5, f
0
= 2.1972, and f
1
= 2.2513. Calculate
L
0
(9.2) =
9.2 9.5
9.0 9.5
= 0.6, L
1
(9.2) =
9.2 9.0
9.5 9.0
= 0.4,
and get the answer
ln 9.2 a = p
1
(9.2) = L
0
(9.2)f
0
+ L
1
(9.2)f
1
= 0.6 2.1972 + 0.4 2.2513 = 2.2188.
The error is = a a = 2.2192 2.2188 = 0.0004.
15
3.1 Quadratic interpolation
This interpolation corresponds to that by polynomials of degree n = 2 when we have three
(dierent) nodes x
0
, x
1
, x
2
. The Lagrange polynomials of degree 2 have the form
l
0
(x) =
(x x
1
)(x x
2
)
(x
0
x
1
)(x
0
x
2
)
;
l
1
(x) =
(x x
0
)(x x
2
)
(x
1
x
0
)(x
1
x
2
)
; (20)
l
2
(x) =
(x x
0
)(x x
1
)
(x
2
x
0
)(x
2
x
1
)
.
Now let us form the sum
P
2
(x) = f
0
l
0
(x) + f
1
l
1
(x) + f
2
l
2
(x). (21)
The following statements hold:
1) P
2
(x) again is a polynomial of degree 2;
2) P
2
(x
j
) = f
j
, because among all terms, only l
j
(x
j
) = 0;
3) if there is another polynomial Q
2
(x) of degree 2 such that Q
2
(x
j
) = f
j
, then R
2
(x) =
P
2
(x)Q
2
(x) is also a polynomial of degree 2, which vanishes at the 3 dierent points x
0
, x
1
, x
2
,
or, in other words, the quadratic equation R
2
(x) = 0 has three dierent roots; therefore,
necessarily, R
2
(x) 0.
Hence (21), or, what is the same, the Lagrange interpolation formula yields the uniquely
determined interpolation polynomial of degree 2 corresponding to the given interpolation points
and ordinates.
Example 3.2
For x
0
= 1, x
1
= 2, x
2
= 4 the Lagrange polynomials (20) are:
l
0
(x) =
x 2
1 2
x 4
1 4
=
1
3
(x
2
6x + 8);
l
1
(x) =
x 1
2 1
x 4
2 4
=
1
2
(x
2
5x + 4);
l
2
(x) =
x 1
4 1
x 2
4 2
=
1
6
(x
2
3x + 2).
Therefore,
P
2
(x) =
f
0
3
(x
2
6x + 8)
f
1
2
(x
2
5x + 4) +
f
2
6
(x
2
3x + 2).
If to take, for example, f
0
= f
2
= 1, f
0
= 0, then this polynomial has the form
P
2
(x) =
1
2
(x
2
5x + 6) =
1
2
(x 2)(x 3)
and it coincides with the given ordinates at interpolation points
P
2
(1) = 1; P
2
(2) = 0; P
2
(4) = 1.
16
If f(x) is a quadratic function, f(x) = ax
2
+ bx + c, a = 0, then interpolation process is
exact and the function P
2
(x) from (21) will coincide with f(x).
The interpolation error
(x) = f(x) P
2
(x)
shows the deviation between interpolation polynomial and interpolated function at the given
point x [x
0
, x
2
]. Of course,
(x
0
) = (x
1
) = (x
2
) = 0
and
r(x) 0, x [x
0
, x
2
]
if f(x) is a quadratic function.
Example 3.3 Quadratic Lagrange interpolation
Compute ln 9.2 from ln 9.0 = 2.1972, ln 9.5 = 2.2513, and ln 11.0 = 2.3979 by the quadratic
Lagrange interpolation and determine the error from a = ln 9.2 = 2.2192 (4D).
Solution. Given (x
0
, f
0
), (x
1
, f
1
), and (x
2
, f
2
) we set
L
0
(x) =
l
0
(x)
l
0
(x
0
)
=
(x x
1
)(x x
2
)
(x
0
x
1
)(x
0
x
2
)
,
L
1
(x) =
l
1
(x)
l
1
(x
1
)
=
(x x
0
)(x x
2
)
(x
1
x
0
)(x
1
x
2
)
,
L
2
(x) =
l
2
(x)
l
2
(x
2
)
=
(x x
0
)(x x
1
)
(x
2
x
0
)(x
2
x
1
)
,
which gives the quadratic Lagrange polynomial
p
2
(x) = L
0
(x)f
0
+ L
1
(x)f
1
+ L
2
(x)f
2
.
In the case under consideration, x
0
= 9.0, x
1
= 9.5, x
2
= 11.0 and f
0
= 2.1972, f
1
= 2.2513,
f
2
= 2.3979. Calculate
L
0
(x) =
(x 9.5)(x 11.0)
(9.0 9.5)(9.0 11.0)
= x
2
20.5x + 104.5, L
0
(9.2) = 0.5400;
L
1
(x) =
(x 9.0)(x 11.0)
(9.5 9.0)(9.5 11.0)
=
1
0.75
(x
2
20x + 99), L
1
(9.2) = 0.4800;
L
2
(x) =
(x 9.0)(x 9.5)
(11.0 9.0)(11.0 9.5)
=
1
3
(x
2
18.5x + 85.5), L
2
(9.2) = 0.0200
and get the answer
ln 9.2 p
2
(9.2) = L
0
(9.2)f
0
+ L
1
(9.2)f
1
+ L
2
(9.2)f
2
=
0.5400 2.1972 + 0.4800 2.2513 0.0200 2.3979 = 2.2192,
which is exact to 4D.
17
The Lagrange polynomials of degree n = 2, 3 . . . are
l
0
(x) = w
0
1
(x)w
0
2
(x) . . . w
0
n
(x);
l
k
(x) = w
k
0
(x)w
k
1
(x) . . . w
k
k1
(x)w
k
k+1
. . . w
k
n
(x), k = 1, 2 . . . , n 1;
l
n
(x) = w
n
0
(x)w
n
1
(x) . . . w
n
n1
(x),
where
w
k
j
(x) =
x x
j
x
k
x
j
; k = 0, 1, . . . n, j = 0, 1, . . . n, k = j.
Furthermore,
l
k
(x
k
) = 1,
l
k
(x
j
) = 0, j = k.
The general Lagrange interpolation polynomial is
P
n
(x) = f
0
l
0
(x) + f
1
l
1
(x) + . . . + f
n1
l
n1
(x) + f
n
l
n
(x),
n = 1, 2, . . . , (22)
and it uniquely determines the interpolation polynomial of degree n corresponding to the given
interpolation points and ordinates.
Error estimate is given by

n
(x) = f(x) p
n
(x) = (x x
0
)(x x
1
) . . . (x x
n
)
f
n+1
(t)
(n + 1)!
,
n = 1, 2, . . . , t (x
0
, x
n
)
if f(x) has a continuous (n + 1)st derivative.
Example 3.3 Error estimate of linear interpolation
Solution. Given (x
0
, f
0
) and (x
1
, f
1
) we set
L
0
(x) =
x x
1
x
0
x
1
, L
1
(x) =
x x
0
x
1
x
0
,
which gives the Lagrange polynomial
p
1
(x) = L
0
(x)f
0
+ L
1
(x)f
1
=
x x
1
x
0
x
1
f
0
+
x x
0
x
1
x
0
f
1
.
In the case under consideration, x
0
= 9.0, x
1
= 9.5, f
0
= 2.1972, and f
1
= 2.2513. and
ln 9.2 a = p
1
(9.2) = L
0
(9.2)f
0
+ L
1
(9.2)f
1
= 0.6 2.1972 + 0.4 2.2513 = 2.2188.
The error is = a a = 2.2192 2.2188 = 0.0004.
Estimate the error according to the general formula with n = 1

1
(x) = f(x) p
1
(x) = (x x
0
)(x x
1
)
f

(t)
2
, t (9.0, 9.5)
18
with f(t) = ln t, f

(t) = 1/t, f

(t) = 1/t
2
. Hence

1
(x) = (x 9.0)(x 9.5)
(1)
t
2
,
1
(9.2) = (0.2)(0.3)
(1)
2t
2
=
0.03
t
2
(t (9.0, 9.5)),
0.00033 =
0.03
9.5
2
= min
t[9.0,9.5]

0.03
t
2

|
1
(9.2)| max
t[9.0,9.5]

0.03
t
2

=
0.03
9.0
2
= 0.00037
so that 0.00033 |
1
(9.2)| 0.00037, which disagrees with the obtained error = a a =
0.0004. In fact, repetition of computations with 5D instead of 4D gives
ln 9.2 a = p
1
(9.2) = 0.6 2.19722 + 0.4 2.25129 = 2.21885.
with an actual error
= 2.21920 2.21885 = 0.00035
which lies in between 0.00033 and 0.00037. A discrepancy between 0.0004 and 0.00035 is thus
caused by the round-o-to-4D error which is not taken into acount in the general formula for
the interpolation error.
3.2 Estimation by the error principle
First we calculate
p
1
(9.2) = 2.21885
and then
p
2
(9.2) = 0.54 2.1972 + 0.48 2.2513 0.02 2.3979 = 2.21916
from Example 2.3 but with 5D. The dierence
p
2
(9.2) p
1
(9.2) = 2.21916 2.21885 = 0.00031
is the approximate error of p
1
(9.2): 0.00031 is an approximation of the error 0.00035 obtained
above.
The Lagrange interpolation formula (22) is very unconvenient for actual calculation. More-
over, when computing with polynomials P
n
(x) for varying n, the calculation of P
n
(x) for a
particular n is of little use in calculating a value with a larger n. These problems are avoided
by using another formula for P
n
(x), employing the divided dierences of the data being
interpolated.
3.3 Divided dierences
Assume that it is given the grid of points x
0
, x
1
, x
2
, . . . ; x
i
= x
j
, i = j and corresponding values
of a function f(x) : f
0
, f
1
, f
2
, . . .
The rst divided dierences are dened as
f[x
0
, x
1
] =
f
1
f
0
x
1
x
0
; f[x
1
, x
2
] =
f
2
f
1
x
2
x
1
; . . .
19
The second divided dierences are
f[x
0
, x
1
, x
2
] =
f[x
1
, x
2
] f[x
0
, x
1
]
x
2
x
0
; (23)
f[x
1
, x
2
, x
3
] =
f[x
2
, x
3
] f[x
1
, x
2
]
x
3
x
1
; . . .
and of order n
f[x
0
, x
1
, . . . , x
n
, x
n+1
] =
f[x
1
, x
2
, . . . , x
n+1
] f[x
0
, x
1
, . . . x
n
]
x
n+1
x
0
.
It is easy to see that the order of x
0
, x
1
, x
2
, . . . , x
n
will not make a dierence in the calculation
of divided dierence. In other words, any permutation of the grid points does not change the
value of divided dierence. Indeed, for n = 1
f[x
1
, x
0
] =
f
0
f
1
x
0
x
1
=
f
1
f
0
x
1
x
0
= f[x
0
, x
1
].
For n = 2, we obtain
f[x
0
, x
1
, x
2
] =
f
0
(x
0
x
1
)(x
0
x
2
)
+
f
1
(x
1
x
0
)(x
1
x
2
)
+
f
2
(x
2
x
0
)(x
2
x
1
)
.
If we interchange values of x
0
, x
1
and x
2
, then the fractions of the right-hand side will inter-
change their order, but the sum will remain the same.
When the grid points (nodes) are spaced at equal intervals (forming the uniform grid),
the divided dierence are coupled with the forward dierences by simple formulas. Set
x
j
= x
0
+ jh, j = 0, 1, 2, . . . and assume that f
j
= f(x
0
+ jh) are given. Then the rst
forward dierence is
f[x
0
, x
1
] = f[x
0
, x
0
+ h] =
f(x
0
+ h) f(x
0
)
x
0
+ h x
0
=
f
1
f
0
h
=
f
0
1!h
.
For the second forward dierence we have
f[x
0
, x
1
, x
2
] =
1
2h
_
f
1
1!h

f
0
1!y
_
=

2
f
0
2!h
2
(24)
and so on. For arbitrary n = 1, 2, . . .
f[x
0
, x
0
+ h, . . . , x
0
+ nh] =

n
f
0
n!h
n
.
It is easy to calculate divided dierences using the table of divided dierences
20
x
0
f(x
0
)
f[x
0
, x
1
]
x
1
f(x
1
) f[x
0
, x
1
, x
2
]
f[x
1
, x
2
] f[x
0
, x
1
, x
2
, x
3
]
x
2
f(x
2
) f[x
1
, x
2
, x
3
]
f[x
2
, x
3
] f[x
1
, x
2
, x
3
, x
4
] . . .
.
.
.
.
.
.
.
.
.
f[x
n1
, x
n
]
x
n
f(x
n
)
Construct the table of divided dierences for the function
f(x) =
1
1 + x
2
,
at the nodes x
k
= kh, k = 0, 1, 2, . . . , 10 with the step h = 0.1.
The values of f
k
= f(x
k
) are found using the table of the function
1
x
with (xed number
of) three decimal places. In the rst column we place the values x
k
, in the second, f
k
, and in
the third, the rst divided dierence f[x
k
, x
k+1
] =
f
k
h
, etc.:
0.0 1.000
0.100
0.1 0.990 0.900
0.280 0.167
0.2 0.962 0.850
0.450
0.3 0.917
Let P
n
(x) denote the polynomial interpolating f(x) at the nodes x
i
for i = 0, 1, 2, . . . , n.
Thus, in general, the degree of P
n
(x) is less or equal n and
P
n
(x
i
) = f(x
i
), i = 0, 1, 2, . . . n. (25)
Then
P
1
(x) = f
0
+ (x x
0
)f[x
0
, x
1
]; (26)
P
2
(x) = f
0
+ (x x
0
)f[x
0
, x
1
] + (x x
0
)(x x
1
)f[x
0
, x
1
, x
2
] (27)
.
.
.
P
n
(x) = f
0
+ (x x
0
)f[x
0
, x
1
] + . . .
+(x x
0
)(x x
1
)(x x
n1
)f[x
0
, x
1
, . . . x
n
]. (28)
This is called the Newtons divided dierence formula for the interpolation polynomial.
Note that for k 0
P
k+1
= P
k
(x) + (x x
0
) . . . (x x
k
)f[x
0
, x
1
, . . . x
k+1
].
21
Thus we can go from degree k to degree k +1 with a minimum of calculation, once the divided
dierences have been computed (for example, with the help of the table of nite dierences).
We will consider only the proof of (26) and (27). For the rst case, one can see that
P
1
(x
0
) = f
0
and
P
1
(x
1
) = f
0
+ (x
1
x
0
)
f(x
1
) f(x
0
)
x
1
x
0
= f
0
+ (f
1
f
0
) = f
1
.
Thus P
1
(x) is a required interpolation polynomial, namely, it is a linear function which satises
the interpolation conditions (25).
For (27) we have the polynomial of a degree 2
P
2
(x) = P
1
(x) + (x x
0
)(x x
1
)f[x
0
, x
1
, x
2
]
and for x
0
, x
1
P
2
(x
i
) = P
1
(x
i
) + 0 = f
i
, i = 0, 1.
Also,
P
2
(x
2
) = f
0
+ (x
2
x
0
)f[x
0
, x
1
] + (x
2
x
0
)(x
2
x
1
)f[x
0
, x
1
, x
2
]
= f
0
+ (x
2
x
0
)f[x
0
, x
1
] + (x
2
x
1
)(f[x
1
, x
2
] f[x
0
, x
1
])
= f
0
+ (x
1
x
0
)f[x
0
, x
1
] + (x
2
x
1
)f[x
1
, x
2
]
= f
0
+ (f
1
f
0
) + (f
2
f
1
) = f
2
.
By the uniqueness of interpolation polynomial this is the quadratic interpolation polynomial
for the function f(x) at x
0
, x
1
, x
2
.
In the general case the formula for the interpolation error may be represented as follows
f(x) = P
n
(x) +
n
(x), x [x
0
, x
n
]
where x
0
, x
1
, . . . , x
n
are (dierent) interpolation points, P
n
(x) is interpolation polynomial con-
structed by any of the formulas veried above and
n
(x) is the interpolation error.
Example 3.5 Newtons divided dierence interpolation formula
Compute f(9.2) from the given values.
8.0 2.079442
0.117783
9.0 2.197225 0.006433
0.108134 0.000411
9.5 2.251292 0.005200
0.097735
11.0 2.397895
We have
f(x) p
3
(x) = 2.079442 + 0.117783(x 8.0)
0.006433(x 8.0)(x 9.0) + 0.000411(x 8.0)(x 9.0)(x 9.5).
22
At x = 9.2,
f(9.2) 2.079442 + 0.141340 0.001544 0.000030 = 2.219208.
We can see how the accuracy increases from term to term:
p
1
(9.2) = 2.220782, p
2
(9.2) = 2.219238, p
3
(9.2) = 2.219208.
Note that interpolation makes sense only in the closed interval between the rst (minimal)
and the last (maximal) interpolation points.
Newtons interpolation formula (28) becomes especially simple when interpolation points
x
0
, x
0
+ h, x
0
+ 2h, . . . are spaced at equal intervals. In this case one can rewrite Newtons
interpolation polynomial using the formulas for divided dierences and introducing the variable
r =
x x
0
h
such that
x = x
0
+ rh, x x
0
= rh, (x x
0
)(x x
0
h) = r(r 1)h
2
, . . .
Substituting this new variable into (28) we obtain the Newtons forward dierence inter-
polation formula
f(x) P
n
(x) = f
0
+ rf
0
+
r(r 1)
2!

2
f
0
+ . . . +
r(r 1) . . . (r n + 1)
n!

n
f
0
.
Error estimate is given by

n
(x) = f(x) p
n
(x) =
h
n+1
(n + 1)!
r(r 1) . . . (r n)f
n+1
(t),
n = 1, 2, . . . , t (x
0
, x
n
)
if f(x) has a continuous (n + 1)st derivative.
Example 3.6 Newtons forward dierence formula. Error estimation
Compute cosh(0.56) from the given values and estimate the error.
Solution. Construct the table of forward dierences
0.5 1.127626
0.057839
0.6 1.185645 0.011865
0.069704 0.000697
0.7 1.255169 0.012562
0.082266
0.8 1.337435
23
We have
x = 0.56, x
0
= 0.50, h = 0.1, r =
x x
0
h
=
0.56 0.50
0.1
= 0.6
and
cosh(0.56) p
3
(0.56) = 1.127626+0.60.057839+
0.6(0.4)
2
0.011865+
0.6(0.4)(1.4)
6
0.000697 =
1.127626 + 0.034703 0.001424 + 0.000039 = 1.160944.
Error estimate. We have f(t) = cosh(t) with f
(4)
(t) = cosh
(4)
(t) = cosh(t), n = 3,
h = 0.1, and r = 0.6, so that

3
(0.56) = cosh(0.56) p
3
(0.56) =
(0.1)
4
(4)!
0.6(0.6 1)(0.6 2)(0.6 3) cosh
(4)
(t) = Acosh(t),
t (0.5, 0.8)
where A = 0.0000036 and
Acosh 0.8
3
(0.56) Acosh 0.5
so that
p
3
(0.56) + Acosh 0.8 cosh(0.56) p
3
(0.56) + Acosh 0.5.
Numerical values
1.160939 cosh(0.56) 1.160941
Problem 3.1 (see Problem 17.3.1, AEM)
Compute ln 9.3 from ln 9.0 = 2.1972 and ln 9.5 = 2.2513 by the linear Lagrange interpolation
and determine the error from a = ln 9.3 = 2.2300 (exact to 4D).
Solution. Given (x
0
, f
0
) and (x
1
, f
1
) we set
L
0
(x) =
x x
1
x
0
x
1
, L
1
(x) =
x x
0
x
1
x
0
,
which gives the Lagrange polynomial
p
1
(x) = L
0
(x)f
0
+ L
1
(x)f
1
=
x x
1
x
0
x
1
f
0
+
x x
0
x
1
x
0
f
1
.
In the case under consideration, x
0
= 9.0, x
1
= 9.5, f
0
= 2.1972, and f
1
= 2.2513.
L
0
(x) =
x 9.5
(0.5)
= 2(9.5 x) = 19 2x, L
1
(x) =
x 9.0
0.5
= 2(x 9) = 2x 18.
The Lagrange polynomial is
p
1
(x) = L
0
(x)f
0
+ L
1
(x)f
1
=
(192x)2.1972+(2x18)2.2513 = 2x(2.25132.1972)+192.1972182.2513 = 0.1082x+1.2234.
Now calculate
L
0
(9.3) =
9.3 9.5
9.0 9.5
= 0.4, L
1
(9.3) =
9.3 9.0
9.5 9.0
= 0.6,
24
and get the answer
ln 9.3 a = p
1
(9.3) = L
0
(9.3)f
0
+ L
1
(9.3)f
1
= 0.4 2.1972 + 0.6 2.2513 = 2.2297.
The error is = a a = 2.2300 2.2297 = 0.0003.
Problem 3.2 (see Problem 17.3.2, AEM)
Estimate the error of calculating ln 9.3 from ln 9.0 = 2.1972 and ln 9.5 = 2.2513 by the linear
Lagrange interpolation (ln 9.3 = 2.2300 exact to 4D).
Solution. We estimate the error according to the general formula with n = 1

1
(x) = f(x) p
1
(x) = (x x
0
)(x x
1
)
f

(t)
2
, t (9.0, 9.5)
with f(t) = ln t, f

(t) = 1/t, f

(t) = 1/t
2
. Hence

1
(x) = (x 9.0)(x 9.5)
(1)
t
2
,
1
(9.3) = (0.3)(0.2)
(1)
2t
2
=
0.03
t
2
(t (9.0, 9.5)),
0.00033 =
0.03
9.5
2
= min
t[9.0,9.5]

0.03
t
2

|
1
(9.3)| max
t[9.0,9.5]

0.03
t
2

=
0.03
9.0
2
= 0.00037
so that 0.00033 |
1
(9.3)| 0.00037, which disagrees with the obtained error = a a = 0.0003
because in the 4D computations we cannot round-o the last digit 3. In fact, repetition of
computations with 5D instead of 4D gives
ln 9.3 a = p
1
(9.3) = 0.4 2.19722 + 0.6 2.25129 = 0.87889 + 1.35077 = 2.22966
with an actual error
= 2.23001 2.22966 = 0.00035
which lies between 0.00033 and 0.00037. A discrepancy between 0.0003 and 0.00035 is thus
caused by the round-o-to-4D error which is not taken into acount in the general formula for
the interpolation error.
Problem 3.3 (optional; see Problem 17.3.3, AEM)
Compute e
0.25
and e
0.75
by linear interpolation with x
0
= 0, x
1
= 0.5 and x
0
= 0.5, x
1
= 1. Then
nd p
2
(x) interpolating e
x
with x
0
= 0, x
1
= 0.5, and x
2
= 1 and from it e
0.25
and e
0.75
. Compare
the errors of these linear and quadratic interpolation.
Solution. Given (x
0
, f
0
) and (x
1
, f
1
) we set
L
0
(x) =
x x
1
x
0
x
1
, L
1
(x) =
x x
0
x
1
x
0
,
which gives the Lagrange polynomial
p
1
(x) = L
0
(x)f
0
+ L
1
(x)f
1
=
x x
1
x
0
x
1
f
0
+
x x
0
x
1
x
0
f
1
.
25
In the case of linear interpolation, we will interpolate e
x
and take rst the nodes x
0
= 0.5 and
x
1
= 0 and f
0
= e
0.5
= 0.6065 and f
1
= e
0
= 1.0000.
L
0
(x) =
x 0
(0.5)
= 2x, L
1
(x) =
x + 0.5
(0.5)
= 2(x + 0.5) = 2x + 1.
The Lagrange polynomial is
p
1
(x) = L
0
(x)f
0
+ L
1
(x)f
1
=
2x 0.6065 + (2x + 1)1.0000 = 2x(1.0000 0.6065) + 1 = 2 0.3935x + 1.
The answer
e
0.25
p
1
(0.25) = 0.25 2 0.3935 + 1 = 1 0.1967 = 0.8033.
The error is = e
0.25
p
1
(0.25) = 0.7788 0.8033 = 0.0245.
Now take the nodes x
0
= 1 and x
1
= 0.5 and f
0
= e
1
= 0.3679 and f
1
= e
0.5
= 0.6065.
L
0
(x) =
x + 0.5
(0.5)
= 2(x + 0.5) = 2x 1, L
1
(x) =
x + 1
(0.5)
= 2(x + 1).
The Lagrange polynomial is
p
1
(x) = L
0
(x)f
0
+ L
1
(x)f
1
=
(2x1) 0.3679+(2x+2)0.6065 = 2x(0.60650.3679) 0.3679+2 0.6065 = 2 0.2386x+0.8451.
The answer
e
0.75
p
1
(0.75) = 0.75 2 0.2386 + 0.8451 = 0.3579 + 0.8451 = 0.4872.
The error is = e
0.75
p
1
(0.75) = 0.4724 0.4872 = 0.0148.
In the case of quadratic interpolation, we will again interpolate e
x
and take the nodes x
0
= 1,
x
1
= 0.5, and x
2
= 0 and f
0
= e
1
= 0.3679, f
1
= e
0.5
= 0.6065 and f
2
= 1.0000.
Given (x
0
, f
0
), (x
1
, f
1
), and (x
2
, f
2
) we set
L
0
(x) =
l
0
(x)
l
0
(x
0
)
=
(x x
1
)(x x
2
)
(x
0
x
1
)(x
0
x
2
)
,
L
1
(x) =
l
1
(x)
l
1
(x
1
)
=
(x x
0
)(x x
2
)
(x
1
x
0
)(x
1
x
2
)
,
L
2
(x) =
l
2
(x)
l
2
(x
2
)
=
(x x
0
)(x x
1
)
(x
2
x
0
)(x
2
x
1
)
,
which gives the quadratic Lagrange polynomial
p
2
(x) = L
0
(x)f
0
+ L
1
(x)f
1
+ L
2
(x)f
2
.
In the case under consideration, calculate
L
0
(x) =
(x + 0.5)(x)
(0.5)(1)
= 2x(x + 0.5);
L
0
(0.25) = 0.5 0.25 = 0.125, L
0
(0.75) = (1.5) (0.25) = 0.375.
L
1
(x) =
(x + 1)(x)
(0.5)(0.5)
= 4x(x + 1);
L
1
(0.25) = 1 0.75 = 0.75, L
1
(0.75) = 3 0.25 = 0.75.
26
L
2
(x) =
(x + 1)(x + 0.5)
(1)(0.5)
= 2(x + 0.5)(x + 1);
L
2
(0.25) = 0.5 0.75 = 0.375, L
2
(0.75) = (0.5) 0.25 = 0.125.
The answers are as follows:
e
0.25
p
2
(0.25) = L
0
(0.25)f
0
+ L
1
(0.25)f
1
+ L
2
(0.25)f
2
=
0.1250 0.3679 + 0.7500 0.6065 + 0.3750 1.0000 = 0.0460 + 0.4549 + 0.3750 = 0.7839.
The error is = e
0.25
p
2
(0.25) = 0.7788 0.7839 = 0.0051..
e
0.75
p
2
(0.75) = L
0
(0.75)f
0
+ L
1
(0.75)f
1
+ L
2
(0.75)f
2
=
0.3750 0.3679 + 0.7500 0.6065 0.1250 1.0000 = 0.1380 + 0.4549 0.1250 = 0.4679.
The error is = e
0.75
p
2
(0.75) = 0.4724 0.4679 = 0.0045..
The quadratic Lagrange polynomial is
p
2
(x) = L
0
(x)f
0
+ L
1
(x)f
1
+ L
2
(x)f
2
=
2x(x + 0.5) 0.3679 4x(x + 1) 0.6065 + 2(x + 0.5)(x + 1) 1.0000 = 0.3095x
2
0.9418x + 1.
Problem 3.4 (quadratic interpolation) (see Problem 17.3.5, AEM)
Calculate the Lagrange polynomial p
2
(x) for 4-D values of the Gamma-function
(x) =
_

0
e
t
t
x1
dt,
(1.00) = 1.0000, (1.02) = 0.9888, and (1.04) = 0.9784, and from it the approximation of
(1.01) and (1.03).
Solution. We will interpolate (x) taking the nodes x
0
= 1.00, x
1
= 1.02, and x
2
= 1.04 and
f
0
= 1.0000, f
1
= 0.9888, and f
2
= 0.9784.
Given (x
0
, f
0
), (x
1
, f
1
), and (x
2
, f
2
) we set
L
0
(x) =
l
0
(x)
l
0
(x
0
)
=
(x x
1
)(x x
2
)
(x
0
x
1
)(x
0
x
2
)
,
L
1
(x) =
l
1
(x)
l
1
(x
1
)
=
(x x
0
)(x x
2
)
(x
1
x
0
)(x
1
x
2
)
,
L
2
(x) =
l
2
(x)
l
2
(x
2
)
=
(x x
0
)(x x
1
)
(x
2
x
0
)(x
2
x
1
)
,
which gives the quadratic Lagrange polynomial
p
2
(x) = L
0
(x)f
0
+ L
1
(x)f
1
+ L
2
(x)f
2
.
In the case under consideration, calculate
L
0
(x) =
(x 1.02)(x 1.04)
(0.02)(0.04)
= 1250(x 1.02)(x 1.04);
27
L
0
(1.01) =
(0.01)(0.03)
(0.02)(0.04)
=
3
8
= 0.375, L
0
(1.03) =
(0.01)(0.01)
(0.02)(0.04)
=
1
8
= 0.125.
L
1
(x) =
(x 1)(x 1.04)
(0.02)(0.02)
= 2500(x 1)(x 1.04);
L
1
(1.01) =
(0.01)(0.03)
(0.02)(0.02)
=
3
4
= 0.75, L
1
(1.03) =
(0.03)(0.01)
(0.02)(0.02)
=
3
4
= 0.75.
L
2
(x) =
(x 1)(x 1.02)
(0.04)(0.02)
= 1250(x 1)(x 1.02);
L
2
(1.01) =
(0.01)(0.01)
(0.04)(0.02)
=
1
8
= 0.125, L
1
(1.03) =
(0.03)(0.01)
(0.04)(0.02)
=
3
8
= 0.375.
The answers are as follows:
(1.01) p
2
(1.01) = L
0
(1.01)f
0
+ L
1
(1.01)f
1
+ L
2
(1.01)f
2
=
0.3750 1.0000 + 0.7500 0.9888 0.1250 0.9784 = 0.3750 + 0.7416 0.1223 = 0.9943.
The error is = (1.01) p
2
(1.01) = 0.9943 0.9943 = 0.0000. The result is exact to 4D.
(1.03) p
2
(1.03) = L
0
(1.03)f
0
+ L
1
(1.03)f
1
+ L
2
(1.03)f
2
=
0.1250 1.0000 + 0.7500 0.9888 + 0.3750 0.9784 = 0.1250 + 0.7416 + 0.3669 = 0.9835.
The error is = (1.03) p
2
(1.03) = 0.9835 0.9835 = 0.0000. The result is exact to 4D.
The quadratic Lagrange polynomial is
p
2
(x) = L
0
(x)f
0
+ L
1
(x)f
1
+ L
2
(x)f
2
=
1250(x1.02)(x1.04)1.00002500(x1)(x1.04)0.9888+1250(x1)(x1.02)0.9784 =
= x
2
(1250(1.9784 2 0.9888)) + . . . = x
2
(1250((2 0.0216) 2(1 0.0112))) + . . . =
x
2
(1250(0.0216+20.0112))+. . . = x
2
(12500.0008)+. . . = x
2
1.000+. . . = x
2
2.580x+2.580.
Problem 3.5 (Newtons forward dierence formula) (see Problem 17.3.11, AEM)
Compute (1.01), (1.03), and (1.05) by Newtons forward dierence formula.
Solution. Construct the table of forward dierences
1.00 1.0000
0.0112
1.02 0.9888 0.0008
0.0104
1.04 0.9784
From Newtons forward dierence formula, we have
x = 1.01, 1.03, 1.05; x
0
= 1.00, h = 0.02, r =
x 1
h
= 50(x 1);
p
2
(x) = f
0
+ rf
0
+
r(r 1)
2

2
f
0
= 1.000 0.0112r + 0.0008
r(r 1)
2
= x
2
2.580x + 2.580,
28
which coincides with the quadratic interpolation polynomial derived above. Therefore, we can
perform direct calculations to obtain
(1.01) p
2
(1.01) = 0.9943, (1.03) p
2
(1.03) = 0.9835, (1.05) p
2
(1.05) = 0.9735.
Problem 3.6 (lower degree) (see Problem 17.3.13, AEM)
What is the degree of the interpolation polynomial for the data (1,5), (2,18), (3,37), (4,62),
(5,93)?
Solution. We nd the polynomial proceeding from an assumption that it has the lowest
possible degree n = 2 and has the form p
2
(x) = ax
2
+ bx + c. In fact, it is easy to check that
the required polynomial is not a linear function, Ax + B, because
A + B = 5
2A + B = 18
yields A = 13 and B = 8, and
3A + B = 37
4A + B = 62
yields A = 25 and B = 38.
For the determination of the coecients a, b, c we have the system of three linear algebraic
equations
a + b + c = 5,
4a + 2b + c = 18
9a + 3b + c = 37
Subtracting the rst equation from the second and from the third, we obtain
3a + b = 13
8a + 2b = 32
or
6a + 2b = 26
8a + 2b = 32
which yields 2a = 6, a = 3, b = 13 3a = 4, and c = 5 b a = 2.
Thus, the desired polynomial of degree n = 2 is p
2
(x) = 3x
2
+ 4x 2.
It is easy to see that p
2
(1) = 5, p
2
(2) = 18, p
2
(3) = 37, p
2
(4) = 62, and p
2
(5) = 93.
4 Numerical Integration and Dierentiation
Numerical integration means numerical evaluation of integrals
J =
_
b
a
f(x)dx,
where a abd b are given numbers and f(x) is a function given analytically by a formula or
empirically by a table of values.
29
4.1 Rectangular rule. Trapezoidal rule
Subdivide the interval of integration a x b into n subintervals of equal length h = (ba)/n
and in each subinterval approximate f(x) by the constant f(x

j
), the value of f(x) at the mid-
point x

j
of the jth subinterval. The f(x) is approximated by a step function (piecewise constant
function). The n rectangles have the areas f(x

1
), f(x

2
), . . . , f(x

n
), and the rectangular rule
is
J =
_
b
a
f(x)dx h[f(x

1
) + f(x

2
) + . . . + f(x

n
)], h = (b a)/n. (29)
The trapezoidal rule is generally more accurate. Take the same subdivision of a x b
into n subintervals of equal length and approximate f(x) by a broken line of segments (chords)
with endpoints [a, f(a)], [x
1
, f(x
1
)], [x
2
, f(x
2
)], . . . , [x
n
, f(x
n
)] on the graph of f(x). The the
area under the graph of f(x) is approximated by n trapezoids of areas
1
2
[f(a) + f(x
1
)],
1
2
[f(x
1
) + f(x
2
)], . . . ,
1
2
[f(x
n1
) + f(b)].
Calculating the sum we obtain the trapezoidal rule
J =
_
b
a
f(x)dx h[
1
2
f(a) + f(x
1
) + f(x
2
) + . . . + f(x
n1
) +
1
2
f(b)], h = (b a)/n. (30)
4.2 Error bounds and estimate for the trapezoidal rule
Recall that the error
n
(x) of the interpolation by the Lagrange polymonial p
n
(x) of degree n
is given by

n
(x) = f(x) p
n
(x) = (x x
0
)(x x
1
) . . . (x x
n
)
f
n+1
(t)
(n + 1)!
,
n = 1, 2, . . . , t (x
0
, x
n
)
if f(x) has a continuous (n + 1)st derivative. The error estimate of the linear interpolation is,
respectively, (for n = 1) is

1
(x) = f(x) p
1
(x) = (x x
0
)(x x
1
)
f

(t)
2
, t [a, b] (31)
We will use this formula to determine the error bounds and estimate for the trapezoidal rule:
integrate it over x from a = x
0
to x
1
= x
0
+ h to obtain
J =
_
x
0
+h
x
0
f(x)dx
h
2
[f(x
0
) + f(x
1
)] =
_
x
0
+h
x
0
(x x
0
)(x x
0
h)
f

(t(x))
2
dx.
Setting xx
0
= v and applying the mean value theorem, which we can use because (xx
0
)(x
x
0
h) does not change sign, we nd that the right-hand side equals
_
h
0
(v)(v h)dv
f

t)
2
=
h
3
12
f

t),

t [x
0
, x
1
]. (32)
which gives the formula for the local error of the trapezoidal rule.
30
Find the total error of the trapezoidal rule with any n. We have
h = (b a)/n, nh
3
= n(b a)
3
/n
3
, (b a)
2
= n
2
h
2
;
therefore
=
_
b
a
f(x)dx h[
1
2
f(a) + f(x
1
) + f(x
2
) + . . . + f(x
n1
) +
1
2
f(b)] =

(b a)
3
12n
2
f

() = h
2
(b a)
12
f

(), [a, b].

Error bounds for the trapezoidal rule
KM
2
KM

2
,
where
K =
(b a)
3
12n
2
= h
2
(b a)
12
and
M
2
= max
x[a,b]
f

(x), M

2
= min
x[a,b]
f

(x).
Example 4.1 The error for the trapezoidal rule
Estimate the error of evaluation of
J =
_
1
0
e
x
2
dx 0.746211 (n = 10)
by the trapezoidal rule.
Solution. We have
(e
x
2
)

= 2(2x
2
1))e
x
2
, (e
x
2
)

= 8xe
x
2
(3/2 x
2
) > 0, 0 < x < 1,
Therefore,
max
x[0,1]
f

(x) = f

(1) = M
2
= 0.735759, min
x[0,1]
f

(x) = f

(0) = M

2
= 2.
Furthermore,
K =
(b a)
3
12n
2
=
1
12 10
2
=
1
1200
and
KM
2
= 0.000614, KM

2
= 0.001667,
so that the error
0.000614 0.001667, (33)
and the estimation is
0.746211 0.000614 = 0.745597 J 0.746211 + 0.001667 = 0.747878. (34)
The 6D-exact value is J = 0.746824 : 0.745597 < 0.746824 < 0.747878.
31
4.3 Numerical integration using MATLAB
The MATLAB command
trapz(x,y) computes the integral of y as a function of of x. Vectors x and y have the same
length and (x
i
, y
i
) represents a point on the curve.
Example 4.2
Evaluate
J =
_
1
0
e
x
2
dx
by the trapezoidal rule.
Using trapz command we rst create vector x. We try this vector with 5 and 10 values:
x5 = linspace(0,1,5);
x10 = linspace(0,1,10);
Then we create vector y as a function of x:
y5 = exp(x5.x5);
y10 = exp(x10.x10);
Now the integral can be computed:
integral5 = trapz(x5,y5), integral10 = trapz(x10,y10)
returns
integral5 =
0.74298409780038
integral10 =
0.74606686791267
4.4 Numerical dierentiation
Since, by denition of the derivative of a function f(x) at a point x,
f

(x) = lim
h0
f(x + h) f(x)
h
,
we may write the approximate formulas for the rst and second derivatives
f

(x) f
1
=
f
1
f
0
h
,
f

(x)
f
2
f
1
h
=
f
2
2f
1
+ f
0
h
2
,
etc., where f
0
= f(x) and f
1
= f(x + h).
32
If we have an equispaced grid {x
i
}
n
i=1
with the spacing h = x
i+1
x
i
(i = 1, 2, . . . , n 1,
n 2) then the corresponding approximate formulas for the rst and second derivatives written
at each node x
i
= x
1
+ (i 1)h (i = 1, 2, . . . , n + 1) are
f

(x
i
)
f
i+1
f
i
h
, i = 1, 2, . . . , n 1,
f

(x
i
)
f
i+1
2f
i
+ f
i1
h
2
, i = 2, 3, . . . , n 1.
More accurate formulas are
f

(x) p

2
(x) =
2x x
1
x
2
2h
2
f
0

2x x
0
x
2
h
2
f
+
2x x
0
x
1
2h
2
f
2
.
Evaluating these ratios at x
0
, x
1
, x
2
we obtain
f

0

4f
1
3f
0
f
2
2h
,
f

1

f
2
f
0
2h
,
f

2

4f
1
+ f
0
+ 3f
2
2h
.
5 Approximation of Functions
Let us introduce the basic terms:
norm; uniform norm; L
2
-norm;
polynomial of best approximation;
to alternate; alternating;
reference;
reference set; reference deviation; reference-polynomial;
level; levelled reference-polynomial;
to minimize;
to terminate;
So far we have primarly been concerned with representing a given function f(x) by an
interpolating polynomial P
n
(x) of the degree n (which coincides with the given values of f(x)
at the interpolation points). Now we consider approximation of a function f(x) which is known
to be continuous in an interval [a, b] (we will denote it f(x) C[a, b]) by a polynomial P
n
(x) of
degree n or less which does not necessarily coincide with f(x) at any points. This approximation
polynomial should be close enough for x [a, b] to the given function. There are various
measures of the closeness of the approximation of f(x) by P
n
(x). We shall be interested in
nding a polynomial P
n
(x) which minimizes the quantity
max
axb
|P
n
(x) f(x)|,
33
which is called the uniform norm of P
n
f and is denoted by ||P
n
f|| or ||P
n
f||
C
. Another
measure which is often used is the L
2
-norm given by
||P
n
f||
2
= {
_
b
a
(P
n
(x) f(x))
2
dx}
1
2
.
We now state without proof two mathematical theorems which are related to the approxi-
mation of functions in the uniform norm.
Theorem 5.1. If f(x) C[a, b], then given any > 0 there exists a polynomial P
n
(x) such
that for x [a, b]
|P
n
(x) f(x)| .
This is the Weierstrass theorem.
Theorem 5.2. If f(x) C[a, b], for a given integer n there is a unique polynomial
n
(x) of
degree n or less such that

n
= max
axb
|
n
(x) f(x)| max
axb
|P
n
(x) f(x)|
for any polynomial P
n
(x) of degree n or less. Moreover, there exists a set of n + 2 points
x
0
, x
1
, . . . , x
n
, x
n+1
in [a, b], x
0
< x
1
< . . . < x
n
< x
n+1
, such that either

n
(x
i
) f(x
i
) = (1)
i

n
, i = 0, 1, . . . , n
or

n
(x
i
) f(x
i
) = (1)
i+1

n
, i = 0, 1, . . . , n.
Note that by Theorem 5.1 it follows that
n
0 as n .
No general nite algorithm is known for nding the polynomial
n
(x) of best approximation.
However, if we replace the interval [a, b] by a discrete set of points, a grid J =
n+2
, then in a
nite number of steps we can nd a polynomial
n
(x) of degree n or less such that
max
xJ
|
n
(x) f(x)| max
xJ
|P
n
(x) f(x)|
for any polynomial P
n
(x) of degree n or less.
We now describe an algorithm for nding
n
(x) which is based on the following statement.
Given n +2 distinct points x
0
, x
1
, . . . , x
n
, x
n+1
, x
0
< x
1
< . . . < x
n
< x
n+1
, there exist non-
vanishing constants
0
,
1
, . . . ,
n
,
n+1
of alternating sign in the sense that
0

1
,
1

2
, . . . ,
n

n+1
are negative and such that for any polynomial P
n
(x) of degree n or less we have
n+1

i=0

i
P
n
(x
i
) = 0. (35)
Moreover,

0
=
1
x
0
x
1
1
x
0
x
2
. . .
1
x
0
x
n+1
;

i
=
1
x
i
x
0
. . .
1
x
i
x
i1
1
x
i
x
i+1
. . .
1
x
i
x
n+1
, i = 1, 2, . . . n, (36)

n+1
=
1
x
n+1
x
0
1
x
n+1
x
1
. . .
1
x
n+1
x
n
.
34
Let us explain the proof considering the simplest case n = 1 when one has three points
x
0
, x
1
, x
2
, three non-zero constants

0
=
1
x
0
x
1
1
x
0
x
2
,

1
=
1
x
1
x
0
1
x
1
x
2
, (37)

2
=
1
x
2
x
0
1
x
2
x
1
,
and P
1
(x) = ax+b is a polynomial of degree 1 or less. Application of formula (35) means that
one interpolates a linear function P
1
(x) by interpolation polynomial of degree 1 or less using
interpolation points x
0
, x
1
, hence (35) yields the identities
P
1
(x) = ax + b =
1

i=0
{
1
j=i
x x
j
x
i
x
j
}P
1
(x
i
) =
x x
1
x
0
x
1
(ax
0
+ b) +
x x
0
x
1
x
0
(ax
1
+ b) ax + b.
If we substitute x = x
2
into the last identity and divide by (x
2
x
0
)(x
2
x
1
) we obtain
1
x
0
x
1
1
x
0
x
2
P
1
(x
0
) +
1
x
1
x
0
1
x
1
x
2
P
1
(x
1
) +
1
x
2
x
0
1
x
2
x
1
P
1
(x
2
) = 0.
what coincides with the required equality for the considered case.
Let us introduce the following terminology.
1. A reference set is a set (grid) J =
n+2
of n+2 distinct numbers x
0
, x
1
, . . . , x
n
, x
n+1
, x
0
<
x
1
< . . . < x
n
< x
n+1
.
2. A reference-polynomial with respect to the function f(x) dened on J (grid function) and
with respect to this reference set is a polynomial P
n
(x) of degree n or less such that the h
i
have
alternating signs, where
h
i
= P
n
(x
i
) f(x
i
) i = 0, 1, . . . , n, n + 1.
3. A levelled reference-polynomial is a reference-polynomial such that all h
i
have the same
magnitude.
Below, we will show that for a given reference set and for a given function f(x) there is a
unique levelled reference-polynomial.
4. The reference deviation of a reference set is the value H = |h
0
| = |h
1
| = . . . = |h
n+1
| for
the levelled-reference polynomial.
The next statement proves the existence and uniqueness of a levelled-reference polynomial
corresponding to a given reference set x
0
, x
1
, . . . , x
n+1
.
35
Theorem 5.3. Given a function f(x) and a reference set R : x
0
, x
1
, . . . , x
n+1
, x
0
< x
1
<
. . . < x
n+1
there exists a unique levelled-reference polynomial P
n
(x) of degree n or less. More-
over, we have
P
n
(x
i
) = f(x
i
) + (1)
i
, i = 0, 1, . . . , n + 1,
where
= sgn
0

n+1
i=0

i
f(x
i
)

n+1
i=0
|
i
|
(38)
and where the
i
are given by (36). The reference deviation is given by
H = || =
|

n+1
i=0

i
f(x
i
)|

n+1
i=0
|
i
|
.
If Q
n
(x) is any polynomial of degree n or less such that
max
xR
|Q
n
(x) f(x)| = H,
then
Q
n
(x) P
n
(x).
Suppose now that x
0
, x
1
, . . . , x
n+1
; x
0
< x
1
< . . . < x
n+1
is a reference set and that P
n
(x) is
a reference-polynomial, not necessarily a levelled reference-polynomial with respect to f(x).
Let
h
i
= P
n
(x
i
) f(x
i
), i = 0, 1, 2, . . . , n + 1.
Then one can show that the reference deviation H is represented in the form
H =
n+1

i=0
w
i
|h
i
|,
where the weights w
i
are positive and are given by
w
i
=
|
i
|

n+1
i=0
|
i
|
,
and where the
i
are given by (35).
Now we can dene the following computational procedures for constructing the polynomial

n
(x) of best approximation.
1. Choose any n + 2 points x
0
, x
1
, . . . , x
n+1
in J such that x
0
< x
1
< . . . < x
n+1
, i.e., choose
any reference set, R (R forms a (nonuniform or uniform) grid
n+2
of n + 2 dierent points).
2. Determine the levelled reference-polynomial P
n
(x) and the reference deviation H for R.
3. Find x = z in J such that
|P
n
(z) f(z)| = max
xJ
|P
n
(x) f(x)| = .
36
If = H, terminate the process.
4. Find a new reference set which contains z and is such that P
n
(x) is a reference-polynomial
with respect to the new reference set. This can be done as follows:
if z lies in an interval x
i
x x
i+1
, i = 0, 1, . . . , n, replace x
i
by z if P
n
(z) f(z) has
the same sign as P
n
(x
i
) f(x
i
). Otherwise replace x
i+1
by z. If z < x
0
then replace x
0
by
z if P
n
(x
0
) f(x
0
) has the same sign as P
n
(z) f(z). Otherwise replace x
n+1
by z. A similar
procedure is used if z > x
n+1
.
5. With the new reference set, return to step 2.
The reference deviations are strictly increasing unless, at a given stage, the process termi-
nates. For the reference deviation H

of the new reference set R

, found from R by replacing

one value of x by another for which the absolute deviation is strictly greater, is greater than
H. Since there are only a nite number of reference sets and since no reference set can occur
more than once, the process must eventually terminate.
When the process terminates, we have a reference set R

with a reference deviation H

and
a polynomial
n
(x) which is a levelled reference-polynomial for R

. If
n
(x) is any polynomial
of degree n or less, then by Theorem 5.3 either
n
(x)
n
(x) or else
n
(x) has a maximum
absolute deviation greater than H

on R

, and hence on J. Thus

n
(x) is the polynomial of
best approximation.
5.1 Best approximation by polynomials
Consider two problems of nding polynomials P
1
(x) of degree 1 or less which give the best
approximation of given continuous functions f(x) in the interval [0, 1].
Example 5.1 The function to be approximated is
f(x) = x
2
.
We will solve this problem analytically by taking
P
1
(x) = ax + b.
In line with Theorem 5.2 one must nd the polynomial
1
(x) of best approximation by mini-
mizing the quantity
r
1
(a, b) = max
0x1
|R
1
(x)|, R
1
(x) = x
2
ax b
with respect, in general, to all real numbers a, b. We will not consider such a problem in its
general statement and impose strong but natural restrictions on the set of admissible values of
a and b which obviously follow from the properties of function R
1
(x). Namely, we will assume
that a = 1, 0.25 < b < 0 so that all the lines y = P
1
(x) = x + b may lie inside the region
ABCD where AD coincides with the tangent line x 0.25 to the curve y = x
2
.
Thus in what follows we will take r
1
= r
1
(b), 0.25 < b < 0.
It is easy to see that
r
1
(b) = max
0x1
|x
2
x b| = max{|R
1
(0)|; |R
(
0.5)|; |R
1
(1)|} = max{b; |b 0.25|},
37
since x = 0, 0.5, 1 are the only possible extremal points of the function |R
1
(x)| in the interval
[0, 1] and R
1
(0) = R
1
(1) = b > 0.
r
1
(b) is a piecewise linear function. It is clear that
min
0.25<b<0
r
1
(b) = r
1
(b
0
) = 0.125; b
0
= 0.125.
Hence the desired polynomial of best approximation

1
(x) = x 0.125,
and
||
1
(x) f(x)|| = max
0x1
|x
2
x + 0.125| = 0.125.
The conditions

1
(x
i
) f(x
i
) = (1)
i+1

2
, i = 0, 1, 2;
2
= 0.125
hold at the reference set of points
x
0
= 0, x
1
= 0.5, x
2
= 1.
The same result may be obtained in the following way. We look for constants a and b such
that
max
0x1
|ax + b x
2
|
is minimized. The extreme values of the function ax+bx
2
occur at x = 0, x = 1 and x = 0.5a
(because (ax +b x
2
)

= a 2x). We will assume that a > 0 since the point x = 0.5a belongs
to the interval [0, 1] if 0 < a < 2. From Theorem 5.2 we seek to choose a and b such that
ax + b x
2
= b =
2
(x = 0);
ax + b x
2
= 0.25a
2
+ b =
2
(x = 0.25a);
ax + b x
2
= a + b 1 =
2
(x = 1);
which gives
a = 1, b = 0.125,
2
= 0.125,
and the required polynomial of best approximation
1
(x) = x 0.125.
Example 5.2 Let us now construct the polynomial of best approximation for the function
f(x) = e
x
in the interval [0, 1]. As we shall see below, this problem can be also solved analytically and to
the accuracy indicated the polynomial of best approximation is

1
(x) = 1.718282x + 0.894607
38
and
max
0x1
|
1
(x) f(x)| = 0.105933.
Let J =
101
be the set of points (the grid)
t
i
=
i
100
, i = 0, 1, 2, . . . , 100,
and let us start with the reference set
x
0
= 0, x
1
= 0.5, x
2
= 1.0.
From a table of e
x
we have
f(x
0
) = 1.0000, f(x
1
) = 1.6487, f(x
2
) = 2.7183.
Let us determine the levelled reference-polynomial P(x) and the reference deviation H.
Taking formulas (37) we have

0
=
1
x
0
x
1
1
x
0
x
2
= 2,

1
=
1
x
1
x
0
1
x
1
x
2
= 4,

2
=
1
x
2
x
0
1
x
2
x
1
= 2.
We then obtain from formula (38):
=
2(1.0000) 4(1.6487) + 2(2.7183)
2 + 4 + 2
= 0.1052.
Then the reference deviation is
H = 0.1052.
Suppose that instead of starting with the reference set J we had started with some other
reference set and that for this reference set we had the levelled reference-polynomial
Q(x) = 0.9 + 1.6x.
Since
h
0
= Q(x
0
) f(x
0
) = 0.9 1.0000 = 0.1000,
h
1
= Q(x
1
) f(x
1
) = 1.7 1.6487 = 0.0513,
h
2
= Q(x
2
) f(x
2
) = 2.5 2.7183 = 0.2183
it follows that Q(x) is a reference-levelled polynomial for the reference set J. We now verify
(37). We have
w
1
=
1
4
, w
2
=
1
2
, w
3
=
1
4
39
and the deviation
H = 0.1052 =
1
4
|h
0
| +
1
2
|h
1
| +
1
4
|h
2
| =
1
4
(0.1000) +
1
2
(0.0513) +
1
4
(0.2183).
We now determine P(x) by linear interpolation at points x
0
and x
1
with
P(x
0
) = f(x
0
) + = 0.8948, P(x
1
) = f(x
1
) = 1.7539.
Obviously,
P(x) = P(x
0
) +
x x
0
x
1
x
0
(P(x
1
) P(x
0
)) = 0.8948 + 1.7182x,
and
P(x
2
) = 2.6130 = f(x
2
) 0.1053 = f(x
2
) + .
We verify that

0
P(x
0
) +
1
P(x
1
) +
2
P(x
2
) = 2(0.8948) 4(1.7539) + 2(2.6130) = 0.
Since the P(x
i
) f(x
i
), i = 0, 1, 2 have alternating signs and equal magnitudes, it follows
that P(x) is the levelled reference-polynomial for the corresponding reference set.
By the direct computation it is found that
max
xJ
|P(x) f(x)| = |P(z) f(z)| = 0.106656, z = 0.54.
Since P(z) f(z) = 0.106656 we replace the reference set {0, 0.5, 1} by {0, 0.54, 1}. We then
determine the levelled reference-polynomial

1
(x) = 1.718282x + 0.894607
for this reference set. Since the deviation is H = 0.105933 and since
max
xJ
|
1
(x) f(x)| = |
1
(1) f(1)| = 0.105933,
the process is terminated and
1
(x) is our desired polynomial of best approximation.
Let us discuss the choice of the initial reference set. One procedure is simply to let x
0
=
a, x
n+1
= b and to choose the remaining n points uniformly (forming an uniform grid
n+2
),
or nearly uniformly, spaced in the interval [a, b].
Another procedure suggested by Chebyshev is to let
x
i
=
b + a
2
+
b a
2
cos
i
n + 1
, i = 0, 1, . . . , n + 1.
In the case a = 1, b = 1, this choice corresponds to the extreme values of the Chebyshev
polynomials of degree n + 1, which is given by
T
n+1
(x) = cos ((n + 1) arccos x) (39)
so that
T
0
(x) = 1; T
1
(x) = x; T
2
(x) = 2x
2
1; T
3
(x) = 4x
3
3x
2
, . . . . (40)
40
5.2 Chebyshev polynomials
The Chebyshev polynomials of the rst kind, T
n
, and the second kind, U
n
, are dened by the
standard formulas
T
0
(x) = 1, T
1
(x) = x; (41)
T
n
(x) = 2xT
n1
(x) T
n2
(x), n = 2, 3, ...; (42)
U
0
(x) = 1, U
1
(x) = 2x; (43)
U
n
(x) = 2xU
n1
(x) U
n2
(x), n = 2, 3, ... . (44)
If |x| 1, then the following representations hold:
T
n
(x) = cos(narccos x), n = 0, 1, ...; (45)
U
n
(x) =
1

1 x
2
sin ((n + 1) arccos x), n = 0, 1, . . . . (46)
For x = 1, a transition to the limit should be performed in formula (46). The Chebyshev
polynomials are coupled by the relationships
U
n
(x) =
1
n + 1
T

n+1
(x), n = 0, 1, ... . (47)
The Chebyshev polynomials are orthogonal functions with a weight on segment [1, 1]:
1
_
1
T
n
(x)T
m
(x)
1

1 x
2
dx =
_

_
0, n = m
/2, n = m = 0
, n = m = 0
; (48)
1
_
1
U
n
(x)U
m
(x)

1 x
2
dx =
_
0, n = m
/2, n = m
. (49)
Thus, the Chebyshev polynomials of the rst kind are orthogonal on [1, 1] with weight
1/

1 x
2
and the Chebyshev polynomials of the second kind are orthogonal on [1, 1] with
weight

1 x
2
.
Problem 5.1
Calculate the quantities ||P
n
f|| using uniform and L
2
-norms in the interval [a, b] if
(a) f(x) = x
2
3x + 2; a = 0, b = 1, n = 1, P
1
(x) = 3x + 2;
(b) f(x) = x
3
2; a = 1, b = 1, n = 2, P
2
(x) = x
2
2;
(c) f(x) =

x + 4; a = 4, b = 0, n = 1, P
1
(x) = 0.5x + 2.
Problem 5.2
Construct reference-polynomial P
0
(x) P
0
= const of degree 0 at the reference set of
2 points, draw the graph and nd reference deviation (with three decimal places) for the function
f(x).
(a) f(x) = x
3
+ 2; x
0
= 0, x
1
= 1.
(b) f(x) = sin x; x
0
= 0.25, x
1
= 0.5.
(c) f(x) =

x; x
0
= 0, x
1
= 2.
41
Problem 5.4
Find coecients
i
and reference deviation H at the reference set of n + 2 points, forming
the uniform grid with the step h, for the function f(x).
(a) f(x) = x
2
+ 1; x
0
= 0, h = 0.1, n = 2.
(b) f(x) = sin x; x
0
= 0, h = 0.25, n = 1.
(c) f(x) = 2
x
; x
0
= 0, h = 1, n = 1.
Problem 5.5
Find analytically the polynomial of best approximation of degree 1 for Example 5.2 using
the approach presented at the end of Example 5.1.
6 Numerical Methods in Linear Algebra. Gauss Elimi-
nation
A linear system of n equations in n unknowns x
1
, x
2
, . . ., x
n
is a set of n equations E
1
, E
2
, . . .,
E
n
of the form
E
1
: a
11
x
1
+ a
12
x
2
+ . . . + a
1n
x
n
= b
1
E
2
: a
21
x
1
+ a
22
x
2
+ . . . + a
2n
x
n
= b
2
. . . . . . . .
E
n
: a
n1
x
1
+ a
n2
x
2
+ . . . + a
nn
x
n
= b
n
The system is called homogeneous if all b
j
= 0; otherwise it is inhomogeneous.
We can rewrite (50) in the vector notation
Ax = b, (50)
where A = [a
jk
] is the n n coecient matrix of system (50)
A =
_

_
a
11
a
12
. . . a
1n
a
21
a
22
. . . a
2n
. . . . . .
a
n1
a
n2
. . . a
nn
_

_
,
and
x =
_

_
x
1
. . .
x
n
_

_ , b =
_

_
b
1
. . .
b
n
_

_
are column vectors.
The augmented matrix

A of system (50) is the n (n + 1) matrix

A = [Ab] =
_

_
a
11
. . . a
1n
b
1
a
21
. . . a
2n
b
2
. . . . . .
a
n1
. . . a
nn
b
n
_

_
.
A solution of system (50) is a set of n numbers x
1
, x
2
, . . ., x
n
that satisfy all n equations.
They form the solution vector x.
42
The three elementary row operations can be applied to individual equations in a system
without altering the solution of the system:
(1) interchange the position of any two rows (i.e. equations);
(2) multiply a row (i.e. equation) by a non-sero number;
(3) replace a row by the sum of the row and a scalar multiple of another row .
6.1 Gauss elimination
Example 6.1
Consider the application of Gauss elimination to solving the equation system
E
1
: 3x
1
+ 6x
2
+ 9x
3
= 3
E
2
: 2x
1
+ (4 + p)x
2
+ 2x
3
= 4
E
3
: 3x
1
4x
2
11x
3
= 5
with the 3 3 coecient matrix
A =
_

_
3 6 9
2 4 + p 2
3 4 11
_

_,
and the column vectors
x =
_

_
x
1
x
2
x
3
_

_, b =
_

_
3
4
5
_

_ .
The augmented matrix

A of system (51) is the 3 4 matrix

A = [Ab] =
_

_
3 6 9 3
2 4 + p 2 4
3 4 11 5
_

_,
and (51) can be written in the form
_

_
3 6 9
2 4 + p 2
3 4 11
_

_
_

_
x
1
x
2
x
3
_

_ =
_

_
3
4
5
_

_ .
Gauss elimination to transform an augmented matrix to upper triangular form
To do this, we will perform only the elementary row operations listed above, in three stages.
At the rst stage, we write the initial augmented matrix and choose the pivot element 3,
denoted by 3 and situated in the rst column of the rst row.
At the second stage, we make the elements of column 1 in rows 2 and 3 zero by
(i) dividing row 1 by the pivot 3 and then
(ii) subtracting a suitable mutiple of the modied row 1 (that is, the modied row 1 times
2) from row 2 and
43
(iii) adding a suitable mutiple of the modied row 1 (that is, the modied row 1 times 3)
to row 3.
Then we select the next pivot, p, situated in the second column of the new second row. To
circumvent problems connected with a small p, we interchange the row in question (second)
with the row with the elements of largest modulus in the column below the pivot (third row).
This is called partial pivoting.
Then at the third stage, we make the elements of column 2 in row 3 zero by
(i) dividing row 2 by the pivot 2 and then
(ii) subtracting a suitable mutiple of the modied row 2 (that is, the modied row 2 times
p) from row 3.
At the fourth stage, we thus obtain an upper triangular matrix. Unknowns x
1
, x
2
, and x
3
are determined by the substitution in the reversed order.
Gauss elimination
to transform an augmented matrix to upper triangular form
Stage 1: Initial matrix
_

_
row1 3 6 9 3
row2 2 4 + p 2 4
row3 3 4 11 5
_

_ .
The element 3 in the rst column of the rst row is the pivot. The rst equation is the pivot
equation.
Stage 2: Reduce col 1 of row 2 and 3 to zero
_

_
oldrow1 3 6 9 3
oldrow2 - 2( oldrow1 )/3 0 p 4 2
oldrow3 +3( oldrow1 )/3 0 2 2 2
_

_,
Interchange rows 2 and 3
_

_
oldrow1 3 6 9 3
oldrow3 0 2 2 2
oldrow2 0 p 4 2
_

_ .
The element 2 in the second column of the second row is the pivot. The second equation is the
pivot equation.
Stage 3: Reduce col 2 of row 3 to zero
_

_
oldrow1 3 6 9 3
oldrow2 0 2 2 2
oldrow3 - p( oldrow2 )/2 0 0 4 + p 2 + p
_

_ ,
Stage 4: Finding unknowns x
1
, x
2
, and x
3
by the substitution in the reversed order.
44
For p = 0, the system with the upper triangular matrix has the form
3x
1
+ 6x
2
+ 9x
3
= 3
2x
2
2x
3
= 2
4x
3
= 2
The substitution in the reversed order beginning from the third equation yields the solution
x
1
= (3 6 (1.5) 9 (0.5))/3 = 5.5
x
2
= 0.5(2 + 2 (0.5)) = 1.5
x
3
= 0.5
Example 6.2
Gauss elimination to transform an augmented 2 3 matrix to upper triangular
form
Consider the application of Gauss elimination to solving the equation system
E
1
: 2x
1
+ 4x
2
= 1
E
2
: 6x
1
+ 3x
2
= 2
with the 2 2 coecient matrix
A =
_
2 4
6 3
_
,
and the column vectors
x =
_
x
1
x
2
_
, b =
_
1
2
_
.
The system can be written in the form
_
2 4
6 3
_ _
x
1
x
2
_
=
_
1
2
_
.
and the augmented matrix

A of this system is the 2 3 matrix

A = [Ab] =
_
2 4 1
6 3 2
_
.
Perform Gauss elimination.
Stage 1: Initial matrix
We do pivoting by interchanging row 2 and 1 to obtain the augmented matrix with the
maximum element a
11

A = [Ab] =
_
row1 6 3 2
row2 2 4 1
_
,
45
or the system
6x
1
+ 3x
2
= 2
2x
1
+ 4x
2
= 1
The (maximum) element a
11
= 6 in the rst column of the rst row is the pivot. The rst
equation is the pivot equation.
Stage 2: Reduce col 1 of row 2 to zero
_
oldrow1 6 3 2
oldrow2 - 2( oldrow1 )/6 0 3 1/3
_
.
Stage 3: Finding unknowns x
1
and x
2
by the substitution in the reversed order.
The system with the upper triangular matrix obtained at Stage 2 has the form
6x
1
+ 3x
2
= 2
3x
2
= 1/3
The substitution in the reversed order beginning from the second equation yields the solution
x
1
= (2 3 (1/9))/6 = 5/18
x
2
= 2/18
Verify the result by the Cramer rule:
det
_
2 4
6 3
_
= 18,
x
1
=
1
18
det
_
1 4
2 3
_
= 5/18, x
2
=
1
18
det
_
2 1
6 2
_
= 2/18.
6.2 Application of MATLAB for solving linear equation systems
The MATLAB operators \ and / perform matrix division and have identical eects. Thus to
solve a linear equation system Ax = b we may write either
A\ b or b

/A

.
. In the latter case the solution x is expressed as a row rather than a column vector. The
operators \ or / selects an appropriate algorithm dependent on the form of the matrix A. We
note two of them:
(i) if A is a triangular matrix the system is solved by the back or forward substitution alone;
(ii) if A is a square matrix, general LU decomposition is applied which is equivalent to Gauss
elimination in terms of elementary row operations.
46
Problem 6.1
Find the solution of the linear equation system
3x
1
+ 5x
2
= 5.9
x
1
4x
2
= 20.1
by Gauss elimination.
Solution. The system has the form
E
1
: 3x
1
+ 5x
2
= 5.9
E
2
: x
1
4x
2
= 20.1
with the 2 2 coecient matrix
A =
_
3 5
1 4
_
,
and the right-hand side column vector
b =
_
5.9
20.1
_
.
The system can be written in the form
_
3 5
1 4
_ _
x
1
x
2
_
=
_
5.9
20.1
_
.
and the augmented matrix

A of this system is the 2 3 matrix

A = [Ab] =
_
3 5 5.9
1 4 20.1
_
.
Perform Gauss elimination.
Stage 1: Initial matrix
We do pivoting by interchanging columns 2 and 1 to obtain the augmented matrix with the
maximum element a
11

A = [Ab] =
_
row1 5 3 5.9
row2 4 1 20.1
_
,
or the system
5x
2
+ 3x
1
= 5.9
4x
2
+ x
1
= 20.1
The (maximum) element a
11
= 5 in the rst column of the rst row is the pivot. The rst
equation is the pivot equation.
Stage 2: Reduce col 1 of row 2 to zero
_
oldrow1 5 3 5.9
oldrow2 +4( oldrow1 )/5 0 3.4 24.82
_
.
47
Stage 3: Finding unknowns x
1
and x
2
by the substitution in the reversed order.
The system with the upper triangular matrix obtained at Stage 2 has the form
5x
2
+ 3x
1
= 5.9
3.4x
1
= 24.82
The substitution in the reversed order beginning from the second equation yields the solution
x
2
= (5.9 3 (7.3))/5 = 3.2
x
1
= 7.3
Problem 6.2
Find the solution of the linear equation system
6x
2
+ 13x
3
= 61
6x
1
8x
3
= 38
13x
1
8x
2
= 79
by Gauss elimination.
Solution.
Stage 1: Initial matrix
_

_
row1 0 6 13 61
row2 6 0 8 38
row3 13 8 0 79
_

_ .
We do pivoting by interchanging rows 3 and 1 to obtain the augmented matrix with the maxi-
mum element a
11
= 13
_

_
row1 13 8 0 79
row2 6 0 8 38
row3 0 6 13 61
_

_ .
The element 13 in the rst column of the rst row is the pivot. Now the rst equation is the
pivot equation.
Stage 2: Reduce col 1 of row 2 and 3 to zero
We do it only for row 2 because col 1 of row 3 is zero:
_

_
oldrow1 13 8 0 79
newrow2 = oldrow2 - 6( oldrow1 )/13 0 48/13 = 3.692308 8 74.461538
oldrow3 0 6 13 61
_

_ ,
Now we do pivoting by interchanging newrows 3 and 2 to obtain the augmented matrix with
the maximum element a
22
= 6:
48
Interchange rows 2 and 3
_

_
oldrow1 13 8 0 79
oldrow3 0 6 13 61
newrow2 0 3.692308 8 74.461538
_

_ .
The element 6 in the second column of the second row is the pivot. The second equation is the
pivot equation.
Stage 3: Reduce col 2 of row 3 to zero
_

_
oldrow1 13 8 0 79
oldrow2 0 6 13 61
oldrow3 - (48/13) ( oldrow2 )/6 0 0 16 112
_

_,
Stage 4: Finding unknowns x
1
, x
2
, and x
3
by the substitution in the reversed order.
The system with the upper triangular matrix has the form
13x
1
8x
2
= 79
6x
2
+ 13x
3
= 61
16x
3
= 112
The substitution in the reversed order beginning from the third equation yields the solution
x
1
= (79 (8) (5))/13 = 3
x
2
= (61 13 7)/6 = 5
x
3
= 7
7 Numerical Methods in Linear Algebra. Solution by
Iteration
7.1 Convergence. Matrix norms
Consider linear system (50) of n equations in n unknowns with matrix A.
The matrix norm is given by one of the following expressions
||A|| =

_
n

j=1
n

k=1
c
2
jk
(Frobenius norm)
||A|| = max
k
n

j=1
|c
jk
| (Column sum norm)
||A|| = max
j
n

k=1
|c
jk
| (Row sum norm).
The number
cond (A) = ||A|| ||A
1
||
49
is called the condition number. When it is very large, the solution of the linear equation system
Ax = b will be very sensitive to small changes in b
An iteration method method for solving Ax = b is said to converge for an initial x
(0)
if
the corresponding iterative sequence x
(0)
, x
(1)
, . . ., x
(n)
, . . . converges to a solution of the given
system with respect to one of the matrix norms.
7.2 Jacobi iteration
Starting with the system
Ax = b
we rearrange it to the form
x = b + (I A)x
where
I =
_

_
1 0 . . . 0
0 1 . . . 0
. . . . . .
0 0 . . . 1
_

_
,
is the identity matrix and dene the Jacobi iteration as
x
(m+1)
= b + (I A)x
(m)
This method converges for every x
(0)
if and only if the spectral radius of I A equal to its
maximum eigenvalue is less than 1. The condition
||I A|| < 1
is sucient for the convergence of the Jacobi iteration.
Example 7.1
Find the solution of the linear equation system
0.5x
1
0.25x
2
= 1
0.25x
1
+ 0.5x
2
= 1
by the Jacobi iteration.
Solution.
The matrix of the system is
A =
_
0.5 0.25
0.25 0.5
_
.
I A =
_
0.5 0.25
0.25 0.5
_
= aT,
T =
_
2 1
1 2
_
, a =
1
4
.
The Jacobi iteration is given by
x
(m+1)
= b + aTx
(m)
.
50
7.3 GaussSeidel iteration
Example 7.2
Consider a linear system of n = 4 equations
x
1
0.25x
2
0.25x
3
= 50
0.25x
1
+ x
2
0.25x
4
= 50
0.25x
1
+ x
3
0.25x
4
= 25
0.25x
2
0.25x
3
+ x
4
= 25
Rewrite this system in the form
x
1
= 0.25x
2
+ 0.25x
3
+ 50
x
2
= 0.25x
1
+ 0.25x
4
+ 50
x
3
= 0.25x
1
+ 0.25x
4
+ 25
x
4
= 0.25x
2
+ 0.25x
3
+ 25
Choose an initial guess
x =
_

_
x
(0)
1
x
(0)
2
x
(0)
3
x
(0)
4
_

_
=
_

_
100
100
100
100
_

_
= 100
_

_
1
1
1
1
_

_
and compute from (51) an approximation (rst iteration)
x
(1)
1
= 0.25x
(0)
2
+ 0.25x
(0)
3
+ 50 = 100
x
(1)
2
= 0.25x
(1)
1
+ 0.25x
(0)
4
+ 50 = 100
x
(1)
3
= 0.25x
(1)
1
+ 0.25x
(0)
4
+ 25 = 75
x
(1)
4
= 0.25x
(1)
2
+ 0.25x
(1)
3
+ 25 = 68.75.
The next step (second iteration) is
x
(2)
1
= 0.25x
(1)
2
+ 0.25x
(1)
3
+ 50 = 93.75
x
(2)
2
= 0.25x
(2)
1
+ 0.25x
(1)
4
+ 50 = 90.62
x
(2)
3
= 0.25x
(2)
1
+ 0.25x
(1)
4
+ 25 = 65.62
x
(2)
4
= 0.25x
(2)
2
+ 0.25x
(2)
3
+ 25 = 64.06.
The exact solution is
x =
_

_
x
1
x
2
x
3
x
4
_

_
=
_

_
87.5
87.5
62.5
62.5
_

_
Describe the algorithm of the GaussSeidel iteration.
Assume that all the matrix diagonal elements a
jj
= 1:
A = I +L +U
51
where L and U are the lower and upper triangular matrices with the zero main diagonal.
Substituting this representation into Ax = b yields
Ax = (I +L +U)x = b
or
x = b Lx U)x
which produces the GaussSeidel iteration method
x
(m+1)
= b Lx
(m+1)
U)x
(m)
(a
jj
= 1)
Rewrite the GaussSeidel iterations as
(I +L)x
(m+1)
= b Ux
(m)
.
Multiplying by (I +L)
1
from the left, we obtain
x
(m+1)
= Cx
(m)
+ (I +L)
1
b.
where the iteration matrix
C = (I +L)
1
U.
The GaussSeidel iteration method converges for every x
(0)
if and only if all eigenvalues of
C (spectral radius) are less than 1.
The condition
||C|| < 1
is sucient for the convergence of the GaussSeidel iteration method.
7.4 Least squares
Consider approximating f(x) by a quadratic polynomial

f(x) = a
1
+ a
2
x + a
3
x
2
on the interval 0 x 1. To this end, choose a
1
, a
2
, and a
3
that minimize the root-mean-square
(rms) error
E =
__
1
0
[a
1
+ a
2
x + a
3
x
2
f(x)]
2
dx
_
1/2
.
The resulting linear equation system is satised by the optimum choice of a
1
, a
2
, and a
3
. They
are determined from
G
a
i
= 0, i = 1, 2, 3, G = E
2
.
We have, denoting R(x) = a
1
+ a
2
x + a
3
x
2
f(x),
G
a
1
=

a
1
_
1
0
R(x)
2
dx =
_
1
0

a
1
R(x)
2
dx =
_
1
0
2R(x)R

a
1
(x)dx = 2
_
1
0
R(x)dx,
G
a
2
=
_
1
0
2R(x)R

a
2
(x)dx = 2
_
1
0
xR(x)dx,
52
G
a
3
=
_
1
0
2R(x)R

a
3
(x)dx = 2
_
1
0
x
2
R(x)dx,
Introduce the moments of f(x)
I
j
=
_
1
0
x
j
f(x)dx, j = 0, 1, 2,
and calculate
_
1
0
R(x)dx = a
1
+ a
2
1
2
+ a
3
1
3
I
0
,
_
1
0
xR(x)dx = a
1
1
2
+ a
2
1
3
+ a
3
1
4
I
1
,
and
_
1
0
x
2
R(x)dx = a
1
1
3
+ a
2
1
4
+ a
3
1
5
I
2
,
Then, the optimum conditions
G
a
j
= 0, j = 0, 1, 2,
yield the linear equation system with the Hilbert matrix
a
1
+ a
2
1
2
+ a
3
1
3
= I
0
a
1
1
2
+ a
2
1
3
+ a
3
1
4
= I
1
a
1
1
3
+ a
2
1
4
+ a
3
1
5
= I
2
.
or
Ha = I
with
H =
_

_
1
1
2
1
3
1
2
1
3
1
4
1
3
1
4
1
5
_

_ .
I =
_

_
I
0
I
1
I
2
_

_, a = [a
1
, a
2
, a
3
].
Problem 7.1
Find the solution of the linear equation system
5x
1
+ x
2
+ 2x
3
= 19
x
1
+ 4x
2
2x
3
= 2
2x
1
+ 3x
2
+ 8x
3
= 39
by the GaussSeidel iteration.
Solution.
53
The matrix of the system is
A =
_

_
5 1 2
1 4 2
2 3 8
_

_ .
Rewrite the system as
x
1
= 1/5x
2
2/5x
3
+ 19/5
x
2
= 1/4x
1
+ 1/2x
3
1/2
x
3
= 1/4x
1
3/8x
2
+ 39/8
and compute the rst iteration choosing an initial guess
x = x
(0)
=
_

_
x
(0)
1
x
(0)
2
x
(0)
3
_

_
=
_

_
1
1
1
_

_
x
(1)
1
= 1/5x
(0)
2
2/5x
(0)
3
+ 19/5 = 1/5 2/5 + 19/5 = 16/5
x
(1)
2
= 1/4x
(1)
1
+ 1/2x
(0)
3
1/2 = (1/4)16/5 + 1/2 1/2 = 4/5
x
(1)
3
= 1/4x
(1)
1
3/8x
(1)
2
+ 39/8 = (1/4)16/5 + (3/8)(4/5) + 39/8 = 35/8
The second iteration is
x
(2)
1
= 1/5x
(1)
2
2/5x
(1)
3
+ 19/5
x
(2)
2
= 1/4x
(2)
1
+ 1/2x
(2)
3
1/2
x
(2)
3
= 1/4x
(2)
1
3/8x
(2)
2
+ 39/8
Problem 7.2
Find a quadratic polynomial of best approximation for
f(x) = e
x
using the least squares and the Hilbert matrix.
8 Numerical Methods for First-Order Dierential Equa-
tions
We will consider numerical methods for solving the initial value problem
y

(x) = f(x, y), y(x

0
) = y
0
. (51)
When step-by-step methods are applied, the solution y(x) is determined at the mesh (or
grid) points
x
1
= x
0
+ h, x
2
= x
0
+ 2h, x
3
= x
0
+ 3h, dots
where h is the step size.
54
8.1 Euler method
The Euler method is suggested by the use of the Taylor series:
y(x + h) = y(x) + hy

(x) +
h
2
2
y

(x) + dots, (52)

y(x + h) y(x) + hy

(x) = y(x) + hf(x, y).

Then, at the rst step, we compute
y
1
= y
0
+ hf(x
0
, y
0
).
At the second step,
y
2
= y
1
+ hf(x
1
, y
1
),
ans so on, so that the general formulas of the Euler method are
y
n+1
= y
n
+ hf(x
n
, y
n
), n = 0, 1, 2, . . . . (53)
Example 5.1 Euler method
Solve the initial value problem
y

(x) = x + y, y(0) = 0.
with the step h = 0.2.
Solution. In this case, the formulas of the Euler method are
y
n+1
= y
n
+ 0.2(x
n
+ y
n
), n = 0, 1, 2, . . . .
The exact solution is
y(x) = e
x
x 1.
Denoting
z
n
= 0.2(x
n
+ y
n
), y = Exact values, = Error
we obtain the table of computed quantities
n x
n
y
n
z
n
y
0 0.0 0.000 0.000 0.000 0.000
1 0.2 0.000 0.040 0.021 0.021
2 0.4 0.040 0.088 0.092 0.052
3 0.6 0.128 0.146 0.222 0.094
4 0.8 0.274 0.215 0.426 0.152
55
8.2 Improved Euler method
This method is based on the substitution of
y

(x) = f(x, y(x)) (54)

from (51) into (52):
y(x + h) = y(x) + hf +
h
2
2
f

+
h
3
6
f

+ . . . ,
where
f

= f
x
+ f
y
y

= f
x
+ f
y
f,
and the further derivatives are more cumbersome.
The general formulas of the improved Euler method are as follows:
at the rst step, we compute
y

n+1
= y
n
+ f(x
n
, y
n
), n = 0, 1, 2, . . . ; (55)
at the second step, we compute
y
n+1
= y
n
+
h
2
[f(x
n
, y
n
) + f(x
n+1
, y

n+1
)], n = 0, 1, 2, . . . . (56)
or
x
n+1
= x
n
+ h
k
1
= hf(x
n
, y
n
)
k
2
= hf(x
n+1
, y
n
+ k
1
) (57)
y
n+1
= y
n
+ 1/2(k
1
+ k
2
) (58)
Example 5.2 Improved Euler method
Solve the initial value problem
y

(x) = x + y, y(0) = 0.
with the step h = 0.2.
Solution. In this case, the formulas of the improved Euler method are
k
1
= 0.2(x
n
+ y
n
), k
1
= hf(x
n
, y
n
);
k
2
= 0.2(x
n
+ 0.2 + y
n
+ 0.2(x
n
+ y
n
)), k
2
= hf(x
n+1
, y

n+1
, y

n+1
= y
n
+ k
1
;
Setting
z
n
= 0.22(x
n
+ y
n
) + 0.02, y = Exact values, = Error
we obtain the table of computed quantities
n x
n
y
n
z
n
y
0 0.0 0.0000 0.0200 0.0000 0.0000
1 0.2 0.0200 0.0684 0.0214 0.014
2 0.4 0.0884 0.1274 0.0918 0.0034
3 0.6 0.2158 0.1995 0.2221 0.0063
4 0.8 0.4153 0.2874 0.4255 0.0102
56
8.3 RungeKutta methods
The general formulas of the RungeKutta method of fourth order are as follows:
k
1
= hf(x
n
, y
n
)
k
2
= hf(x
n
+ 1/2h, y
n
+ 1/2k
1
)
k
3
= hf(x
n
+ 1/2h, y
n
+ 1/2k
2
) (59)
k
4
= hf(x
n
+ h, y
n
+ k
3
)
x
n+1
= x
n
+ h
y
n+1
= y
n
+ 1/6(k
1
+ 2k
2
+ 2k
3
+ k
4
) (60)
Example 5.3 Classical RungeKutta method
Solve the initial value problem
y

(x) = x + y, y(0) = 0.
with the step h = 0.2.
Solution. In this case, the formulas of the classical RungeKutta method are
k
1
= 0.2(x
n
+ y
n
), k
2
= 0.2(x
n
+ 0.1 + y
n
+ 0.5k
1
),
k
3
= 0.2(x
n
+ 0.1 + y
n
+ 0.5k
2
), k
4
= 0.2(x
n
+ 0.2 + y
n
+ k
3
).
By inserting the formulas for k
1
and k
2
into the respective expressions for k
2
, k
3
, and k
4
we
have
k
2
= 0.22(x
n
+ y
n
) + 0.22,
k
3
= 0.222(x
n
+ y
n
) + 0.022,
k
4
= 0.2444(x
n
+ y
n
) + 0.0444,
and obtain
y
n+1
= y
n
+ 1/6(k
1
+ 2k
2
+ 2k
3
+ k
4
) = y
n
+ z
n
,
where
z
n
= 0.2214(x
n
+ y
n
) + 0.0214.
Setting
y = e
x
x 1 = exact values, = 10
6
error of y
n
we obtain the table of computed quantities
n x
n
y
n
z
n
y
0 0.0 0.000000 0.021400 0.000000 0
1 0.2 0.021400 0.070418 0.021403 3
2 0.4 0.091818 0.130289 0.091825 7
3 0.6 0.222106 0.203414 0.222119 11
4 0.8 0.425521 0.292730 0.425541 20
57
Problem 8.1
Solve the initial value problem
y

+ 0.1y = 0, y(0) = 2
with the step h = 0.1.
Solution. In this case, we have
y

= 0.1y,
The exact solution is
y(x) = 2e
0.1x
The formulas of the Euler method are
y
n+1
= y
n
0.1 0.1y
n
= 0.99y
n
, n = 0, 1, 2, . . . , y
0
= 2.
Denote
x
n
= nh = 0.1n, y = y(x
n
) Exact values, = Error = y(x
n
) y
n
.
We have (performing calculations with 6D)
y
1
= y
0
0.01y
0
= 2 0.02 = 1.98;
= y(x
1
) y
1
= 2e
0.10.1
1.98 = 1.980100 1.980000 = 0.000100.
y
2
= y
1
0.01y
1
= 1.980 0.0198 = 1.782;
= y(x
2
) y
2
= 2e
0.10.2
1.782 = 1.960397 1.960200 = 0.000197.
In general,
y
n+1
= 0.99y
n
= 0.99
2
y
n1
= . . . = 0.99
n+1
y
0
= 2 0.99
n+1
.
We have (performing calculations with 6D)
y
5
= 2 0.99
5
; = y(x
5
) y
5
= 2e
0.10.5
1.901980 = 1.902458 1.901980 = 0.000478.
y
10
= 2 0.99
10
; = y(x
10
) y
10
= 2e
0.1
1.901980 = 1.809674 1.808764 = 0.000910.
58
Problem 8.2
Solve the initial value problem
y

= y, y(0) = 1
with the step h = 0.1.
Solution. The exact solution is
y(x) = e
x
.
The formulas of the improved Euler method are as follows:
x
n+1
= x
n
+ 0.1, n = 0, 1, 2, . . . ,
k
1
= hf(x
n
, y
n
) = 0.1y
n
k
2
= hf(x
n+1
, y
n
+ k
1
) = 0.1(y
n
+ 0.1y
n
) = 0.11y
n
,
y
n+1
= y
n
+ 1/2(k
1
+ k
2
) = y
n
+ 0.5(0.1y
n
+ 0.11y
n
) = 1.105y
n
We have
y
n+1
= 1.105y
n
= 1.105
2
y
n1
= . . . = 1.105
n+1
y
0
= 1.105
n+1
.
so that, for example,
y
5
= 1.105
5
= 1.647446; = y(x
5
) y
5
= e
0.5
1.647446 = 1.648721 1.647446 = 0.001274.
y
10
= 1.105
10
= 2.714081; = y(x
10
) y
5
= e 2.714081 = 2.718281 2.714081 = 0.004201.
8.4 Numerical solution of ordinary dierential equations using MAT-
LAB
In this section we will use the notations x, x(t) for the sought-for function and t, t
0
, etc. for its
argument (sometimes referred to as time) instead of the corresponding quantities y, y(x) and
x, x
0
, etc. used in the previous section.
MATLAB uses RungeKuttaFehlberg methods to solve the initial-value problems for
ODEs. The solution is computed in a nite number of points where the spacing depends on
the solution. Fewer points are used in intervals where the solution is smooth and more points
where the solution varies rapidly.
[t,X] = ode23(str,t0,tt,x0) computes the solution of the ODE or the system of
ODEs given by the string str. The solution is given in the vector t containing the t-values,
and matrix X containing the solutions at these points. If the problem is scalar the result is
returned to vector x.
Solutions are computed from t0 to tt and x0 = x(t0) are the initial values. The system is
determined using the function dened in the M-le str, which is required to have two parame-
ters, scalar a and vector x, and to return vector x (the derivative). For a scalar ODE, x and
x

are scalars.
59
The approximation has a relative error of 10
3
, at most.
Example 8.4
Solve the initial value problem
x

= x
2
, x(0) = 1.
Solution.
We create the function xprim1 stored in the M-le xprim1.m
function z = xprim1(t,x)
z = x.x;
Then we call the MATLAB solver
[t,x] = ode23(xprim1,0,1,1);
Finally we can plot the solution
plot(t,x,-,t,x,o);
xlabel(time t0 = 0, tt = 1); ylabel(x values x(0) = 1);
Example 8.5
Solve the initial value problem
x

= x
2
, x(0) = 1.
Solution.
We create the function xprim2 stored in the M-le xprim2.m
function z = xprim2(t,x)
z = x.x;
Then we call the MATLAB solver
[t,x] = ode23(xprim2,0,0.95,1);
and plot the solution
plot(t,x,-,t,x,o);
xlabel(time t0 = 0, tt = 0.95); ylabel(x values x(0) = 1);
Note that the points where MATLAB computes the solution are denser in the area with
high derivative.
Example 8.6
Solve the initial value problem for the system of ODEs
x

1
= x
1
0.1x
1
x
2
+ 0.01t
x

2
= x
2
+ 0.02x
1
x
2
+ 0.04t
x
1
(0) = 30
x
2
(0) = 20
60
We create the function xprim3 stored in the M-le xprim3.m
function xprim = xprim3(t,x)
xprim(1) = x(1) 0.1x(1)x(2)+ 0.01t;
xprim(2) = x(2) 0.02x(1)x(2) + 0.04t;
Then we call the MATLAB solver
[t,x] = ode23(xprim3,0,20,[30;20]);
and plot the solution
plot(t,x);
xlabel(time t0 = 0, tt = 20); ylabel(x values x1(0) = 30, x2(0) = 20);
It is also possible to plot x
1
as a function of x
2
plot(x(:,2), x(:,1));
61