Section-1

(These notes are based mainly on the 6th Edition of the textbook
Numerical Methods for Engineers by S.C. Chapra and R.P. Canale.)

Error Definitions
Et = True Error = True value Approximate value
True relative error =
True percent relative error is given by t where

100

True percent absolute relative error is denoted by | |

If the true (or exact) value or answer is not known (which is usually the case in real-world engineering
problems), then one can use approximate-error definitions as follows,
=

where

100 =

100

is the approximate percent relative error.

Approximate percent absolute relative error is denoted by | |

Error Types
Round-off errors arise from the fact that computers can only represent/store quantities with a finite
number of digits.
=3.14159265358979 the omission of the remaining digits by the computer is called Round-off error.
Truncation errors result from applying numerical methods which employ approximations to represent
exact mathematical operations and quantities.
Ex: True versus approximate derivatives
Significant Digits and Error Criteria
Significant digits (or figures) of a number are those that can be used with confidence (i.e. which we know for
sure). The word significant means important, meaningful. Significance is a sign of precision. Here are
some examples:
a) 0.01625
0.001625
0.0001625
All of the three numbers have 4 significant digits. The leading
zeros locate the decimal point only. Using scientific notation can help to see the number of significant digits.
Rewrite the given numbers using scientific notation:
1625 10 5
1625 10 6
1625 10 7
b) The trailing zeros may cause confusion. Consider the number 73600 (e.g. consider the population of a
town. Is it really and exactly 73600 or youve just rounded it up, not being sure about the last two zeros). So,
are the last two zeros exact? Scientific notation can be used to indicate clearly the number of significant
digits. Then, 7.3600 10 4 , 7.360 10 4 and 7.36 10 4 have five, four and three significant digits,
respectively.
c) Write down the result 63.0180057 using 3, 6 and 8 significant digits and apply rounding.
3 significant digits: 63.0
6 significant digits: 63.0180 (the last 0 is significant)
8 significant digits: 63.018006
1

d) 5001.35 has 6 significant digits.

63.18 has 4 significant digits.
63.1800 has 6 significant digits (youre sure that this number has exactly two trailing zeros).
Note that 63.1800 is actually and exactly equal to 63.18 thus you may still get confused regarding whether
the last two zeros are significant or not.
e) The number 12345678912345678912345 which has 23 digits is typed in Matlab and when the enter key
is pressed, the number is stored and displayed as 1.234567891234568e+022 . It can be seen that the output
has the right order of magnitude but is correct only to the first 16 significant digits because by default Matlab
stores numeric quantities using 16 significant decimal digits. Notice that the output can also be expressed as
12345678912345680000000 .
f) Consider that 135.8 (in scientific notation 0.1358 10 3 ) is the true value of a quantity. The numbers 100,
140 and 136 are approximations for 135.8. Now, starting from the leftmost digit, apply 1 to 3 significant
digit approximations for 135.8 and employ rounding as follows:
100 is correct to 1 significant digit, in scientific notation: 0.1 103 . (Note: 104 can also be said to be correct
to 1 significant digit)
140 is correct to 2 significant digits, in scientific notation: 0.14 10 3 . (The zero which comes after 4, is
insignificant (i.e. not significant). Also, the zero in the end is not obtained due to rounding so it is not
significant. 141 is also correct to 2 significant digits.)
136 is correct to 3 significant digits, in scientific notation: 0.136 10 3 .

Computers can hold only a finite number of significant digits and this leads to round-off errors.
Fractional quantities are typically represented in computers using the floating-point form m b e where m
is the mantissa (or significand), b is the base of the number system and e is the exponent. Floating-point
numbers can be stored in single or double precision format according to IEEE 754 standard. By default,
Matlab stores all numeric quantities, values as double-precision floating-point. Single and double
precision correspond to about 7 and 16 significant decimal digits, respectively. When storing floating-point
numbers and doing arithmetic with floating-point numbers, double precision should be used to limit roundoff errors.
In numerical analyses which involve iterative procedures, computations can be repeated until | | <
where
is the prespecified tolerance or error criterion. It can be shown that when the criterion
s (0.5 10 2 n ) % is met (i.e. when | | < ), then the result is correct to at least n significant digits. Note
that this is a conservative criterion and it can yield results which are correct to more than n significant digits
(see Example 3.2 in the textbook).
Another criterion that gives a measure of accuracy in terms of significant digits is as follows. The number
x is said to approximate xt to n significant digits if
xt x
5 10 n
xt
The above criterion is appropriate to use if | | can be calculated (i.e. the true value is known). Hence, the
number x is said to approximate xt (the true value) to n significant digits when t (5 10 2 n )%

Taylor Series
Taylor Series is an infinite power-series and it predicts the value of the function at one point using the value
of the function and its derivatives at another point as follows:
(

) = ( )+

( ) +

The nth-order Taylor Series approximation of

(

) ( )+

( )
+
2!

( )

( )
+ +
3!

) around (about)

( )
+ +
3!

The remainder term (the truncation error for the nth-order approximation)
=

)(

)!

where

Zero-order approximation:

) ( )

First-order approximation:

) ( )+

Second-order approximation:

<

<

and =

( )
+
!

is therefore given by:

( )

( )
+
2!

( ) +

( )

( )

( )

is given by:
(h is the step size)

( )

) ( )+

( )

( ) +

The nth-order Taylor Series expansion will be exact for an nth-order polynomial.
In general, the truncation error is decreased by including additional terms in the expansion but we should
also limit the step size. We can also decrease the truncation error if we reduce the step size sufficiently (note
that when the step size is reduced, the term ( ) ( ) will change). We usually select the step size as
| | < 1. For example, consider two successive terms in the Taylor Series expansion which involve the
factors and , respectively. If the step size is taken as 10 (where p is a positive integer), then the
ratio /
equals 10 . This shows that if
increases, then the ratio /
also increases.
Lets make a formal definition of Taylor Series as follows:
If ( ) has derivatives of all orders at

(i.e. if
( )(

!
is called the Taylor series of
of Taylor series.

about the number

. If

( )

( ) exists for

= 0,1,2, ), then the series

( )
= 0, then we use the term Maclaurin series instead
( )(

The important question is: Can we write ( ) =

( ) , in other words does the Taylor
!
series of converge to ( ) ? It is clear that the Taylor series of converges to ( ) at = . However, we
cannot directly say that the Taylor series of (that is infinitely differentiable at = ) converges to ( ) in
an open interval containing = ; the series may not converge anywhere except at = , and if it does
converge elsewhere, it may converge to something else other than ( ). If the Taylor series of converges
to ( ) in an open interval containing = , it is said that is analytic at = .

Taylor Polynomials
Taylor polynomials are partial sums of the Taylor series of a function about a given point = . We use
Taylor polynomials to obtain approximations for a given function . The nth-order Taylor polynomial of
about = is denoted by which is given as follows.
( ) = ( )+

( )( ) +

( )

( )
( ) +
2!
( )(

( )=

( )
( ) + +
3!

( )

( )
( )
!

( )

Recall that the infinite series obtained by taking the limit of ( ) as is called the Taylor series of
about = . Notice that matches and its first n derivatives at = i.e.
( ) = ( ),

( )=

( ),

( )

( )=

( )(

For example, the 2nd-order Taylor Series approximation of a function around point =
is equivalent
to constructing a 2nd-order polynomial ( ) as given below. In other words, ( ) is the 2nd-order Taylor
polynomial of around = .
( )= ( )+
Note that

( )= ( ),

( ) = ( ) , and

Taylors Theorem says that if

order Taylor polynomial of about
(

( )=

Fig. 1.1

( )(

)+

( )
(
2!

( ) = ( ).

exists in an interval containing and , and if

= , then ( ) = ( ) + ( ) where

)(

( + 1)!

( ) is the nth-

( )

( ) : known function or exact solution, ( ): 0-order approx.,

( ): 1st-order approx., ( ): 2nd-order approx.
, and are Taylor polynomials of about = .

Two-variable Version of Taylor Series

If a function
has two independent variables x and y, then it can be shown that the Taylor Series expansion
of = ( , ) about the base point ( , ) is written as
(

)= ( ,

)+

where =

should recall that

1
!

2!

+ 2

and all the partial derivatives are evaluated at the base point. Here, you
=

The remainder term

(Note that

= 1,

<

is given below where

=1,

1
( + 1)!

+1

)(

<

<

and

<

) (

( , )

=1 for n being a positive integer.)

and

Taylor Series in Higher Dimensions (Multivariate Taylor's Theorem)

Consider a scalar-valued function ( ) where = ( , , , ). Let = ( , , , ) and suppose that
is differentiable on an open ball around . Noting that , the first-order Taylor's theorem is given as
( )= ( )+

( )(

) = ( )+ ( )( )

where is on the line segment joining and . Note that ( ) is the gradient vector of at and we have
a dot product operation between ( ) and ( ). Here , and can be regarded as both points and
vectors. (You should refresh your knowledge on vectors and vector calculus). If is close to , then is
close to , then we can make the following approximation if the first partial derivatives of are continuous
( ) ( )+ ( )( )
Therefore, the linearisation of

at

is defined as

( ) = ( )+ ( )( )

Note: Gradient is the vector of all of the first partial derivatives of a scalar-valued function of several
variables. Hence, if is a scalar-valued function of variables , , ,
then the gradient of is a row

]. Then, the gradient of at a point = ( , , , ) is

vector given by = [
( )=

( )

( )

( )

Note: Consider a multivariable function ( ) = ( , , ). The partial derivative of

the derivative of with respect to treating all other variables as constants.
( ,,

) = lim

( ,,

+ ,

,,

) ( ,,

with respect to

is

Example:

( ,

)=

Solution:

( ) = ( )+

( )=2+

This means the function

Example:

( )(

)+

) (

= (1,0). Determine the linearisation of

( )(
)+

] = [6

at

. Find the linearisation of

Solution: ( , , ) = [
,

). We have

) (

)=1+

( ) around (1,0).

is approximated by the linear function

( , , )=3

( ,

at ( ,

12

) = (1,2, 1) = [ (1,2, 1)

(1,2, 1)

) = (1,2, 1)

]
(1,2, 1)] = [48

36 96]

( , , ) = (1,2, 1) + (1,2, 1) ( 1, 2, + 1)
= 24 + (48, 36, 96) ( 1, 2, + 1)
= 24 + 48( 1) + 36( 2) 96( + 1)
= 48 + 36 96 192

Error Propagation
Consider a function ( , ) and let and be the approximations of x and y respectively. The problem is that
x and y are not known exactly, hence the exact value of ( , ) is unknown. If and are close enough to x
and y respectively, then a Taylor Series can be written to estimate ( , ) by dropping all second and higherorder terms as follows (note that the partial derivatives are evaluated at ( , )) :
( , ) ( , )+( )

+( )

The terms in the above equation can be arranged to get ( , )

where

( , ) = | ( , ) ( , )| , = | | and = | | .

and are the estimates of the errors in x and y respectively, whereas ( , ) represents the estimate
of the resulting error in ( , ).
Similarly, for n independent variables
analysis reveals,
( ,

,,

,,

with errors

,,

, a first-order error

++

Example: Consider the following formula for a stream through a rectangular channel where Q = flow rate
(m3/s), n = roughness coefficient, B = width (m), S = slope, and H = depth (m). For a particular case, the
width and the depth of the stream are 20 m and 0.3 m respectively. However, the roughness and the slope can
only be known with a precision of 10%. That is to say, the roughness coefficient is known to be about 0.03
with a range from 0.027 to 0.033 and the slope is known to be about 0.0003 with a range from 0.00027 to
0.00033. Apply a first-order error analysis to estimate the uncertainty (or error) in the flow rate.
=

)/
( +2 )
(

Solution:

We have

= 20,

Note that

= 0.3,

= 0.03,

)/
( +2 )
(

= 0.0003, = 0.003 and

. The partial derivatives

Substituting the given values yield

Finally,

= 50.74 ,

and

)/
( +2 )
(

0.5
/

= 0.00003

are evaluated at ( , ).

= 2536.9

= |50.74| 0.003 + |2536.9|0.00003 = 0.152 + 0.076 = 0.228

Therefore, the error in the flow rate will be about 0.228 m3/s

Condition Number
A computations condition is an indication of its sensitivity to uncertainty. A first-order Taylor Series
expansion ( ) ( ) + ( )( ) can be used to describe the condition number (CN) for functions of
single variable as follows where is an approximation of . Define,
[ ( )] =

( ) ( )

( )

( )( )
( )

[ ]=

[ ( )]
=
[ ]

( )
( )

where [ ( )] is the relative error of ( ) with respect to ( ) and [ ] is the relative error of x with respect
to . CN is the ratio of these two relative error terms. When CN=1, the functions relative error is identical to
the relative error in x. Functions having | | values much larger than 1 are said to be ill-conditioned i.e.
such functions will amplify severely the uncertainty in x. If | | is not much larger than 1, then the function
is said to be well-conditioned. For values of | | less than 1, the functions relative error is attenuated in
comparison to the relative error in x.
Example: Compute the condition numbers for ( ) = | 1| + 1 at the given x-values.
Solution:
| 1|
1
( )=

1
2 | 1|

[ ( )]
=
=
[ ]

It can be seen that ( ) is ill-conditioned near

( )
( )

= 1 since

| 1|
1

| 1| + 1

( ) is very high i.e. ( ) changes rapidly.

(2.5) = 0.233

(1.5) = 0.184

(0.01) = 0.00252
(1.5) = 0.621

( )=

2 | 1|

(0.5) = 0.207 ,

(0.5) = 0.092
(1.0001) = 49.51

(2.5) = 0.459

Operator Precedence in Matlab

An operator is a symbol representing an action (i.e. an operation) on one or two items. Operands are the
items which operators operate on. In the expression h+4 , the operator is + and the operands are h and 4.
7

Operators can be either unary or binary i.e. they have one or two operands, respectively. A unary operator is
an operator which acts on only one operand. - is a unary operator when used to negate a value, as in b=-2.
A binary operator is an operator which operates on two operands.
Operator precedence is the order in which Matlab evaluates an expression. Within each precedence level as
shown below, operators have equal precedence and are evaluated from left to right. The operator precedence
rules for the operators in Matlab are shown in the below list, ordered from the highest precedence level to the
lowest precedence level:
1) Parentheses
2) Transpose (.'), power (.^), complex conjugate transpose ('), matrix power (^)
3) Unary plus (+), unary minus (-), logical negation (~)
(Note: A unary operator is an operator which acts on only one operand. - is a unary operator when used to
negate a value, as in b=-2. In Matlab, when we type -2^2, we get -4; and when we type (-2)^2, we get 4.)
4) Multiplication (.*), right division (./), left division (.\), matrix multiplication (*), matrix right division (/),
matrix left division (\)
5) Addition (+), subtraction (-)
6) Colon operator (:)
7) Less than (<), less than or equal to (<=), greater than (>), greater than or equal to (>=), equal to (==), not
equal to (~=)
8) Element-wise AND (&)
9) Element-wise OR (|)
10) Short-circuit AND (&&)
11) Short-circuit OR (||)

Total Numerical Error

As shown in the below figure, the total numerical error is the sum of round-off errors and truncation errors.
Round-off errors increase when there is a considerably large number of computations. Round-off errors may
also be of concern due to problems associated with computer arithmetics (such as subtractive cancellation).
In other words, round-off errors can increase if the step size is decreased for a given numerical procedure.
On the other hand, truncation errors decrease when the step size is decreased. Hence, the step size should be
chosen optimally. However, computers today carry enough number of significant digits (extended precision)
and there are also smart algorithms (such as recasting the formulation, adjusting the order of computations,
efficient and accurate numerical methods) which are capable of diminishing the effects of total numerical
error. We may still need some trial-and-error, experience and intuition in choosing appropriate step sizes.