Numerical Differentiation:

The derivative of a funcion f (x) at a point x is defined by

f (x) = lim

f (x + h) f (x) h

(1)

Numerical differntiation becomes important as a result of the increased reliance on computers to solve problems. For example, if a function exists only as tabulated values at discrete intervals along the abscissa, the derivatives of the function must be calculated numerically. Since derivatives of a function exist only as limiting values, such as that given by Eq. (1), it is usually impossible in numerical work to deal with them directly. Instead we use finite differences, meaning small differences in the function at nearby points. A straightforward generalization of Eq. (1) is to make use of the difference

f (x) = f (x + h) f (x)
and equate f (x) as the quotient f (x)/h for finite h . The quantity f (x) is known as the forward difference of the function f (x) at x . This is not the only way to define the value of the difference of a function at two adjacent points. For example, we could have used the backward difference

f (x) = f (x) f (x h).

However, applied uncritically, the above procedures are almost guranteed to produce inaccurate results. There are two sources of error in the above procedures : truncation error and roundoff error. The truncation error comes from higher terms in the Taylor series expansion

f (x + h) = f (x) + hf (x) + h2 f (x)/2! + h3 f (x)/3! + ,

whence

[f (x + h) f (x)]/h = f (x) + hf (x)/2! + ,

so the truncation error is on the order of h . The roundoff error has various contributions. First there is roundoff error in h ; Suppose that you are at a point x = 10.3 and you blindly choose h = 0.0001 . Neither x = 10.3 nor x + h = 10.30001 is a number with an exact representation in binary; each is therefore represented with some fractional error characteristic of the machines floating-point format, whose value in single precision may be ~ 107 . One improvement is use the symmerized "3-point" formula (Central difference)

f (x ) =

f (x + h ) f (x h) 2h

whose truncation error is on the order of h2 . Another widely employed extrpolated difference form is

f (x) =
One can even use the "5-point" formula

8[f (x + h/4) f (x h/4)] [f (x + h/2) f (x h/2)] . 3h

f (x ) =
Formulas for higher derivatives can be constructed as

f (x 2h) 8f (x h) + 8f (x + h) f (x + 2h) + O(h4 ) 12h

f (x + h) 2f (x) + f (x h) = h2 f (x) + O(h4 )

so that an approximation to the second derivative accurate to order h2 is

f (x) =

f (x + h) 2f (x) + f (x h) h2

The 5-point difference formula for the second derivative of f(x) that are accurate to a higher order in h can be derived straightforwardly :

f (x) =

f (x 2h) + 16f (x h) 30f (x) + 16f (x + h) f (x + 2h) 12h2

Exercises 1. Implement various formulae given in the text above. 2. Change f(x) and see how the performance varies. 3. Change float to double and observe how the performance changes. * Default precision for REAL numbers is 32 bit (at least for Intel PC with Visual Studio), which has 7 significant digits and is called SINGLE PRECISION, while DOUBLE PRECISION numbers use 64-bits and have 15 significanr digits. Below is a double precision version of the program. Function tested is 1/x.