Numerical Methods with Worked Examples:

Matlab Edition
C. Woodford r C. Phillips

Numerical Methods
with Worked
Examples:
Matlab Edition
Second Edition
C. Woodford Prof. C. Phillips
Department of Computing Service School of Computing Science
Newcastle University Newcastle University
Newcastle upon Tyne, NE1 7RU Newcastle upon Tyne, NE1 7RU
UK UK
c.h.woodford@newcastle.ac.uk

Additional material to this book can be downloaded from http://extras.springer.com.

ISBN 978-94-007-1365-9 e-ISBN 978-94-007-1366-6

DOI 10.1007/978-94-007-1366-6
Springer Dordrecht Heidelberg London New York

Library of Congress Control Number: 2011937948

1st edition: © Chapman & Hall (as part of Springer SBM) 1997
© Springer Science+Business Media B.V. 2012
No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by
any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written
permission from the Publisher, with the exception of any material supplied specifically for the purpose
of being entered and executed on a computer system, for exclusive use by the purchaser of the work.

Cover design: deblik

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

Preface

This book is a survey of the numerical methods that are common to undergraduate
courses in Science, Computing, Engineering and Technology. The aim is to present
sufficient methods to facilitate the numerical analysis of mathematical models likely
to be encountered in practice. Examples of such models include the linear equations
describing the stress on girders, bridges and other civil engineering structures, the
differential equations of chemical and thermal reactions, and the inferences to be
drawn from observed data.
The book is written primarily for the student, experimental scientist and design
engineer for whom it should provide a range of basic tools. The presentation is
novel in that mathematical justification follows rather than precedes the description
of any method. We encourage the reader first to gain a familiarity with a particular
method through experiment. This is the approach we use when teaching this mate-
rial in university courses. We feel it is a necessary precursor to understanding the
underlying mathematics. The aim at all times is to use the experience of numerical
experiment and a feel for the mathematics to apply numerical methods efficiently
and effectively.
Methods are presented in a problem–solution–discussion order. The solution may
not be the most elegant but it represents the one most likely to suggest itself on the
basis of preceding material. The ensuing discussion may well point the way to better
things. Dwelling on practical issues we have avoided traditional problems having
neat, analytical solutions in favour of those drawn from more realistic modelling
situations which generally have no analytic solution.
It is accepted that the best way to learn is to teach. But even more so, the best
way to understand a mathematical procedure is to implement the method on a to-
tally unforgiving computer. Matlab enables mathematics as it is written on paper
to be transferred to a computer with unrivalled ease and so offers every encourage-
ment. The book will show how programs for a wide range of problems from solving
equations to finding optimum solutions may be developed. However we are not rec-
ommending re-inventing the wheel. Matlab provides an enormous range of ready to
use programs. Our aim is to give insight into which programs to use, what may be
expected and how results are to be interpreted. To this end we will include details of
the Matlab versions of the programs we develop and how they are to be employed.

v
vi Preface

We hope that readers will enjoy our book. It has been a refreshing experience to
reverse the usual form of presentation. We have tried to simplify the mathematics
as far as possible, and to use inference and experience rather than formal proof as a
first step towards a deeper understanding. Numerical analysis is as much an art as a
science and like its best practitioners we should be prepared to pick and choose from
the methods at our disposal to solve the problem at hand. Experience, a readiness to
experiment and not least a healthy scepticism when examining computer output are
qualities to be encouraged.
Newcastle University, Newcastle upon Tyne, UK Chris Woodford
Chris Phillips
Contents

1 Basic Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Matlab—The History and the Product . . . . . . . . . . . . . . . . 1
1.2 Creating Variables and Using Basic Arithmetic . . . . . . . . . . . 2
1.3 Standard Functions . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Vectors and Matrices . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 M-Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 The colon Notation and the for Loop . . . . . . . . . . . . . . . . 6
1.7 The if Construct . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.8 The while Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.9 Simple Screen Output . . . . . . . . . . . . . . . . . . . . . . . . 9
1.10 Keyboard Input . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.11 User Defined Functions . . . . . . . . . . . . . . . . . . . . . . . 10
1.12 Basic Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.13 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.14 Formatted Screen Output . . . . . . . . . . . . . . . . . . . . . . 12
1.15 File Input and Output . . . . . . . . . . . . . . . . . . . . . . . . 14
1.15.1 Formatted Output to a File . . . . . . . . . . . . . . . . . . 14
1.15.2 Formatted Input from a File . . . . . . . . . . . . . . . . . 14
1.15.3 Unformatted Input and Output (Saving and Retrieving Data) 15
2 Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3 Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3.1 Row Interchanges . . . . . . . . . . . . . . . . . . . . . . 24
2.3.2 Partial Pivoting . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.3 Multiple Right-Hand Sides . . . . . . . . . . . . . . . . . 30
2.4 Singular Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.5 Symmetric Positive Definite Systems . . . . . . . . . . . . . . . . 33
2.6 Iterative Refinement . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.7 Ill-Conditioned Systems . . . . . . . . . . . . . . . . . . . . . . . 37
2.8 Gauss–Seidel Iteration . . . . . . . . . . . . . . . . . . . . . . . . 37

vii
viii Contents

3 Nonlinear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.2 Bisection Method . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2.1 Finding an Interval Containing a Root . . . . . . . . . . . 50
3.3 Rule of False Position . . . . . . . . . . . . . . . . . . . . . . . . 51
3.4 The Secant Method . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.5 Newton–Raphson Method . . . . . . . . . . . . . . . . . . . . . . 55
3.6 Comparison of Methods for a Single Equation . . . . . . . . . . . 58
3.7 Newton’s Method for Systems of Nonlinear Equations . . . . . . . 59
3.7.1 Higher Order Systems . . . . . . . . . . . . . . . . . . . . 63
4 Curve Fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 Linear Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.2.1 Differences . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3 Polynomial Interpolation . . . . . . . . . . . . . . . . . . . . . . 77
4.3.1 Newton Interpolation . . . . . . . . . . . . . . . . . . . . 77
4.3.2 Neville Interpolation . . . . . . . . . . . . . . . . . . . . . 80
4.3.3 A Comparison of Newton and Neville Interpolation . . . . 81
4.3.4 Spline Interpolation . . . . . . . . . . . . . . . . . . . . . 83
4.4 Least Squares Approximation . . . . . . . . . . . . . . . . . . . . 86
4.4.1 Least Squares Straight Line Approximation . . . . . . . . . 86
4.4.2 Least Squares Polynomial Approximation . . . . . . . . . 89
5 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.2 Integration of Tabulated Functions . . . . . . . . . . . . . . . . . 98
5.2.1 The Trapezium Rule . . . . . . . . . . . . . . . . . . . . . 99
5.2.2 Quadrature Rules . . . . . . . . . . . . . . . . . . . . . . 101
5.2.3 Simpson’s Rule . . . . . . . . . . . . . . . . . . . . . . . 101
5.2.4 Integration from Irregularly-Spaced Data . . . . . . . . . . 102
5.3 Integration of Functions . . . . . . . . . . . . . . . . . . . . . . . 104
5.3.1 Analytic vs. Numerical Integration . . . . . . . . . . . . . 104
5.3.2 The Trapezium Rule (Again) . . . . . . . . . . . . . . . . 104
5.3.3 Simpson’s Rule (Again) . . . . . . . . . . . . . . . . . . . 106
5.4 Higher Order Rules . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.5 Gaussian Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . 110
5.6 Adaptive Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . 112
6 Numerical Differentiation . . . . . . . . . . . . . . . . . . . . . . . . 119
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
6.2 Two-Point Formula . . . . . . . . . . . . . . . . . . . . . . . . . 120
6.3 Three- and Five-Point Formulae . . . . . . . . . . . . . . . . . . . 122
6.4 Higher Order Derivatives . . . . . . . . . . . . . . . . . . . . . . 125
6.4.1 Error Analysis . . . . . . . . . . . . . . . . . . . . . . . . 126
6.5 Cauchy’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Contents ix

7 Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7.2 Forming a Linear Programming Problem . . . . . . . . . . . . . . 136
7.3 Standard Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
7.4 Canonical Form . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
7.5 The Simplex Method . . . . . . . . . . . . . . . . . . . . . . . . . 142
7.5.1 Starting the Simplex Method . . . . . . . . . . . . . . . . 146
7.6 Integer Programming . . . . . . . . . . . . . . . . . . . . . . . . 149
7.6.1 The Branch and Bound Method . . . . . . . . . . . . . . . 151
7.7 Decision Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 153
7.8 The Travelling Salesman Problem . . . . . . . . . . . . . . . . . . 155
7.9 The Machine Scheduling Problem . . . . . . . . . . . . . . . . . . 156
8 Optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
8.2 Grid Searching Methods . . . . . . . . . . . . . . . . . . . . . . . 171
8.2.1 Simple Grid Search . . . . . . . . . . . . . . . . . . . . . 171
8.2.2 Golden Section Search . . . . . . . . . . . . . . . . . . . . 173
8.3 Unconstrained Optimisation . . . . . . . . . . . . . . . . . . . . . 175
8.3.1 The Method of Steepest Descent . . . . . . . . . . . . . . 176
8.3.2 A Rank-One Method . . . . . . . . . . . . . . . . . . . . . 178
8.3.3 Generalised Rank-One Method . . . . . . . . . . . . . . . 181
8.4 Constrained Optimisation . . . . . . . . . . . . . . . . . . . . . . 184
8.4.1 Minimisation by Use of a Simple Penalty Function . . . . . 185
8.4.2 Minimisation Using the Lagrangian . . . . . . . . . . . . . 187
8.4.3 The Multiplier Function Method . . . . . . . . . . . . . . 188
9 Ordinary Differential Equations . . . . . . . . . . . . . . . . . . . . 197
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
9.2 First-Order Equations . . . . . . . . . . . . . . . . . . . . . . . . 200
9.2.1 Euler’s Method . . . . . . . . . . . . . . . . . . . . . . . . 200
9.2.2 Runge–Kutta Methods . . . . . . . . . . . . . . . . . . . . 202
9.2.3 Fourth-Order Runge–Kutta . . . . . . . . . . . . . . . . . 204
9.2.4 Systems of First-Order Equations . . . . . . . . . . . . . . 206
9.2.5 Higher Order Equations . . . . . . . . . . . . . . . . . . . 207
9.3 Boundary Value Problems . . . . . . . . . . . . . . . . . . . . . . 208
9.3.1 Shooting Method . . . . . . . . . . . . . . . . . . . . . . 208
9.3.2 Difference Equations . . . . . . . . . . . . . . . . . . . . 209
10 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . 215
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
10.2 The Characteristic Polynomial . . . . . . . . . . . . . . . . . . . 217
10.3 The Power Method . . . . . . . . . . . . . . . . . . . . . . . . . . 218
10.3.1 Power Method, Theory . . . . . . . . . . . . . . . . . . . 219
10.4 Eigenvalues of Special Matrices . . . . . . . . . . . . . . . . . . . 222
10.4.1 Eigenvalues, Diagonal Matrix . . . . . . . . . . . . . . . . 222
10.4.2 Eigenvalues, Upper Triangular Matrix . . . . . . . . . . . 223
x Contents

10.5 A Simple QR Method . . . . . . . . . . . . . . . . . . . . . . . . 223

11 Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
11.2 Statistical Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
11.2.1 Random Variable . . . . . . . . . . . . . . . . . . . . . . 232
11.2.2 Frequency Distribution . . . . . . . . . . . . . . . . . . . 232
11.2.3 Expected Value, Average and Mean . . . . . . . . . . . . . 234
11.2.4 Variance and Standard Deviation . . . . . . . . . . . . . . 234
11.2.5 Covariance and Correlation . . . . . . . . . . . . . . . . . 236
11.3 Least Squares Analysis . . . . . . . . . . . . . . . . . . . . . . . 239
11.4 Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . 241
11.4.1 Generating Random Numbers . . . . . . . . . . . . . . . . 242
11.5 Random Number Generators . . . . . . . . . . . . . . . . . . . . 243
11.5.1 Customising Random Numbers . . . . . . . . . . . . . . . 243
11.6 Monte Carlo Integration . . . . . . . . . . . . . . . . . . . . . . . 244
Matlab Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253