Meshfree Approximation Methods With MATLAB

.
e o
587?
I
S
Meshfree Approximation
Methods with MATLAB
1

INTERDISCIPLINARY MATHEMATICAL SCIENCES
Series Editor: Jinqiao Duan (Illinois Inst, of Tech., USA)

Editorial Board: Ludwig Arnold, Roberto Camassa, Peter Constantin,
Charles Doering, Paul Fischer, Andrei V. Fursikov, Fred R. McMorris,
Daniel Schertzer, Bjorn Schmalfuss, Xiangdong Ye, and
Jerzy Zabczyk
Published
V o l . 1: G l o b a l A t t r a c t o r s o f N o n a u t o n o m o u s D i s s i p a t i v e D y n a m i c a l Systems
David N. Cheban
V o l . 2: Stochastic D i f f e r e n t i a l E q u a t i o n s : T h e o r y a n d Applications
A V o l u m e i n H o n o r o f Professor B o r i s L . R o z o v s k i i
eds. Peter H. Baxendale & Sergey V. Lototsky
V o l . 3: A m p l i t u d e E q u a t i o n s f o r Stochastic P a r t i a l D i f f e r e n t i a l E q u a t i o n s
Dirk Blomker
V o l . 4: Mathematical Theory of Adaptive Control

Vladimir G. Sragovich
V o l . 5: T h e H i l b e r t - H u a n g T r a n s f o r m and Its A p p l i c a t i o n s
Norden E. Huang & Samuel S. P. Shen
Vol. 6: Meshfree A p p r o x i m a t i o n Methods w i t h MATLAB

Gregory E. Fasshauer
Meshfree Approximation
Methods with M A T L A B
G r e g o r y E . F a s s h a u e r r _ s
Illinois Institute of Technology, U S A - ^
W r i f l r l ^ H o n t i f i n
. lb. /
Published by
W o r l d
fif-?, Y~ Scientific Publishing Co. Pte. Ltd.
5 Toh Tuck Link, Singapore 596224
U S A c e : 2 7 W a r r e n
/ t>&^~ ^ Street, Suite 401-402, Hackensack, NJ 07601
/] UK office: 57 Shelton Street, Covent Garden, London W C 2 H 9 H E
British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library.
M E S H F R E E APPROXIMATION M E T H O D S W I T H MATLAB
(With CD-ROM)
Interdisciplinary Mathematical Sciences Vol. 6
Copyright 2007 by World Scientific Publishing Co. Pte. Ltd.
All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or
mechanical, including photocopying, recording or any information storage and retrieval system now known or to
be invented, without written permission from the Publisher.
For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center,
Inc., 222 Rosewood Drive, Danvers, M A 01923, U S A . In this case permission to photocopy is not required from
the publisher.
ISBN-13 978-981-270-633-1
ISBN-10 981-270-633-X
ISBN-13 978-981-270-634-8 (pbk)
ISBN-10 981-270-634-8 (pbk)
Printed by Mainland Press Pte Ltd

This book is dedicated to
Inge, Conny, Marc and Patrick.
Preface 34
T r a d i t i o n a l n u m e r i c a l m e t h o d s , s u c h as finite e l e m e n t , finite difference, o r finite v o l -

u m e m e t h o d s , w e r e m o t i v a t e d m o s t l y b y e a r l y one- a n d t w o - d i m e n s i o n a l s i m u l a t i o n s
of engineering problems v i a p a r t i a l differential equations ( P D E s ) . T h e discretiza-
t i o n i n v o l v e d i n a l l o f these m e t h o d s r e q u i r e s some s o r t o f u n d e r l y i n g c o m p u t a t i o n a l
m e s h , e.g., a t r i a n g u l a t i o n o f t h e r e g i o n o f i n t e r e s t . C r e a t i o n o f these meshes ( a n d
possible r e - m e s h i n g ) becomes a r a t h e r d i f f i c u l t t a s k i n t h r e e d i m e n s i o n s , a n d v i r t u -
a l l y i m p o s s i b l e for h i g h e r - d i m e n s i o n a l p r o b l e m s . T h i s is w h e r e meshfree methods
enter t h e p i c t u r e . Meshfree m e t h o d s are o f t e n b u t b y n o m e a n s h a v e t o be
r a d i a l l y s y m m e t r i c i n n a t u r e . T h i s is a c h i e v e d b y c o m p o s i n g some u n i v a r i a t e basic
function w i t h a (Euclidean) n o r m , a n d therefore t u r n i n g a p r o b l e m i n v o l v i n g m a n y
space d i m e n s i o n s i n t o o n e t h a t is v i r t u a l l y o n e - d i m e n s i o n a l . S u c h radial basis func-
tions are a t t h e h e a r t o f t h i s b o o k . Some p e o p l e have a r g u e d t h a t t h e r e are t h r e e
" b i g t e c h n o l o g i e s " for t h e n u m e r i c a l s o l u t i o n o f P D E s , n a m e l y finite difference, fi-
n i t e element, a n d s p e c t r a l m e t h o d s . W h i l e these t e c h n o l o g i e s c a m e i n t o t h e i r o w n
r i g h t i n successive decades, n a m e l y finite difference m e t h o d s i n t h e 1950s, finite e l -
e m e n t m e t h o d s i n t h e 1960s, a n d s p e c t r a l m e t h o d s i n t h e 1970s, meshfree methods
s t a r t e d t o a p p e a r i n t h e m a t h e m a t i c s l i t e r a t u r e i n t h e 1980s, a n d t h e y are n o w o n
their w a y t o b e c o m i n g " b i g technology" n u m b e r four. I n fact, we w i l l demonstrate
i n l a t e r p a r t s o f t h i s b o o k h o w d i f f e r e n t t y p e s o f meshfree m e t h o d s c a n b e v i e w e d
as g e n e r a l i z a t i o n s o f t h e t r a d i t i o n a l " b i g t h r e e " .
M u l t i v a r i a t e meshfree a p p r o x i m a t i o n m e t h o d s are b e i n g s t u d i e d b y m a n y re-
searchers. T h e y exist i n m a n y flavors a n d are k n o w n u n d e r m a n y n a m e s , e.g.,
diffuse e l e m e n t m e t h o d , element-free G a l e r k i n m e t h o d , g e n e r a l i z e d finite element
m e t h o d , / i p - c l o u d s , meshless l o c a l P e t r o v - G a l e r k i n m e t h o d , m o v i n g least squares
method, partition of unity finite e l e m e n t m e t h o d , r a d i a l basis f u n c t i o n method,
reproducing kernel particle m e t h o d , smooth particle hydrodynamics m e t h o d .
I n t h i s b o o k w e are c o n c e r n e d m o s t l y w i t h t h e m o v i n g least squares ( M L S ) a n d
r a d i a l basis f u n c t i o n ( R B F ) m e t h o d s . W e w i l l c o n s i d e r a l l d i f f e r e n t k i n d s o f aspects
o f these meshfree approximation methods: H o w to construct them? A r e these
c o n s t r u c t i o n s m a t h e m a t i c a l l y j u s t i f i a b l e ? H o w a c c u r a t e are t h e y ? A r e theEg~wavs
t o i m p l e m e n t t h e m e f f i c i e n t l y w i t h s t a n d a r d m a t h e m a t i c a l software-packages such
vii
viii Meshfree Approximation Methods with M A T L A B
as M A T L A B ? H O W d o t h e y c o m p a r e w i t h t r a d i t i o n a l m e t h o d s ? H o w d o t h e v a r i o u s
flavors o f meshfree m e t h o d s differ f r o m one a n o t h e r , a n d h o w are t h e y s i m i l a r t o one
a n o t h e r ? Is t h e r e a g e n e r a l f r a m e w o r k t h a t c a p t u r e s a l l o f these m e t h o d s ? What
s o r t o f a p p l i c a t i o n s are t h e y e s p e c i a l l y w e l l s u i t e d for?
W h i l e w e d o p r e s e n t m u c h o f t h e u n d e r l y i n g t h e o r y for R B F a n d M L S ap-
proximation methods, t h e e m p h a s i s i n t h i s b o o k is n o t o n p r o o f s . For read-
ers w h o are interested i n all the mathematical details and intricacies of the
t h e o r y we r e c o m m e n d t h e t w o e x c e l l e n t recent monographs [ B u h m a n n (2003);
W e n d l a n d (2005a)]. I n s t e a d , o u r o b j e c t i v e is t o m a k e t h e t h e o r y accessible t o a
w i d e a u d i e n c e t h a t i n c l u d e s g r a d u a t e s t u d e n t s a n d p r a c t i t i o n e r s i n a l l s o r t s o f sci-
ence a n d e n g i n e e r i n g fields. W e w a n t t o p u t t h e m a t h e m a t i c a l t h e o r y i n t h e c o n t e x t
o f a p p l i c a t i o n s a n d p r o v i d e M A T L A B i m p l e m e n t a t i o n s w h i c h g i v e t h e r e a d e r a n easy
e n t r y i n t o meshfree a p p r o x i m a t i o n m e t h o d s . T h e s k i l l e d r e a d e r s h o u l d t h e n easily
be able t o m o d i f y t h e p r o g r a m s p r o v i d e d here for h i s / h e r specific p u r p o s e s .
I n a c e r t a i n sense t h e present b o o k was i n s p i r e d b y t h e b e a u t i f u l l i t t l e b o o k [ T r e -
f e t h e n ( 2 0 0 0 ) ] . W h i l e t h e present b o o k is m u c h m o r e e x p a n s i v e ( f i l l i n g m o r e t h a n
1 2
five h u n d r e d pages w i t h f o r t y - s e v e n M A T L A B p r o g r a m s , one M a p l e p r o g r a m , one
h u n d r e d figures, m o r e t h a n f i f t y t a b l e s , a n d m o r e t h a n five h u n d r e d references), i t is
o u r a i m t o p r o v i d e t h e reader w i t h r e l a t i v e l y s i m p l e M A T L A B c o d e t h a t i l l u s t r a t e s
j u s t a b o u t e v e r y aspect discussed i n t h e b o o k .
A l l M A T L A B p r o g r a m s p r i n t e d i n t h e t e x t (as w e l l as a few m o d i f i c a t i o n s dis-
cussed) are also i n c l u d e d o n t h e enclosed C D . T h e f o l d e r MATLAB c o n t a i n s M - f i l e s
a n d d a t a files o f t y p e M A T t h a t h a v e b e e n w r i t t e n a n d t e s t e d w i t h M A T L A B 7. F o r
t h o s e readers w h o d o n o t have access t o M A T L A B 7, t h e folder MATLAB6 c o n t a i n s
versions o f these files t h a t are c o m p a t i b l e w i t h t h e o l d e r M A T L A B release. The
m a i n difference b e t w e e n t h e t w o v e r s i o n s is t h e use o f a n o n y m o u s f u n c t i o n s i n t h e
M A T L A B 7 code as c o m p a r e d t o i n l i n e f u n c t i o n s i n t h e M A T L A B 6 v e r s i o n . T w o
packages f r o m t h e M A T L A B C e n t r a l F i l e E x c h a n g e [ M C F E ] are used t h r o u g h o u t t h e
b o o k : t h e f u n c t i o n h a l t o n s e q w r i t t e n b y D a n i e l D o u g h e r t y a n d used t o g e n e r a t e
sequences o f H a l t o n p o i n t s ; t h e /cd-tree l i b r a r y ( g i v e n as a set o f M A T L A B M E X - f i l e s )
w r i t t e n b y G u y Shechter a n d used t o g e n e r a t e t h e kd-tvee data structure underlying
o u r sparse m a t r i c e s based o n c o m p a c t l y s u p p o r t e d basis f u n c t i o n s . B o t h o f these
packages are discussed i n A p p e n d i x A a n d need t o be d o w n l o a d e d separately. The
folder M a p l e o n t h e C D c o n t a i n s t h e one M a p l e file m e n t i o n e d a b o v e .
T h e m a n u s c r i p t for t h i s b o o k a n d some o f i t s e a r l i e r i n c a r n a t i o n s h a v e b e e n
used i n g r a d u a t e l e v e l courses a n d s e m i n a r s a t N o r t h w e s t e r n U n i v e r s i t y , V a n d e r b i l t
U n i v e r s i t y , a n d t h e I l l i n o i s I n s t i t u t e o f T e c h n o l o g y . S p e c i a l t h a n k s are d u e t o J o n
1
M A T L A B is a trademark of T h e MathWorks, Inc. and is used with permission. T h e Math-
Works does not warrant the accuracy of the text or exercises in this book. T h i s book's use or
discussion of M A T L A B software or related products does not constitute endorsement or sponsor-
ship by T h e MathWorks of a particular pedagogical approach or particular use of the M A T L A B
software.
2
M a p l e is a registered trademark of Waterloo Maple Inc.
Preface ix
C h e r r i e , J o h n E r i c k s o n , P a r i t o s h M o k h a s i , L a r r y S c h u m a k e r , a n d J a c k Z h a n g for
reading various p o r t i o n s of t h e m a n u s c r i p t a n d / o r M A T L A B code a n d p r o v i d i n g
h e l p f u l feedback. F i n a l l y , t h a n k s are d u e t o a l l t h e p e o p l e a t W o r l d Scientific
P u b l i s h i n g C o . w h o h e l p e d m a k e t h i s p r o j e c t a success: R a j e s h B a b u , Y i n g O i
Chiew, L i n d a K w a n , R o k T i n g Tan, and Yubing Zhai.
Greg Fasshauer
Chicago, I L , J a n u a r y 2007
Contents
Preface vii
1. Introduction 1
s
1.1 M o t i v a t i o n : Scattered D a t a I n t e r p o l a t i o n i n K 2
1.1.1 The Scattered D a t a I n t e r p o l a t i o n P r o b l e m 2
1.1.2 Example: I n t e r p o l a t i o n w i t h Distance Matrices 4
1.2 Some H i s t o r i c a l R e m a r k s 13
2. R a d i a l Basis F u n c t i o n I n t e r p o l a t i o n i n M A T L A B 17
2.1 R a d i a l (Basis) F u n c t i o n s 17
2.2 R a d i a l Basis F u n c t i o n I n t e r p o l a t i o n 19
3. Positive Definite Functions 27
3.1 Positive Definite Matrices and Functions 27

3.2 I n t e g r a l C h a r a c t e r i z a t i o n s for ( S t r i c t l y ) P o s i t i v e D e f i n i t e
Functions 31
3.2.1 Bochner's Theorem 31
3.2.2 Extensions t o Strictly Positive Definite Functions 32
3.3 Positive Definite Radial Functions 33
4. Examples of Strictly Positive Definite Radial Functions 37
4.1 E x a m p l e 1: G a u s s i a n s 37
4.2 E x a m p l e 2: L a g u e r r e - G a u s s i a n s 38
4.3 E x a m p l e 3: P o i s s o n R a d i a l F u n c t i o n s 39
4.4 E x a m p l e 4: M a t e r n F u n c t i o n s 41
4.5 E x a m p l e 5: G e n e r a l i z e d Inverse M u l t i q u a d r i c s 41
4.6 E x a m p l e 6: T r u n c a t e d P o w e r F u n c t i o n s 42
4.7 E x a m p l e 7: P o t e n t i a l s a n d W h i t t a k e r R a d i a l F u n c t i o n s 43
4.8 E x a m p l e 8: I n t e g r a t i o n A g a i n s t S t r i c t l y P o s i t i v e
Definite Kernels 45
xi
xii Meshfree Approximation Methods with MATLAB
4.9 Summary 45
5. Completely Monotone and M u l t i p l y Monotone Functions 47
5.1 Completely Monotone Functions 47

5.2 M u l t i p l y Monotone Functions 49
6. Scattered D a t a I n t e r p o l a t i o n w i t h P o l y n o m i a l Precision 53
6.1 Interpolation w i t h Multivariate Polynomials 53

6.2 Example: Reproduction of Linear Functions Using
Gaussian R B F s 55
6.3 Scattered D a t a Interpolation w i t h M o r e General
Polynomial Precision 57
6.4 Conditionally Positive Definite Matrices and Reproduction
of Constant Functions 59
7. Conditionally Positive Definite Functions 63
7.1 Conditionally Positive Definite Functions Defined 63

7.2 C o n d i t i o n a l l y Positive Definite Functions and Generalized
Fourier Transforms 65
8. Examples of Conditionally Positive Definite Functions 67
8.1 E x a m p l e 1: G e n e r a l i z e d M u l t i q u a d r i c s 67
8.2 E x a m p l e 2: R a d i a l P o w e r s 69
8.3 E x a m p l e 3: T h i n P l a t e Splines 70
9. Conditionally Positive Definite Radial Functions 73
9.1 Conditionally Positive Definite R a d i a l Functions and Completely

Monotone Functions 73
9.2 Conditionally Positive Definite R a d i a l Functions and
M u l t i p l y Monotone Functions 75
9.3 Some S p e c i a l P r o p e r t i e s o f C o n d i t i o n a l l y P o s i t i v e D e f i n i t e
Functions of Order One 76
10. Miscellaneous T h e o r y : Other N o r m s a n d Scattered D a t a F i t t i n g

on Manifolds 79
10.1 C o n d i t i o n a l l y P o s i t i v e D e f i n i t e F u n c t i o n s a n d p-Norms 79
10.2 Scattered D a t a F i t t i n g o n Manifolds 83
10.3 Remarks 83
11. C o m p a c t l y S u p p o r t e d R a d i a l Basis F u n c t i o n s 85
11.1 O p e r a t o r s for R a d i a l F u n c t i o n s a n d D i m e n s i o n W a l k s 85
11.2 Wendland's Compactly Supported Functions 87
Contents xiii
11.3 Wu's Compactly Supported Functions 88

11.4 Oscillatory Compactly Supported Functions 90
11.5 O t h e r C o m p a c t l y S u p p o r t e d R a d i a l Basis Functions 92
12. Interpolation w i t h Compactly Supported RBFs in M A T L A B 95
12.1 A s s e m b l y o f t h e Sparse I n t e r p o l a t i o n M a t r i x 95
12.2 Numerical Experiments w i t h CSRBFs 99
13. R e p r o d u c i n g K e r n e l H i l b e r t Spaces a n d N a t i v e Spaces for

Strictly Positive Definite Functions 103
13.1 R e p r o d u c i n g K e r n e l H i l b e r t Spaces 103

13.2 N a t i v e Spaces for S t r i c t l y P o s i t i v e D e f i n i t e F u n c t i o n s 105
13.3 E x a m p l e s o f N a t i v e Spaces for P o p u l a r R a d i a l B a s i c F u n c t i o n s . . 108
14. T h e P o w e r F u n c t i o n a n d N a t i v e Space E r r o r E s t i m a t e s 111
14.1 F i l l Distance and A p p r o x i m a t i o n Orders I l l

14.2 Lagrange F o r m of the Interpolant and Cardinal
Basis Functions 112
14.3 T h e Power F u n c t i o n 115
14.4 G e n e r i c E r r o r E s t i m a t e s for F u n c t i o n s i n Af$(l) 117
14.5 E r r o r Estimates i n Terms of the F i l l Distance 119
15. Refined and I m p r o v e d E r r o r B o u n d s 125
15.1 N a t i v e Space E r r o r B o u n d s f o r Specific B a s i s F u n c t i o n s 125

15.1.1 I n f i n i t e l y S m o o t h Basis Functions 125
15.1.2 Basis F u n c t i o n s w i t h F i n i t e Smoothness 126
15.2 I m p r o v e m e n t s for N a t i v e Space E r r o r B o u n d s 127
15.3 E r r o r B o u n d s for F u n c t i o n s O u t s i d e t h e N a t i v e Space 128
15.4 E r r o r B o u n d s for S t a t i o n a r y A p p r o x i m a t i o n 130
15.5 Convergence w i t h Respect t o the Shape P a r a m e t e r 132
15.6 P o l y n o m i a l I n t e r p o l a t i o n as t h e L i m i t o f R B F I n t e r p o l a t i o n . . . 133
16. Stability and Trade-Off Principles 135
16.1 S t a b i l i t y and C o n d i t i o n i n g o f R a d i a l Basis F u n c t i o n I n t e r p o l a n t s . 135

16.2 Trade-Off Principle I : A c c u r a c y vs. S t a b i l i t y 138
16.3 T r a d e - O f f P r i n c i p l e I I : A c c u r a c y a n d S t a b i l i t y vs. P r o b l e m Size . 140
16.4 T r a d e - O f f P r i n c i p l e I I I : A c c u r a c y vs. E f f i c i e n c y 140
17. N u m e r i c a l E v i d e n c e for A p p r o x i m a t i o n O r d e r R e s u l t s 141
17.1 I n t e r p o l a t i o n for e > 0 141

17.1.1 C h o o s i n g a G o o d Shape P a r a m e t e r v i a T r i a l a n d E r r o r . . 142
xiv Meshfree Approximation Methods with MATLAB
17.1.2 T h e P o w e r F u n c t i o n as I n d i c a t o r for a G o o d S h a p e
Parameter 142
17.1.3 Choosing a G o o d Shape P a r a m e t e r v i a Cross V a l i d a t i o n . 1 4 6
17.1.4 The Contour-Pade A l g o r i t h m 151
17.1.5 Summary 152
17.2 Non-stationary Interpolation 153
17.3 Stationary Interpolation 155
18. The O p t i m a l i t y of R B F Interpolation 159
18.1 T h e Connection to O p t i m a l Recovery 159

18.2 O r t h o g o n a l i t y i n R e p r o d u c i n g K e r n e l H i l b e r t Spaces 160
18.3 Optimality Theorem I 162
18.4 Optimality Theorem I I 163
18.5 Optimality Theorem I I I 164
19. L e a s t Squares R B F A p p r o x i m a t i o n w i t h M A T L A B 165
19.1 O p t i m a l Recovery Revisited 165

19.2 R e g u l a r i z e d L e a s t Squares A p p r o x i m a t i o n 166
19.3 L e a s t Squares A p p r o x i m a t i o n W h e n R B F C e n t e r s D i f f e r from
D a t a Sites 168
19.4 L e a s t Squares S m o o t h i n g o f N o i s y D a t a 170
20. T h e o r y for L e a s t Squares A p p r o x i m a t i o n 177
20.1 Well-Posedness o f R B F L e a s t Squares A p p r o x i m a t i o n 177

20.2 E r r o r B o u n d s for L e a s t Squares A p p r o x i m a t i o n 179
21. A d a p t i v e L e a s t Squares A p p r o x i m a t i o n 181
21.1 A d a p t i v e L e a s t Squares u s i n g K n o t I n s e r t i o n 181

21.2 A d a p t i v e L e a s t Squares u s i n g K n o t R e m o v a l 184
21.3 Some N u m e r i c a l E x a m p l e s 188
22. M o v i n g L e a s t Squares A p p r o x i m a t i o n 191
22.1 Discrete W e i g h t e d Least Squares A p p r o x i m a t i o n 191

22.2 Standard Interpretation of M L S A p p r o x i m a t i o n 192
22.3 The Backus-Gilbert Approach to M L S A p p r o x i m a t i o n 194
22.4 Equivalence of the T w o Formulations of M L S A p p r o x i m a t i o n . . . 198
22.5 D u a l i t y a n d B i - O r t h o g o n a l Bases 199
22.6 S t a n d a r d M L S A p p r o x i m a t i o n as a C o n s t r a i n e d Quadratic
Optimization Problem 202
22.7 Remarks 202
23. Examples of M L S Generating Functions 205

Contents xv
23.1 Shepard's M e t h o d 205

23.2 M L S Approximation w i t h Nontrivial Polynomial Reproduction . . 207
24. MLS Approximation with M A T L A B 211
24.1 A p p r o x i m a t i o n w i t h Shepard's M e t h o d 211

24.2 M L S A p p r o x i m a t i o n w i t h Linear Reproduction 216
24.3 Plots of Basis-Dual Basis Pairs 222
25. E r r o r B o u n d s for M o v i n g L e a s t Squares A p p r o x i m a t i o n 225
25.1 A p p r o x i m a t i o n O r d e r o f M o v i n g Least Squares 225
26. A p p r o x i m a t e M o v i n g Least Squares A p p r o x i m a t i o n 229
26.1 High-order Shepard M e t h o d s via M o m e n t Conditions 229

26.2 Approximate Approximation .. . 230
26.3 C o n s t r u c t i o n o f G e n e r a t i n g F u n c t i o n s for A p p r o x i m a t e M L S
Approximation 232
27. N u m e r i c a l E x p e r i m e n t s for A p p r o x i m a t e M L S A p p r o x i m a t i o n 237
27.1 Univariate Experiments 237

27.2 Bivariate Experiments 241
28. Fast F o u r i e r T r a n s f o r m s 243
28.1 NFFT 243

28.2 A p p r o x i m a t e M L S A p p r o x i m a t i o n v i a N o n - u n i f o r m Fast Fourier
Transforms 245
29. Partition of Unity Methods 249
29.1 Theory 249

29.2 Partition of Unity Approximation w i t h M A T L A B 251
30. Approximation of Point Cloud D a t a i n 3D 255
30.1 A G e n e r a l A p p r o a c h v i a I m p l i c i t Surfaces 255

30.2 A n Illustration in 2D 257
30.3 A Simplistic Implementation i n 3 D via Partition of U n i t y
Approximation in M A T L A B 260
31. Fixed Level Residual Iteration 265
31.1 Iterative Refinement 265

31.2 Fixed Level Iteration 267
31.3 Modifications o f the Basic F i x e d Level I t e r a t i o n A l g o r i t h m . . . . 269
31.4 Iterated Approximate M L S Approximation i n M A T L A B 270
31.5 Iterated Shepard A p p r o x i m a t i o n 274
xvi Meshfree Approximation Methods with M A T L A B
32. Multilevel Iteration 277
32.1 Stationary Multilevel Interpolation 277

32.2 A M A T L A B Implementation of Stationary Multilevel
Interpolation 279
32.3 Stationary Multilevel A p p r o x i m a t i o n 283
32.4 Multilevel Interpolation w i t h Globally Supported RBFs 287
33. Adaptive Iteration 291
33.1 A Greedy Adaptive A l g o r i t h m 291

33.2 T h e Faul-Powell A l g o r i t h m 298
34. I m p r o v i n g the C o n d i t i o n N u m b e r of the I n t e r p o l a t i o n M a t r i x 303
34.1 Preconditioning: T w o Simple Examples 304

34.2 Early Preconditioners 305
34.3 Preconditioned G M R E S via A p p r o x i m a t e Cardinal Functions . . 309
34.4 Change o f Basis 311
34.5 Effect o f t h e " B e t t e r " B a s i s o n t h e C o n d i t i o n N u m b e r o f t h e
Interpolation M a t r i x 314
34.6 Effect o f t h e " B e t t e r " B a s i s o n t h e A c c u r a c y o f t h e I n t e r p o l a n t . 316
35. O t h e r Efficient N u m e r i c a l M e t h o d s 321
35.1 T h e Fast M u l t i p o l e M e t h o d 321

35.2 F a s t Tree C o d e s 327
35.3 Domain Decomposition 331
36. Generalized H e r m i t e I n t e r p o l a t i o n 333
36.1 T h e Generalized H e r m i t e I n t e r p o l a t i o n P r o b l e m 333

36.2 M o t i v a t i o n for t h e S y m m e t r i c F o r m u l a t i o n 335
37. R B F Hermite Interpolation in M A T L A B 339
38. Solving Elliptic Partial Differential Equations via R B F Collocation 345
38.1 Kansa's Approach 345

38.2 A n Hermite-based Approach 348
38.3 E r r o r B o u n d s for S y m m e t r i c C o l l o c a t i o n 349
38.4 O t h e r Issues 350
39. Non-Symmetric R B F Collocation in M A T L A B 353
39.1 Kansa's Non-Symmetric Collocation M e t h o d 353
40. Symmetric R B F Collocation i n M A T L A B 365

Contents xvii
40.1 Symmetric Collocation M e t h o d 365

40.2 Summarizing Remarks on the Symmetric and Non-Symmetric
Collocation Methods 372
41. Collocation w i t h CSRBFs i n M A T L A B 375
41.1 Collocation w i t h Compactly Supported RBFs 375

41.2 Multilevel R B F Collocation 380
42. U s i n g R a d i a l Basis Functions i n Pseudospectral M o d e 387
42.1 Differentiation Matrices 388

42.2 P D E s w i t h B o u n d a r y Conditions via Pseudospectral Methods . . 390
42.3 A Non-Symmetric RBF-based Pseudospectral M e t h o d 391
42.4 A Symmetric RBF-based Pseudospectral M e t h o d 394
42.5 A Unified Discussion 396
42.6 Summary 398
43. RBF-PS Methods in M A T L A B 401
43.1 C o m p u t i n g the RBF-Differentiation M a t r i x i n M A T L A B . . . . . . 401

43.1.1 S o l u t i o n o f a 1-D T r a n s p o r t E q u a t i o n 403
43.2 Use o f t h e C o n t o u r - P a d e A l g o r i t h m w i t h t h e P S A p p r o a c h . . . . 405
43.2.1 Solution of the I D Transport Equation Revisited 405
43.3 C o m p u t a t i o n of Higher-Order Derivatives 407
43.3.1 Solution of the Allen-Cahn Equation 409
43.4 Solution of a 2D Helmholtz Equation 411
43.5 S o l u t i o n o f a 2 D Laplace E q u a t i o n w i t h Piecewise B o u n d a r y
Conditions 415
43.6 Summary 416
44. R B F Galerkin Methods 419
44.1 A n Elliptic P D E w i t h Neumann Boundary Conditions 419

44.2 A Convergence E s t i m a t e 420
44.3 A Multilevel R B F Galerkin Algorithm 421
45. R B F Galerkin Methods in M A T L A B 423
Appendix A Useful Facts f r o m Discrete M a t h e m a t i c s 427
A.l H a l t o n Points 427

A. 2 kd-Tvees 428
Appendix B U s e f u l Facts f r o m A n a l y s i s 431
B. l Some I m p o r t a n t C o n c e p t s f r o m M e a s u r e T h e o r y 431
B.2 A Brief S u m m a r y of Integral Transforms 432
xviii Meshfree Approximation Methods with MATLAB
B. 3 T h e S c h w a r t z Space a n d t h e G e n e r a l i z e d F o u r i e r T r a n s f o r m . . . 433
Appendix C Additional Computer Programs 435
C. l M A T L A B Programs 435
C. 2 Maple Programs 440
Appendix D Catalog of R B F s w i t h Derivatives 443
D. l Generic Derivatives 443

D.2 F o r m u l a s for Specific B a s i c F u n c t i o n s 444
D.2.1 Globally Supported, Strictly Positive Definite Functions . 444
D.2.2 Globally Supported, Strictly Conditionally Positive
Definite Functions of Order 1 445
Definite Functions o f Order 2 446
Definite Functions o f Order 3 446
Definite Functions of Order 4 447
D.2.6 Globally Supported, Strictly Positive Definite and
Oscillatory Functions 447
D.2.7 Compactly Supported, Strictly Positive Definite
Functions 448
Bibliography 451
Index 491
Chapter 1
Introduction
Meshfree m e t h o d s have g a i n e d m u c h a t t e n t i o n i n recent years, n o t o n l y i n t h e

m a t h e m a t i c s b u t also i n t h e e n g i n e e r i n g c o m m u n i t y . T h u s , m u c h o f t h e w o r k c o n -
cerned w i t h meshfree a p p r o x i m a t i o n m e t h o d s is i n t e r d i s c i p l i n a r y a t t h e interface
b e t w e e n m a t h e m a t i c s a n d n u m e r o u s a p p l i c a t i o n areas (see t h e p a r t i a l l i s t b e l o w ) .
M o r e o v e r , c o m p u t a t i o n w i t h h i g h - d i m e n s i o n a l d a t a is a n i m p o r t a n t issue i n m a n y
areas o f science a n d e n g i n e e r i n g . M a n y t r a d i t i o n a l n u m e r i c a l m e t h o d s c a n e i t h e r
n o t h a n d l e such p r o b l e m s a t a l l , o r are l i m i t e d t o v e r y special ( r e g u l a r ) s i t u a t i o n s .
Meshfree m e t h o d s are o f t e n b e t t e r s u i t e d t o cope w i t h changes i n t h e g e o m e t r y
o f t h e d o m a i n o f i n t e r e s t (e.g., free surfaces a n d large d e f o r m a t i o n s ) t h a n classical
d i s c r e t i z a t i o n techniques s u c h as finite differences, finite elements o r finite v o l u m e s .
A n o t h e r o b v i o u s a d v a n t a g e o f meshfree d i s c r e t i z a t i o n s is o f course t h e i r i n -
dependence f r o m a mesh. M e s h g e n e r a t i o n is s t i l l t h e m o s t t i m e c o n s u m i n g p a r t
o f a n y mesh-based n u m e r i c a l s i m u l a t i o n . Since meshfree d i s c r e t i z a t i o n t e c h n i q u e s
are based o n l y o n a set o f i n d e p e n d e n t p o i n t s , these costs o f m e s h g e n e r a t i o n are
eliminated. Meshfree a p p r o x i m a t i o n m e t h o d s c a n be seen t o p r o v i d e a n e w gen-
e r a t i o n o f n u m e r i c a l t o o l s . O t h e r t r a d i t i o n a l n u m e r i c a l m e t h o d s such as t h e finite
element, finite difference o r finite v o l u m e m e t h o d s are u s u a l l y l i m i t e d t o p r o b l e m s
i n v o l v i n g t w o o r t h r e e p a r a m e t e r s (space d i m e n s i o n s ) . H o w e v e r , i n m a n y a p p l i c a -
t i o n s t h e n u m b e r o f p a r a m e t e r s c a n easily r a n g e i n t h e h u n d r e d s o r even t h o u s a n d s .
M u l t i v a r i a t e a p p r o x i m a t i o n m e t h o d s present one w a y t o address these issues.
A p p l i c a t i o n s o f meshfree m e t h o d s c a n be f o u n d
i n m a n y different areas o f science a n d e n g i n e e r i n g v i a scattered data mod-

eling (e.g., fitting o f p o t e n t i a l energy surfaces i n c h e m i s t r y ; c o u p l i n g o f
e n g i n e e r i n g m o d e l s w i t h sets o f i n c o m p a t i b l e p a r a m e t e r s ; mapping prob-
lems i n geodesy, geophysics, m e t e o r o l o g y ) ;
i n m a n y different areas o f science a n d e n g i n e e r i n g v i a solution of partial
differential equations (e.g., s o l u t i o n o f gas d y n a m i c s e q u a t i o n s , B o l t z m a n n
a n d F o k k e r - P l a n c k e q u a t i o n s i n s i x - d i m e n s i o n a l phase space; p r o b l e m s i n -
v o l v i n g m o v i n g d i s c o n t i n u i t i e s such as cracks a n d shocks, m u l t i - s c a l e resolu-
t i o n , large m a t e r i a l d i s t o r t i o n s ; e l a s t i c i t y studies i n p l a t e a n d s h e l l b e n d i n g
l
2 Meshfree Approximation Methods with MATLAB
problems; applications i n nanotechnology);

i n non-uniform sampling (e.g., medical imaging, tomographic reconstruc-
tion);
i n mathematical finance (e.g., option pricing);
i n computer graphics (e.g., r e p r e s e n t a t i o n o f surfaces f r o m p o i n t i n f o r m a t i o n
such as laser r a n g e scan d a t a , i m a g e w a r p i n g ) ;
i n learning theory, neural networks a n d data mining (e.g., kernel approxi-
m a t i o n , support vector machines);
in optimization.
Since m a n y o f these a p p l i c a t i o n s e i t h e r c o m e d o w n t o a f u n c t i o n a p p r o x i m a t i o n
p r o b l e m , o r i n c l u d e f u n c t i o n a p p r o x i m a t i o n as a f u n d a m e n t a l c o m p o n e n t , we w i l l
b e g i n o u r discussion w i t h a n d i n fact base a l a r g e p a r t o f t h e c o n t e n t s o f t h i s
book on the multivariate scattered data i n t e r p o l a t i o n problem.
s
1.1 Motivation: Scattered Data Interpolation in M
W e w i l l n o w describe t h e g e n e r a l process o f s c a t t e r e d d a t a f i t t i n g , w h i c h is one o f

t h e f u n d a m e n t a l p r o b l e m s i n a p p r o x i m a t i o n t h e o r y a n d d a t a m o d e l i n g i n general.
O u r desire t o have a w e l l - p o s e d p r o b l e m f o r m u l a t i o n w i l l n a t u r a l l y l e a d t o a n i n -
t r o d u c t o r y e x a m p l e based o n t h e use o f so-called distance matrices. I n the next
c h a p t e r s we w i l l generalize t h i s a p p r o a c h b y i n t r o d u c i n g t h e c o n c e p t o f a r a d i a l
basis f u n c t i o n .
1.1.1 The Scattered Data Interpolation Problem
I n m a n y scientific d i s c i p l i n e s one faces t h e f o l l o w i n g p r o b l e m : W e are g i v e n a set o f

d a t a ( m e a s u r e m e n t s , a n d l o c a t i o n s a t w h i c h these m e a s u r e m e n t s w e r e o b t a i n e d ) ,
a n d we w a n t t o f i n d a r u l e w h i c h a l l o w s us t o deduce i n f o r m a t i o n a b o u t t h e process
we are s t u d y i n g also at l o c a t i o n s different f r o m t h o s e a t w h i c h we o b t a i n e d o u r
m e a s u r e m e n t s . T h u s , we are t r y i n g t o f i n d a f u n c t i o n Vf w h i c h is a " g o o d " f i t t o
t h e g i v e n d a t a . T h e r e are m a n y w a y s t o decide w h a t w e m e a n b y " g o o d " , a n d t h e
o n l y c r i t e r i o n we w i l l consider n o w is t h a t w e w a n t t h e f u n c t i o n Vf to e x a c t l y m a t c h
the given measurements at the corresponding locations. T h i s a p p r o a c h is c a l l e d
interpolation, a n d i f t h e l o c a t i o n s at w h i c h t h e m e a s u r e m e n t s are t a k e n do n o t lie
o n a u n i f o r m o r r e g u l a r g r i d , t h e n t h e process is c a l l e d scattered data interpolation.
T o give a precise d e f i n i t i o n we assume t h a t t h e m e a s u r e m e n t l o c a t i o n s (or data
sites) are l a b e l e d xj, j = 1 , . . . , j V , a n d t h e c o r r e s p o n d i n g m e a s u r e m e n t s (or data
values) are c a l l e d yj. W e w i l l use X t o d e n o t e t h e set o f d a t a sites a n d assume
s
t h a t X C fl for some r e g i o n O i n R . T h r o u g h o u t t h i s b o o k we w i l l r e s t r i c t o u r
discussion t o s c a l a r - v a l u e d d a t a , i.e., V j e R. However, m u c h of the following can
be g e n e r a l i z e d easily t o p r o b l e m s w i t h v e c t o r - v a l u e d d a t a . I n m a n y of our later
i . .ft
1. Introduction 3
discussions we w i l l assume t h a t t h e d a t a are o b t a i n e d b y s a m p l i n g some ( u n k n o w n )

f u n c t i o n / at t h e d a t a sites, i.e., yj = f(xj), j = 1,... ,N. O u r n o t a t i o n Vf for t h e
i n t e r p o l a t i n g f u n c t i o n emphasizes t h e c o n n e c t i o n b e t w e e n t h e i n t e r p o l a n t a n d t h e
d a t a f u n c t i o n / . W e are n o w r e a d y for a precise f o r m u l a t i o n o f t h e s c a t t e r e d d a t a
interpolation problem.
P r o b l e m 1.1 ( S c a t t e r e d D a t a I n t e r p o l a t i o n ) . Given data (xj,yj), j =

s
1,...,N, with xj e R ; yj G R, find a (continuous) function Vf such that
x
^f( o) = yj, j = i,...,N.
s
T h e fact t h a t we a l l o w Xj t o l i e i n a n a r b i t r a r y s - d i m e n s i o n a l space R means
t h a t t h e f o r m u l a t i o n o f P r o b l e m 1.1 allows us t o cover m a n y different t y p e s o f ap-
plications. I f s = 1 t h e d a t a c o u l d , e.g., b e a series o f m e a s u r e m e n t s t a k e n over
a c e r t a i n t i m e p e r i o d , t h u s t h e " d a t a sites" Xj w o u l d c o r r e s p o n d t o c e r t a i n t i m e
instances. F o r s = 2 we c a n t h i n k o f t h e d a t a b e i n g o b t a i n e d over a p l a n a r r e g i o n ,
a n d so xj c o r r e s p o n d s t o t h e t w o c o o r d i n a t e s i n t h e p l a n e . F o r instance, w e m i g h t
w a n t t o p r o d u c e a m a p t h a t shows t h e r a i n f a l l i n t h e s t a t e we l i v e i n based o n t h e
d a t a c o l l e c t e d at w e a t h e r s t a t i o n s l o c a t e d t h r o u g h o u t t h e s t a t e . F o r s 3 w e m i g h t
t h i n k o f a s i m i l a r s i t u a t i o n i n space. O n e p o s s i b i l i t y is t h a t w e c o u l d be i n t e r e s t e d
i n t h e t e m p e r a t u r e d i s t r i b u t i o n i n s i d e some s o l i d b o d y . H i g h e r - d i m e n s i o n a l e x a m -
ples m i g h t n o t be t h a t i n t u i t i v e , b u t a m u l t i t u d e o f t h e m e x i s t , e.g., i n finance,
o p t i m i z a t i o n , economics or s t a t i s t i c s , b u t also i n a r t i f i c i a l i n t e l l i g e n c e or l e a r n i n g
theory.
A c o n v e n i e n t a n d c o m m o n a p p r o a c h t o s o l v i n g t h e s c a t t e r e d d a t a p r o b l e m is t o
m a k e t h e a s s u m p t i o n t h a t t h e f u n c t i o n Vf is a l i n e a r c o m b i n a t i o n o f c e r t a i n basis
functions B,k i.e.,
N
s
Vf{x) = c B (x),
k k x e R . (1.1)
A;=l
S o l v i n g t h e i n t e r p o l a t i o n p r o b l e m u n d e r t h i s a s s u m p t i o n leads t o a s y s t e m o f
linear equations of the f o r m
Ac = y,
w h e r e t h e entries o f t h e interpolation matrix A are g i v e n b y Ajk = Bk(xj), j ,k =
T T
1,...,N, c= [a,... ,c ] ,
N a n d y = [y ,... 1 ,y ] .
N
P r o b l e m 1.1 w i l l be well-posed, i.e., a s o l u t i o n t o t h e p r o b l e m w i l l exist a n d be

u n i q u e , i f a n d o n l y i f t h e m a t r i x A is n o n - s i n g u l a r .
I n t h e u n i v a r i a t e s e t t i n g i t is w e l l k n o w n t h a t one c a n i n t e r p o l a t e t o a r b i t r a r y
d a t a at N d i s t i n c t d a t a sites u s i n g a p o l y n o m i a l o f degree Nl. For the m u l t i v a r i a t e
s e t t i n g , however, t h e r e is t h e f o l l o w i n g n e g a t i v e r e s u l t (see [ M a i r h u b e r ( 1 9 5 6 ) ; C u r t i s
(1959)]).
T h e o r e m 1.1 ( M a i r h u b e r - C u r t i s ) . If Q, C R, s
s > 2 , contains an interior
point, then there exist no Haar spaces of continuous functions except for one-
dimensional ones.
I n o r d e r t o u n d e r s t a n d t h i s t h e o r e m w e need
D e f i n i t i o n 1.1. L e t t h e f i n i t e - d i m e n s i o n a l l i n e a r f u n c t i o n space B C C ( f 2 ) have a

basis {Bi,..., Bjy}. T h e n B is a Haar space o n fl i f
det A ^ O
for a n y set o f d i s t i n c t X i , . . . , CCJV i n f2. H e r e A is t h e m a t r i x w i t h entries Ajk =
B ( ).
k Xj
N o t e t h a t existence o f a H a a r space g u a r a n t e e s i n v e r t i b i l i t y o f t h e i n t e r p o l a t i o n
m a t r i x A, i.e., existence a n d uniqueness o f a n i n t e r p o l a n t o f t h e f o r m (1.1) t o
d a t a specified a t XI,...,XN f r o m t h e space B. A s m e n t i o n e d above, u n i v a r i a t e
p o l y n o m i a l s o f degree N 1 f o r m a n A ^ - d i m e n s i o n a l H a a r space for d a t a g i v e n a t
X i , . . . , X N .
T h e M a i r h u b e r - C u r t i s t h e o r e m t e l l s us t h a t i f w e w a n t t o have a w e l l - p o s e d
m u l t i v a r i a t e scattered d a t a i n t e r p o l a t i o n p r o b l e m we can no longer fix i n advance
t h e set o f basis f u n c t i o n s w e p l a n t o use for i n t e r p o l a t i o n o f a r b i t r a r y s c a t t e r e d d a t a .
For e x a m p l e , i t is n o t possible t o p e r f o r m u n i q u e i n t e r p o l a t i o n w i t h ( m u l t i v a r i a t e )
2
p o l y n o m i a l s o f degree N t o d a t a g i v e n a t a r b i t r a r y l o c a t i o n s i n M . Instead, the
basis s h o u l d d e p e n d o n t h e d a t a l o c a t i o n s . W e w i l l g i v e a s i m p l e e x a m p l e o f s u c h
a n i n t e r p o l a t i o n scheme i n t h e n e x t s u b s e c t i o n .
Proof. [of T h e o r e m 1.1] L e t s > 2 a n d assume t h a t B is a H a a r space w i t h basis

{Bi,..., B N } w i t h N > 2. W e need t o s h o w t h a t t h i s leads t o a c o n t r a d i c t i o n .
W e l e t x\,..., XN be a set o f d i s t i n c t p o i n t s i n ft C M s
and A the m a t r i x w i t h
entries Ajk = Bk(xj), j , k = 1 , . . . , N. T h e n , b y t h e d e f i n i t i o n o f a H a a r space, w e
have
det A ^ O . (1.2)
Now, consider a closed p a t h P i n f2 c o n n e c t i n g o n l y x a n d X2- T h i s is p o s s i b l e
since b y a s s u m p t i o n fl c o n t a i n s a n i n t e r i o r p o i n t . W e c a n exchange the
p o s i t i o n s o f X \ a n d X2 b y m o v i n g t h e m c o n t i n u o u s l y a l o n g t h e p a t h P (without
interfering w i t h any of the other X j ) . T h i s means, however, t h a t rows 1 a n d 2 o f
t h e d e t e r m i n a n t ( 1 . 2 ) have been e x c h a n g e d , a n d so t h e d e t e r m i n a n t has changed
sign.
Since t h e d e t e r m i n a n t is a c o n t i n u o u s f u n c t i o n o f X \ a n d X2 w e m u s t have h a d
d e t = 0 a t some p o i n t a l o n g P. T h i s c o n t r a d i c t s ( 1 . 2 ) .
1.1.2 Example: Interpolation with Distance Matrices
In order t o o b t a i n data dependent a p p r o x i m a t i o n spaces, as suggested by the

M a i r h u b e r - C u r t i s t h e o r e m w e n o w consider a s i m p l e e x a m p l e . A s a " t e s t f u n c t i o n "
we e m p l o y t h e f u n c t i o n
f (x)
a
s
= 4 ]^[a:d(l - x d ) , X = ( X ! , . . . , x a ) e [o, i ] .s
d=l
1. Introduction 5
S
T h i s f u n c t i o n is zero o n t h e b o u n d a r y o f t h e u n i t c u b e i n R a n d has a m a x i m u m
v a l u e o f one at t h e center o f t h e c u b e . A s i m p l e M A T L A B s c r i p t d e f i n i n g f s is g i v e n
as P r o g r a m C . l i n A p p e n d i x C .
We w i l l use a set o f u n i f o r m l y s c a t t e r e d d a t a sites i n t h e u n i t c u b e a t w h i c h
w e sample o u r t e s t f u n c t i o n f . T h i s w i l l be a c c o m p l i s h e d here ( a n d i n m a n y o t h e r
s
examples l a t e r o n ) b y r e s o r t i n g t o t h e so-called Halton points. T h e s e are u n i f o r m l y

s
d i s t r i b u t e d r a n d o m p o i n t s i n (0, l ) . A set o f 289 H a l t o n p o i n t s i n t h e u n i t s q u a r e
2
i n M. is s h o w n i n F i g u r e 1.1. M o r e d e t a i l s o n H a l t o n p o i n t s are p r e s e n t e d i n
A p p e n d i x A . I n our c o m p u t a t i o n a l experiments we generate H a l t o n points using
the program h a l t o n s e q . m w r i t t e n by Daniel Dougherty. T h i s function can be
d o w n l o a d e d f r o m t h e M A T L A B C e n t r a l F i l e E x c h a n g e (see [ M C F E ] ) .
2
Fig. 1.1 289 Halton points in the unit square in R .
A s e x p l a i n e d i n t h e p r e v i o u s s u b s e c t i o n we are i n t e r e s t e d i n c o n s t r u c t i n g a ( c o n -
t i n u o u s ) f u n c t i o n Vf t h a t interpolates t h e samples o b t a i n e d f r o m f s a t t h e set o f
H a l t o n p o i n t s , i.e., s u c h t h a t
Vf(xj) f (xj),
s Xj a Halton point.
A s p o i n t e d o u t above, i f s = 1, t h e n t h i s p r o b l e m is o f t e n s o l v e d u s i n g u n i v a r i a t e
p o l y n o m i a l s or splines. F o r a s m a l l n u m b e r o f d a t a sites p o l y n o m i a l s m a y w o r k
s a t i s f a c t o r i l y . H o w e v e r , i f t h e n u m b e r o f p o i n t s increases, i.e., t h e p o l y n o m i a l degree
g r o w s , t h e n i t is w e l l k n o w n t h a t one s h o u l d use splines ( o r piecewise p o l y n o m i a l s )
t o a v o i d o s c i l l a t i o n s . T h e s i m p l e s t s o l u t i o n is t o use a c o n t i n u o u s piecewise l i n e a r
spline, i.e., t o " c o n n e c t t h e d o t s " . I t is also w e l l k n o w n t h a t one possible basis f o r
t h e space o f piecewise l i n e a r splines i n t e r p o l a t i n g d a t a at a g i v e n set o f p o i n t s i n
[0,1] consists o f t h e shifts o f t h e a b s o l u t e v a l u e f u n c t i o n t o t h e d a t a sites. I n o t h e r
w o r d s , we c a n c o n s t r u c t t h e piecewise l i n e a r s p l i n e i n t e r p o l a n t b y a s s u m i n g Vf is
of the form
N
V (x)
f = ^Tc \x k - x \,
k are [0,1],
fc=i
a n d t h e n d e t e r m i n e t h e coefficients c k by satisfying the interpolation conditions
V {xj)
f = h{ ), Xj j = l,...,N.
C l e a r l y , t h e basis f u n c t i o n s B k = |- x \ k are d e p e n d e n t o n t h e d a t a sites as suggested

by the Mairhuber-Curtis theorem. T h e points x k t o w h i c h t h e basic f u n c t i o n B(x)
\x\ is s h i f t e d are u s u a l l y referred t o as centers. W h i l e t h e r e m a y be circumstances
t h a t suggest c h o o s i n g these centers d i f f e r e n t f r o m t h e d a t a sites one g e n e r a l l y p i c k s
t h e centers t o c o i n c i d e w i t h t h e d a t a sites. T h i s simplifies the analysis of the
m e t h o d , a n d is sufficient for m a n y a p p l i c a t i o n s . Since t h e f u n c t i o n s B k are ( r a d i a l l y )
s y m m e t r i c a b o u t t h e i r centers x k t h i s c o n s t i t u t e s t h e first e x a m p l e o f radial basis
functions. We w i l l formally introduce the n o t i o n of a radial function i n the next
chapter.
O f course, one c a n i m a g i n e m a n y o t h e r w a y s t o c o n s t r u c t a n i V - d i m e n s i o n a l
d a t a - d e p e n d e n t basis for t h e p u r p o s e o f s c a t t e r e d d a t a i n t e r p o l a t i o n . H o w e v e r , t h e
use o f shifts o f one single basic f u n c t i o n m a k e s t h e r a d i a l basis f u n c t i o n a p p r o a c h
p a r t i c u l a r l y elegant.
N o t e t h a t we d i s t i n g u i s h b e t w e e n basis f u n c t i o n s B k a n d t h e basic f u n c t i o n B.
W e use t h i s t e r m i n o l o g y t o e m p h a s i z e t h a t t h e r e is one basic f u n c t i o n B which
generates t h e basis v i a shifts t o t h e v a r i o u s centers.
C o m i n g b a c k t o t h e s c a t t e r e d d a t a p r o b l e m , w e find t h e coefficients c k by solving
the linear system
\xi - X i | X2\ \Xi - X\

N ' Cl " "/l(*l)~
|x - Xi\ \X2 X2\ \X2 X\ /l(*2)

N
2
(1.3)
\X N - ml \X N - X2\ \x N - XN -CN _
A s m e n t i o n e d earlier, for h i g h e r space d i m e n s i o n s s s u c h a d a t a d e p e n d e n t basis

is r e q u i r e d . T h u s , even t h o u g h t h e c o n s t r u c t i o n o f piecewise l i n e a r splines i n h i g h e r
space d i m e n s i o n s is a different one ( t h e y are closely a s s o c i a t e d w i t h a n u n d e r l y i n g
c o m p u t a t i o n a l m e s h ) , t h e idea j u s t p r e s e n t e d suggests a v e r y s i m p l e g e n e r a l i z a t i o n
o f u n i v a r i a t e piecewise l i n e a r splines t h a t w o r k s for a n y space d i m e n s i o n .
T h e m a t r i x i n (1.3) above is a n e x a m p l e o f a distance matrix. S u c h m a t r i c e s have
been s t u d i e d i n g e o m e t r y a n d a n a l y s i s i n t h e c o n t e x t o f i s o m e t r i c e m b e d d i n g s o f
m e t r i c spaces for a l o n g t i m e (see, e.g., [ B a x t e r ( 1 9 9 1 ) ; B l u m e n t h a l ( 1 9 3 8 ) ; B o c h n e r
(1941); M i c c h e l l i ( 1 9 8 6 ) ; S c h o e n b e r g ( 1 9 3 8 a ) ; W e l l s a n d W i l l i a m s (1975)] a n d also
C h a p t e r 10). I t is k n o w n t h a t t h e d i s t a n c e m a t r i x based o n t h e E u c l i d e a n d i s t a n c e
S
b e t w e e n a set o f d i s t i n c t p o i n t s i n R is a l w a y s n o n - s i n g u l a r (see S e c t i o n 9.3 for
m o r e d e t a i l s ) . T h e r e f o r e , we c a n solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m we
s
posed o n [0, l ] b y assuming
Vf( ) x
= Y2 W Ck x
~ x
k h , xe[0,iy (1.4)
fc=i
1. Introduction 7
a n d t h e n d e t e r m i n e t h e coefficients c k by solving the linear system
1*1 Xi\\2 11*1 352112 1*1 - X \\

N 2 Cl /s(*l)
\x ~
2 Xi\\ 2 11*2 *2|| 2 \x 2 X \\
N 2 C2 fs{x2)
\XN - * 1 2 \\XN - X2\\2 \XN N\\2.

X
JS(XN)
T h i s is precisely t h e i n t e r p o l a t i o n m e t h o d w e w i l l choose t o i l l u s t r a t e w i t h o u r first

M A T L A B s c r i p t D i s t a n c e M a t r i x F i t .m (see P r o g r a m 1.2 b e l o w ) a n d t h e s u p p o r t i n g
figures a n d t a b l e s . A t y p i c a l basis f u n c t i o n for t h e E u c l i d e a n d i s t a n c e m a t r i x fit,
B {x)
k \\x Efc||2, is s h o w n i n F i g u r e 1.2 for t h e case xk 0 a n d s = 2.
Fig. 1.2 A typical basis function for the Euclidean distance matrix centered at the origin in R .
Before we discuss t h e a c t u a l i n t e r p o l a t i o n p r o g r a m we f i r s t l i s t a subroutine

used i n m a n y o f o u r l a t e r e x a m p l e s . I t is c a l l e d D i s t a n c e M a t r i x . m a n d w e use i t
t o compute the m a t r i x o f pairwise E u c l i d e a n distances o f t w o (possibly different)
s
sets o f p o i n t s i n M . I n t h e c o d e these t w o sets are d e n o t e d b y d s i t e s a n d c t r s . I n
m o s t o f o u r e x a m p l e s b o t h o f these sets w i l l c o i n c i d e w i t h t h e set X o f d a t a sites.
Program 1.1. D i s t a n c e M a t r i x . m
% DM = DistanceMatrix(dsites,ctrs)
% Forms t h e d i s t a n c e m a t r i x of two s e t s o f p o i n t s in R"s,
% i.e., DM(i,j) = II datasite_i - center_j I I_2.
% Input
% dsites: Mxs m a t r i x r e p r e s e n t i n g a set of M data s i t e s in R~s
7, (i.e., each row c o n t a i n s one s-dimensional point)
7o ctrs: Nxs m a t r i x r e p r e s e n t i n g a set of N centers in R~s
7o (one center per row)
7. O u t p u t
7. DM: MxN m a t r i x w h o s e i , j p o s i t i o n c o n t a i n s the Euclidean
7o d i s t a n c e between the i - t h data site and j - t h c e n t e r
1 f u n c t i o n DM = D i s t a n c e M a t r i x ( d s i t e s , c t r s )
2 [M,s] = s i z e ( d s i t e s ) ; [N,s] = size(ctrs);
3 DM = zeros(M,N);
% Accumulate sum of s q u a r e s of c o o r d i n a t e differences
% The n d g r i d command produces two MxN matrices:
'/ . d r , c o n s i s t i n g of N i d e n t i c a l columns ( e a c h containing
% the d-th coordinate of t h e M d a t a sites)
7o c c , c o n s i s t i n g of M i d e n t i c a l rows ( e a c h containing
% the d-th coordinate of t h e N c e n t e r s )
4 f o r d=l:s
5 [dr.cc] = n d g r i d ( d s i t e s ( : , d ) , c t r s ( : , d ) ) ;
6 DM = DM + ( d r - c c ) . " 2 ;
7 end
8 DM = sqrt(DM);
N o t e t h a t t h i s s u b r o u t i n e c a n easily b e m o d i f i e d t o p r o d u c e a p - n o r m d i s t a n c e
m a t r i x b y m a k i n g t h e o b v i o u s changes t o lines 6 a n d 8 o f t h e code i n P r o g r a m 1.1.
W e w i l l come back t o t h i s idea i n C h a p t e r 10.
O u r first m a i n s c r i p t is P r o g r a m 1.2. T h i s s c r i p t c a n be u s e d t o c o m p u t e the
distance m a t r i x interpolant to d a t a sampled f r o m the test function / s provided by
P r o g r a m C l . W e use H a l t o n p o i n t s a n d are a b l e t o select t h e space d i m e n s i o n
s and number of points N b y e d i t i n g lines 1 a n d 2 o f t h e code. The subrou-
t i n e MakeSDGrid.m w h i c h w e use t o c o m p u t e t h e e q u a l l y spaced p o i n t s i n t h e s-
dimensional u n i t cube o n line 6 o f D i s t a n c e M a t r i x F i t .mis p r o v i d e d i n A p p e n d i x C.
T h e s e e q u a l l y spaced p o i n t s are used as e v a l u a t i o n p o i n t s a n d t o c o m p u t e errors.
N o t e t h a t since t h e d i s t a n c e m a t r i x i n t e r p o l a n t is o f t h e f o r m ( 1 . 4 ) i t s s i m u l t a n e o u s
e v a l u a t i o n a t t h e e n t i r e set o f e v a l u a t i o n p o i n t s a m o u n t s t o a m a t r i x - v e c t o r p r o d u c t
o f t h e e v a l u a t i o n m a t r i x EM a n d t h e coefficients c. H e r e t h e e v a l u a t i o n m a t r i x has
t h e same s t r u c t u r e as t h e i n t e r p o l a t i o n m a t r i x a n d c a n also be c o m p u t e d u s i n g t h e
subroutine Distancematrix.m ( o n l y u s i n g e v a l u a t i o n p o i n t s i n place o f t h e d a t a
sites, see l i n e 9 o f D i s t a n c e M a t r i x F i t .m). T h e coefficient v e c t o r c is s u p p l i e d d i -
r e c t l y as s o l u t i o n o f t h e l i n e a r s y s t e m Ac = f (see ( 1 . 3 ) a n d t h e M A T L A B e x p r e s s i o n
IM\rhs o n l i n e 1 0 o f t h e p r o g r a m ) . T h e e v a l u a t i o n p o i n t s are s u b s e q u e n t l y used
for t h e e r r o r c o m p u t a t i o n i n lines 1 1 - 1 3 a n d are also u s e d for p l o t t i n g p u r p o s e s i n
t h e last p a r t o f t h e p r o g r a m (lines 1 6 - 3 5 ) . N o t e t h a t for t h i s e x a m p l e w e k n o w t h e
function f s t h a t g e n e r a t e d t h e d a t a , a n d t h e r e f o r e are a b l e t o c o m p u t e t h e e r r o r i n
o u r r e c o n s t r u c t i o n . T h e s u b r o u t i n e s t h a t p r o d u c e t h e 2 D a n d 3 D p l o t s o n lines 2 4
3 2 are p r o v i d e d i n A p p e n d i x C . N o t e t h a t t h e use o f r e s h a p e o n lines 2 2 - 2 3 and
2 7 - 2 9 c o r r e s p o n d s t o t h e use o f meshgrid for p l o t t i n g p u r p o s e s .
P r o g r a m 1.2. D i s t a n c e M a t r i x F i t .m
% DistanceMatrixFit
% S c r i p t that uses E u c l i d e a n d i s t a n c e matrices t o perform
1. Introduction
% s c a t t e r e d d a t a i n t e r p o l a t i o n f o r a r b i t r a r y space dimensions
% C a l l s on: D i s t a n c e M a t r i x , MakeSDGrid, t e s t f u n c t i o n
/ Uses:
0 h a l t o n s e q ( w r i t t e n by D a n i e l Dougherty from MATLAB
% C e n t r a l F i l e Exchange)
1 s = 3;
2 k = 2; N = ( 2 ~ k + l ) ~ s ;
3 n e v a l = 10; M = n e v a l ' s ;
% Use Halton p o i n t s a s d a t a s i t e s and c e n t e r s
4 d s i t e s = haltonseq(N,s);
5 ctrs = dsites;
% Create n e v a l ~ s e q u a l l y spaced e v a l u a t i o n l o c a t i o n s i n t h e
% s-dimensional u n i t cube
6 e p o i n t s = MakeSDGrid(s,neval);
% Create right-hand side vector,
% i . e . , evaluate the t e s t function a t the data s i t e s
7 rhs= testfunction(s,dsites);
% Compute d i s t a n c e m a t r i x f o r t h e d a t a s i t e s and c e n t e r s
8 IM = D i s t a n c e M a t r i x ( d s i t e s , c t r s ) ;
% Compute d i s t a n c e m a t r i x f o r e v a l u a t i o n p o i n t s and c e n t e r s
9 EM = D i s t a n c e M a t r i x ( e p o i n t s , c t r s ) ;
% E v a l u a t e t h e i n t e r p o l a n t on e v a l u a t i o n p o i n t s
/ ( e v a l u a t i o n m a t r i x * s o l u t i o n of i n t e r p o l a t i o n system)
0
10 Pf = EM * ( I M \ r h s ) ;
% Compute exact s o l u t i o n ,
% i . e . , e v a l u a t e t e s t f u n c t i o n on e v a l u a t i o n p o i n t s
11 exact = t e s t f u n c t i o n ( s , e p o i n t s ) ;
% Compute maximum and RMS e r r o r s on e v a l u a t i o n g r i d
12 maxerr = n o r m ( P f - e x a c t , i n f ) ;
13 rms.err = n o r m ( P f - e x a c t ) / s q r t ( M ) ;
J
14 f p r i n t f ( R M S e r r o r : 7 e\n', r m s _ e r r )
0
0
15 f p r i n t f ('Maximum e r r o r : / e\n', maxerr)
0
16 s w i t c h s
17 case 1
18 plot(epoints, Pf)
19 figure; plot(epoints, abs(Pf-exact))
20 case 2
21 fview = [-30,30];
22 xe = r e s h a p e ( e p o i n t s ( : , 2 ) , n e v a l , n e v a l ) ;
23 ye = r e s h a p e ( e p o i n t s ( : , 1 ) , n e v a l , n e v a l ) ;
24 PlotSurf(xe,ye,Pf,neval,exact.maxerr,fview);
25 PlotError2D(xe,ye,Pf,exact,maxerr,neval,fview);
26 case 3
10 Meshfree Approximation Methods with M A T L A B
27 xe = reshapeCepoints(:,2),neval,neval,neval);
38 ye = reshape(epoints(:,1),neval,neval,neval);
29 ze = reshape(epoints(:,3),neval,neval,neval);
30 xslice = .25:.25:1; yslice = 1; zslice = [0,0.5];
31 PlotSIices(xe,ye,ze,Pf,neval,xslice,yslice,zslice);
32a PlotErrorSlices(xe,ye,ze,Pf,exact,neval,...
32b xslice,yslice,zslice);
33 otherwise
34 disp('Cannot display plots for s>3')
35 end
I n T a b l e s 1.1 a n d 1.2 as w e l l as F i g u r e s 1.3 a n d 1.4 w e p r e s e n t some e x a m p l e s

c o m p u t e d w i t h P r o g r a m 1.2. The number M of evaluation points (determined by
n e v a l o n l i n e 3 o f t h e code) w e used for t h e cases s = 1, 2 , . . . , 6, was 1000, 1600,
1000, 256, 1024, a n d 4096, r e s p e c t i v e l y {i.e., neval = 1000, 40, 10, 4, 4 , a n d 4,
r e s p e c t i v e l y ) . N o t e t h a t , as t h e space d i m e n s i o n s increases, m o r e a n d m o r e o f t h e
e v a l u a t i o n p o i n t s lie o n t h e b o u n d a r y o f t h e d o m a i n , w h i l e t h e d a t a sites ( w h i c h are
g i v e n as H a l t o n p o i n t s ) are l o c a t e d i n t h e i n t e r i o r o f t h e d o m a i n . T h e v a l u e k l i s t e d
i n Tables 1.1 a n d 1.2 is t h e same as t h e k i n l i n e 2 o f P r o g r a m 1.2. T h e f o r m u l a for
t h e r o o t - m e a n - s q u a r e e r r o r ( R M S - e r r o r ) is g i v e n b y
RMS-error (1.5)
w h e r e t h e j, j = 1 , . . . , M are t h e evaluation points. F o r m u l a (1.5) is used o n

l i n e 13 o f P r o g r a m 1.2.
T h e basic M A T L A B code for t h e s o l u t i o n o f a n y k i n d o f R B F i n t e r p o l a t i o n p r o b -
l e m w i l l be v e r y s i m i l a r t o P r o g r a m 1.2. N o t e i n p a r t i c u l a r t h a t t h e d a t a u s e d
even for t h e d i s t a n c e m a t r i x i n t e r p o l a t i o n c o n s i d e r e d h e r e c a n also be "real"
data. I n t h a t case one s i m p l y needs t o r e p l a c e lines 4 a n d 7 o f t h e p r o g r a m b y
a p p r o p r i a t e code t h a t generates t h e d a t a sites a n d d a t a values for t h e r i g h t - h a n d
side.
T h e p l o t s o n t h e left o f F i g u r e s 1.3 a n d 1.4 d i s p l a y t h e g r a p h s o f t h e d i s t a n c e
m a t r i x fits for space d i m e n s i o n s s = 1, 2, a n d 3, r e s p e c t i v e l y , w h i l e t h o s e o n t h e
right depict the corresponding errors. F o r t h e I D p l o t s ( i n F i g u r e 1.3) w e u s e d
5 H a l t o n p o i n t s t o i n t e r p o l a t e t h e t e s t f u n c t i o n / i . T h e piecewise l i n e a r n a t u r e o f
t h e i n t e r p o l a n t is c l e a r l y v i s i b l e a t t h i s r e s o l u t i o n . I f w e use m o r e p o i n t s t h e n t h e
fit becomes m o r e a c c u r a t e see T a b l e 1.1 b u t t h e n i t is n o l o n g e r p o s s i b l e t o
d i s t i n g u i s h t h e piecewise l i n e a r n a t u r e o f t h e i n t e r p o l a n t . T h e 2 D p l o t ( t o p left o f
F i g u r e 1.4) i n t e r p o l a t e s t h e t e s t f u n c t i o n f 2 a t 289 H a l t o n p o i n t s . T h e g r a p h o f Vf is
false-colored a c c o r d i n g t o t h e a b s o l u t e e r r o r ( i n d i c a t e d b y t h e c o l o r b a r a t t h e r i g h t
o f t h e p l o t ) . T h e b o t t o m p l o t i n F i g u r e 1.4 shows a slice p l o t o f t h e d i s t a n c e m a t r i x
i n t e r p o l a n t t o fa b a s e d o n 729 H a l t o n p o i n t s . For this p l o t the colors represent
f u n c t i o n values ( a g a i n i n d i c a t e d b y t h e c o l o r b a r o n t h e r i g h t ) .
1. Introduction 11
s
Table 1.1 Distance matrix fit to N Halton points in [0, l ] , s 1, 2, 3.
ID 2D 3D
k N RMS-error N RMS-error N RMS-error
1 3 5.896957e-001 9 1.937341e-001 27 9.721476e-002

2 5 3.638027e-001 25 6.336315e-002 125 6.277141e-002
3 9 1.158328e-001 81 2.349093e-002 729 2.759452e-002
4 17 3.981270e-002 289 1.045010e-002
5 33 1.406188e-002 1089 4.326940e-003
6 65 5.068541e-003 4225 1.797430e-003
7 129 1.877013e-003
8 257 7.264159e-004
9 513 3.016376e-004
10 1025 1.381896e-004
11 2049 6.907386e-005
12 4097 3.453179e-005
X X
Fig. 1.3 F i t (left) and absolute error (right) for 5 point distance matrix interpolation in I D .
I n t h e r i g h t h a l f o f F i g u r e s 1.3 a n d 1.4 w e s h o w a b s o l u t e e r r o r s for t h e d i s t a n c e

m a t r i x i n t e r p o l a n t s d i s p l a y e d i n t h e left c o l u m n . W e use a n a l o g o u s c o l o r schemes,
i.e., t h e 2 D p l o t ( t o p p a r t o f F i g u r e 1.4) is false-colored a c c o r d i n g t o t h e a b s o l u t e
e r r o r , a n d so is t h e 3 D p l o t ( b o t t o m ) since n o w t h e " f u n c t i o n v a l u e " corresponds
to the absolute error. W e c a n see c l e a r l y t h a t m o s t o f t h e e r r o r is c o n c e n t r a t e d
near t h e b o u n d a r y o f t h e d o m a i n . I n f a c t , t h e a b s o l u t e e r r o r is a b o u t one o r d e r o f
m a g n i t u d e l a r g e r n e a r t h e b o u n d a r y t h a n i t is i n t h e i n t e r i o r o f t h e d o m a i n . T h i s
is n o s u r p r i s e since t h e d a t a sites are l o c a t e d i n t h e i n t e r i o r . H o w e v e r , even f o r
u n i f o r m l y spaced d a t a sites ( i n c l u d i n g p o i n t s o n t h e b o u n d a r y ) t h e m a i n e r r o r i n
r a d i a l basis f u n c t i o n i n t e r p o l a t i o n is u s u a l l y l o c a t e d near t h e b o u n d a r y .
F r o m t h i s f i r s t s i m p l e e x a m p l e w e c a n observe a n u m b e r o f o t h e r f e a t u r e s . Most
o f t h e m are c h a r a c t e r i s t i c f o r t h e r a d i a l basis f u n c t i o n i n t e r p o l a n t s w e w i l l be s t u d y -
i n g l a t e r o n . F i r s t , t h e basis f u n c t i o n s e m p l o y e d , B k = || a?&||2> are r a d i a l l y s y m -
y 0 0 x y 0 0
Fig. 1.4 F i t s (left) and errors (right) for distance matrix interpolation with 289 points in 2D
(top), and 729 points in 3D (bottom).
s
Table 1.2 Distance matrix fit to N Halton points in [0, l ] , s 4, 5, 6.
4D 5D 6D
1 81 1.339581e-001 243 9.558350e-002 729 5.097600e-002

2 625 6.817424e-002 3125 3.118905e-002
metric. Second, as t h e M A T L A B s c r i p t s s h o w , t h e m e t h o d is e x t r e m e l y s i m p l e t o
i m p l e m e n t for a n y space d i m e n s i o n s. For example, no u n d e r l y i n g c o m p u t a t i o n a l
m e s h is r e q u i r e d t o c o m p u t e t h e i n t e r p o l a n t . T h e process o f m e s h g e n e r a t i o n is
a m a j o r f a c t o r w h e n w o r k i n g i n h i g h e r space d i m e n s i o n s w i t h polynomial-based
m e t h o d s such as splines o r finite e l e m e n t s . A l l t h a t is r e q u i r e d for o u r m e t h o d is
t h e p a i r w i s e d i s t a n c e b e t w e e n t h e d a t a sites. T h e r e f o r e , w e h a v e w h a t is k n o w n as
a meshfree (or meshless) method.
T h i r d , t h e a c c u r a c y o f t h e m e t h o d i m p r o v e s i f w e a d d m o r e d a t a sites. I n f a c t ,
i t seems t h a t t h e R M S - e r r o r i n T a b l e s 1.1 a n d 1.2 is r e d u c e d b y a f a c t o r o f a b o u t
s
t w o f r o m one r o w t o t h e n e x t . Since w e use (2* + l ) uniformly distributed random
1. Introduction 13
d a t a p o i n t s i n r o w k t h i s i n d i c a t e s a convergence r a t e o f r o u g h l y O(h), where h can

be v i e w e d as s o m e t h i n g l i k e t h e average d i s t a n c e o r meshsize o f t h e set X o f d a t a
sites (we w i l l be m o r e precise l a t e r o n ) .
A n o t h e r t h i n g t o n o t e is t h a t t h e s i m p l e d i s t a n c e f u n c t i o n i n t e r p o l a n t used here
(as w e l l as m a n y o t h e r r a d i a l basis f u n c t i o n i n t e r p o l a n t s used l a t e r ) r e q u i r e s the
s o l u t i o n o f a s y s t e m o f l i n e a r e q u a t i o n s w i t h a dense N x N m a t r i x . T h i s makes
i t v e r y c o s t l y t o a p p l y t h e m e t h o d i n i t s s i m p l e f o r m t o l a r g e d a t a sets. M o r e o v e r ,
as we w i l l see l a t e r , these m a t r i c e s also t e n d t o be r a t h e r i l l - c o n d i t i o n e d . T h e s e are
t h e reasons w h y we c a n o n l y present r e s u l t s for r e l a t i v e l y s m a l l d a t a sets i n h i g h e r
space d i m e n s i o n s u s i n g t h i s s i m p l e a p p r o a c h .
I n t h e r e m a i n d e r o f t h i s b o o k i t is o u r g o a l t o present a l t e r n a t i v e s t o t h i s basic
i n t e r p o l a t i o n m e t h o d t h a t address t h e p r o b l e m s m e n t i o n e d a b o v e such as l i m i t a t i o n
t o s m a l l d a t a sets, i l l - c o n d i t i o n i n g , l i m i t e d a c c u r a c y a n d l i m i t e d s m o o t h n e s s o f t h e
interpolant.
1.2 Some Historical R e m a r k s
O r i g i n a l l y , t h e m o t i v a t i o n for t h e basic meshfree approximation methods (ra-

d i a l basis f u n c t i o n a n d m o v i n g least squares m e t h o d s ) c a m e f r o m a p p l i c a t i o n s i n
geodesy, geophysics, m a p p i n g , o r m e t e o r o l o g y . L a t e r , applications were f o u n d i n
m a n y o t h e r areas such as i n t h e n u m e r i c a l s o l u t i o n o f P D E s , c o m p u t e r graph-
ics, artificial intelligence, statistical learning theory, neural networks, signal a n d
i m a g e processing, sampling theory, statistics (kriging), finance, and optimiza-
tion. I t s h o u l d be p o i n t e d o u t t h a t meshfree l o c a l regression m e t h o d s have b e e n
used i n d e p e n d e n t l y i n s t a t i s t i c s for w e l l over 100 years (see, e.g., [Cleveland and
L o a d e r (1996)] a n d t h e references t h e r e i n ) . I n fact, t h e basic m o v i n g least squares
m e t h o d ( k n o w n also as l o c a l regression i n t h e s t a t i s t i c s l i t e r a t u r e ) c a n be t r a c e d
back at least t o t h e w o r k o f [ G r a m ( 1 8 8 3 ) ; W o o l h o u s e ( 1 8 7 0 ) ; D e Forest ( 1 8 7 3 ) ;
D e Forest ( 1 8 7 4 ) ] .
I n t h e l i t e r a t u r e o n a p p r o x i m a t i o n t h e o r y a n d r e l a t e d a p p l i c a t i o n s areas some
h i s t o r i c a l l a n d m a r k c o n t r i b u t i o n s have come f r o m
D o n a l d S h e p a r d , w h o as a n u n d e r g r a d u a t e s t u d e n t a t H a r v a r d U n i v e r s i t y ,
suggested t h e use o f w h a t are n o w c a l l e d Shepard functions in the late
1960s (see C h a p t e r 22). T h e p u b l i c a t i o n [Shepard (1968)] discusses t h e
basic inverse d i s t a n c e w e i g h t e d S h e p a r d m e t h o d a n d some m o d i f i c a t i o n s
thereof. T h e m e t h o d was a t t h e t i m e i n c o r p o r a t e d i n t o a c o m p u t e r p r o -
g r a m , S Y M A P , for m a p m a k i n g .
R o l l a n d H a r d y , w h o was a geodesist at I o w a S t a t e U n i v e r s i t y . H e i n t r o -
d u c e d t h e so-called multiquadrics ( M Q s ) i n t h e e a r l y 1970s (see, e.g., [ H a r d y
(1971)] or C h a p t e r 8 ) . H a r d y ' s w o r k was p r i m a r i l y c o n c e r n e d w i t h a p p l i -
c a t i o n s i n geodesy a n d m a p p i n g .
Meshfree Approximation Methods with MATLAB
R o b e r t L . H a r d e r a n d R o b e r t N . D e s m a r a i s , w h o w e r e aerospace engineers
at M a c N e a l - S c h w e n d l e r C o r p o r a t i o n ( M S C S o f t w a r e ) , a n d N A S A ' s L a n g l e y
Research C e n t e r . T h e y i n t r o d u c e d t h e s o - c a l l e d thin plate splines (TPSs)
i n 1972 (see, e.g., [ H a r d e r a n d D e s m a r a i s ( 1 9 7 2 ) ] o r C h a p t e r 8 ) . T h e i r w o r k
was c o n c e r n e d m o s t l y w i t h a i r c r a f t d e s i g n .
Jean Duchon, a m a t h e m a t i c i a n at the Universite Joseph Fourier i n Greno-
ble, France. D u c h o n suggested a v a r i a t i o n a l a p p r o a c h minimizing the
2 2
integral of V / in R w h i c h also leads t o t h e t h i n p l a t e splines. This
w o r k was d o n e i n t h e m i d 1970s a n d is c o n s i d e r e d t o b e t h e foundation
o f t h e v a r i a t i o n a l a p p r o a c h t o r a d i a l basis f u n c t i o n s (see [ D u c h o n ( 1 9 7 6 ) ;
D u c h o n (1977); D u c h o n (1978); D u c h o n (1980)]) or C h a p t e r 13).
Jean Meinguet, a mathematican a t U n i v e r s i t e C a t h o l i q u e de L o u v a i n i n
L o u v a i n , B e l g i u m . M e i n g u e t i n t r o d u c e d w h a t he c a l l e d surface splines in
t h e l a t e 1970s. Surface splines a n d t h i n p l a t e splines f a l l u n d e r w h a t w e
w i l l refer t o as polyharmonic splines (see, e.g., [ M e i n g u e t ( 1 9 7 9 a ) ; M e i n g u e t
(1979b); M e i n g u e t (1979c); M e i n g u e t (1984)] or C h a p t e r 8).
P e t e r L a n c a s t e r a n d K e s Salkauskas, m a t h e m a t i c i a n s at t h e U n i v e r s i t y o f
Calgary, Canada. T h e y p u b l i s h e d [ L a n c a s t e r a n d Salkauskas ( 1 9 8 1 ) ] i n -
t r o d u c i n g t h e moving least squares method (a g e n e r a l i z a t i o n o f S h e p a r d
functions).
R i c h a r d Franke, a m a t h e m a t i c i a n at the N a v a l Postgraduate School i n M o n -
terey, C a l i f o r n i a . I n [ F r a n k e (1982a)] he c o m p a r e d v a r i o u s s c a t t e r e d d a t a
i n t e r p o l a t i o n methods, a n d concluded M Q s a n d T P S s were the best. Franke
also c o n j e c t u r e d t h a t t h e i n t e r p o l a t i o n m a t r i x for M Q s is i n v e r t i b l e .
W o l o d y m y r ( W a l l y ) M a d y c h , a m a t h e m a t i c i a n at the U n i v e r s i t y of C o n -
necticut, and Stuart A l a n Nelson, a m a t h e m a t i c i a n from Iowa State Univer-
sity. I n 1983 t h e y c o m p l e t e d t h e i r m a n u s c r i p t [ M a d y c h a n d N e l s o n (1983)]
i n w h i c h t h e y p r o v e d Franke's conjecture ( a n d m u c h more) based o n a varia-
t i o n a l approach. However, t h i s m a n u s c r i p t was never published. O t h e r fun-
d a m e n t a l p a p e r s b y these t w o a u t h o r s are, e.g., [ M a d y c h a n d N e l s o n ( 1 9 8 8 ) ;
M a d y c h and Nelson (1990a); M a d y c h and Nelson (1992)].
Charles Micchelli, a m a t h e m a t i c i a n at the I B M W a t s o n Research Center.
M i c c h e l l i p u b l i s h e d t h e p a p e r [ M i c c h e l l i ( 1 9 8 6 ) ] . H e also p r o v e d F r a n k e ' s
conjecture. H i s p r o o f s are r o o t e d i n t h e w o r k o f [ B o c h n e r ( 1 9 3 2 ) ; B o c h n e r
(1933)] a n d [Schoenberg ( 1 9 3 7 ) ; S c h o e n b e r g ( 1 9 3 8 a ) ; S c h o e n b e r g ( 1 9 3 8 b ) ]
on positive definite a n d c o m p l e t e l y m o n o t o n e functions. T h i s is also t h e
approach we w i l l follow t h r o u g h o u t m u c h of this book.
G r a c e W a h b a , a s t a t i s t i c i a n a t t h e U n i v e r s i t y o f W i s c o n s i n . She s t u d i e d t h e
use o f t h i n p l a t e splines for s t a t i s t i c a l p u r p o s e s i n t h e c o n t e x t o f s m o o t h i n g
noisy data a n d d a t a o n spheres, a n d i n t r o d u c e d t h e A N O V A a n d cross
v a l i d a t i o n a p p r o a c h e s t o t h e r a d i a l basis f u n c t i o n s e t t i n g ( s e e , e.g., [Wahba
1. Introduction 15
(1979); W a h b a ( 1 9 8 1 ) ; W a h b a a n d W e n d e l b e r g e r ( 1 9 8 0 ) ] ) . O n e o f t h e first
m o n o g r a p h s o n t h e s u b j e c t is [ W a h b a ( 1 9 9 0 b ) ] .
R o b e r t Schaback, a m a t h e m a t i c i a n a t t h e U n i v e r s i t y o f G o t t i n g e n , Ger-
many. Compactly supported radial basis functions ( C S R B F s ) were i n t r o -
d u c e d i n [Schaback ( 1 9 9 5 a ) ] , a n d a v e r y p o p u l a r f a m i l y o f C S R B F s was
presented b y H o l g e r W e n d l a n d (also a m a t h e m a t i c i a n i n G o t t i n g e n ) i n his
P h . D . thesis (see also [ W e n d l a n d (1995)] a n d C h a p t e r 1 1 ) . B o t h o f these
a u t h o r s have c o n t r i b u t e d e x t e n s i v e l y t o t h e field o f r a d i a l basis f u n c t i o n s .
W e m e n t i o n p a r t i c u l a r l y t h e recent m o n o g r a p h [ W e n d l a n d ( 2 0 0 5 a ) ] .
Chapter 2
Radial Basis Function Interpolation

in MATLAB
Before we discuss a n y o f t h e t h e o r e t i c a l f o u n d a t i o n o f r a d i a l basis f u n c t i o n s w e w a n t

t o get a feel for w h a t t h e y are a l l a b o u t . W e saw i n t h e i n t r o d u c t o r y c h a p t e r t h a t
i t is easy t o use E u c l i d e a n d i s t a n c e m a t r i c e s t o c o m p u t e a s o l u t i o n t o t h e s c a t t e r e d
d a t a i n t e r p o l a t i o n p r o b l e m . H o w e v e r , we also p o i n t e d o u t a n u m b e r o f l i m i t a t i o n s
t o t h a t a p p r o a c h such as t h e l i m i t e d a c c u r a c y a n d l i m i t e d s m o o t h n e s s . I t t u r n s o u t
t h a t we c a n m a i n t a i n t h e u n d e r l y i n g s t r u c t u r e presented b y t h e d i s t a n c e m a t r i x
a p p r o a c h a n d address these l i m i t a t i o n s b y c o m p o s i n g t h e d i s t a n c e f u n c t i o n w i t h
certain "good" univariate functions.
2.1 Radial (Basis) Functions
A s a first e x a m p l e we p i c k a f u n c t i o n w e l l - r e p r e s e n t e d i n m a n y branches o f m a t h e -
matics, namely the Gaussian
2
^( )= -(^) ,
r e r e i
2
O u r shape parameter e is r e l a t e d t o t h e v a r i a n c e a of the normal distribution
2 2
function by e = l/(2o~ ). I f we c o m p o s e t h e G a u s s i a n w i t h t h e E u c l i d e a n d i s t a n c e
s
f u n c t i o n 11 - j12 we o b t a i n for a n y f i x e d center x k GR a multivariate function
3
$ (sc) = e - " -
fc
e a a ! a ? f c
ll2, x e R .
Obviously, the connection between a n d (p is g i v e n b y
^fc(x) = (p(\\x - x \\ ).
k 2
I t is t h i s c o n n e c t i o n t h a t gives rise t o t h e n a m e radial basis function (RBF). The

f o l l o w i n g is a f o r m a l d e f i n i t i o n o f a r a d i a l f u n c t i o n .
S
D e f i n i t i o n 2.1. A f u n c t i o n $ : R > R is c a l l e d radial p r o v i d e d t h e r e exists a
univariate f u n c t i o n R s u c h t h a t
&(x) = (f(r), where r \\x\\,
a n d || || is some n o r m o n M u s u a l l y t h e E u c l i d e a n n o r m .
s
17
D e f i n i t i o n 2.1 says t h a t for a r a d i a l f u n c t i o n <&

s
||a;i|| = ||x2|| $(JCI) = $(x ), 2 x u x eR .
2
I n other words, the value o f a t a n y p o i n t a t a c e r t a i n fixed d i s t a n c e f r o m t h e o r i g i n

(or a n y o t h e r fixed center p o i n t ) is c o n s t a n t . T h u s , <3> is r a d i a l l y (or s p h e r i c a l l y )
s y m m e t r i c a b o u t i t s center. D e f i n i t i o n 2 . 1 s h o w s t h a t t h e E u c l i d e a n d i s t a n c e func-
t i o n we used i n t h e i n t r o d u c t i o n is j u s t a s p e c i a l case o f a r a d i a l (basis) f u n c t i o n .
N a m e l y , w i t h ip(r) = r.
F i g u r e 2.1 shows t h e g r a p h s o f t w o G a u s s i a n r a d i a l basis f u n c t i o n s , one w i t h
shape p a r a m e t e r e 1 (left) a n d one w i t h e 3 ( r i g h t ) ( b o t h c e n t e r e d a t t h e o r i g i n
2
i n R ) . A s m a l l e r v a l u e o f e (i.e., l a r g e r v a r i a n c e ) causes t h e f u n c t i o n t o b e c o m e
" f l a t t e r " , w h i l e i n c r e a s i n g e leads t o a m o r e p e a k e d R B F , a n d t h e r e f o r e localizes i t s
influence. W e w i l l see s o o n t h a t t h e choice o f e has a p r o f o u n d i n f l u e n c e o n b o t h
the accuracy and numerical s t a b i l i t y o f the s o l u t i o n t o our i n t e r p o l a t i o n p r o b l e m .
u is
2
Fig. 2.1 Gaussian with e = 1 (left) and e = 3 (right) centered at the origin in R .
D e f i n i t i o n 2.1 a n d t h e d i s c u s s i o n l e a d i n g u p t o i t s h o w a g a i n w h y i t m a k e s sense
t o c a l l ip t h e basic f u n c t i o n , a n d &k(\\ H2) (centered at x ) k a r a d i a l basis f u n c t i o n .
O n e single basic f u n c t i o n generates a l l o f t h e basis f u n c t i o n s t h a t are u s e d i n t h e
expansion (1.1).
R a d i a l f u n c t i o n i n t e r p o l a n t s h a v e t h e nice p r o p e r t y o f b e i n g i n v a r i a n t u n d e r a l l
Euclidean transformations (i.e., translations, rotations, a n d reflections). B y this
w e m e a n t h a t i t does n o t m a t t e r w h e t h e r w e first c o m p u t e t h e R B F i n t e r p o l a n t
a n d t h e n a p p l y a E u c l i d e a n t r a n s f o r m a t i o n , o r i f w e first t r a n s f o r m t h e d a t a a n d
t h e n c o m p u t e t h e i n t e r p o l a n t . T h i s is a n i m m e d i a t e consequence o f t h e fact t h a t
E u c l i d e a n t r a n s f o r m a t i o n s are c h a r a c t e r i z e d by orthogonal transformation matri-
ces a n d are t h e r e f o r e 2 - n o r m - i n v a r i a n t . I n v a r i a n c e u n d e r t r a n s l a t i o n , r o t a t i o n a n d
r e f l e c t i o n is o f t e n d e s i r a b l e i n a p p l i c a t i o n s .
Moreover, the application of radial functions to the solution of the scattered data
i n t e r p o l a t i o n p r o b l e m (as w e l l as m a n y o t h e r m u l t i v a r i a t e a p p r o x i m a t i o n p r o b l e m s )
benefits f r o m t h e fact t h a t t h e i n t e r p o l a t i o n p r o b l e m b e c o m e s i n s e n s i t i v e t o the
2. Radial Basis Function Interpolation in M A T L A B 19
d i m e n s i o n s o f t h e space i n w h i c h t h e d a t a sites l i e . I n s t e a d o f h a v i n g t o d e a l w i t h
a multivariate function $ (whose c o m p l e x i t y w i l l increase w i t h i n c r e a s i n g space
d i m e n s i o n s) w e c a n w o r k w i t h t h e same u n i v a r i a t e f u n c t i o n ip for a l l choices o f s.
2.2 Radial Basis Function Interpolation
I n s t e a d o f u s i n g s i m p l e d i s t a n c e m a t r i c e s as w e d i d earlier, w e n o w use a r a d i a l
s
basis f u n c t i o n e x p a n s i o n t o solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m i n M. b y
assuming
N
s
V (x)
f = J2ck<p(\\x-x \\ ), k 2 x e R . (2.1)
fc=i
T h e coefficients c k are f o u n d b y e n f o r c i n g t h e i n t e r p o l a t i o n c o n d i t i o n s , a n d t h u s
solving the linear system
p(\\x X - Xi|| ) 2 (p (\\x 1 - X \\ )

2 2 <p(\\Xi - X \\ )
N 2
tp (\\x 2 - SC1U2) <P (\\x 2 - x \\ ) 2 2 ... ip(\\x 2 - X \\ )

N 2 C2 /(*2)
=
_ip(\\x N - xi\\ ) 2 (p(\\x

N - X2W2) f(\\x N - x \\ )
N 2
A s t h e s o l u t i o n o f t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m hinges e n t i r e l y o n t h e
solution of this system of linear equations we w i l l devote t h e next chapter t o the
q u e s t i o n o f w h e n (i.e., for w h a t t y p e o f basic f u n c t i o n s <p) t h e s y s t e m m a t r i x is
non-singular.
F o r t h e n u m e r i c a l e x a m p l e p r e s e n t e d b e l o w w e r e s t r i c t ourselves t o t h e t w o -
d i m e n s i o n a l case s = 2. A s basic f u n c t i o n ip w e w i l l use b o t h Gaussians a n d t h e
l i n e a r f u n c t i o n <p(r) = r w h i c h gives rise t o t h e E u c l i d e a n d i s t a n c e m a t r i x a p p r o a c h
used earlier.
T h e code o f t h e MATLAB script RBFInterpolation2D.m (see P r o g r a m 2.1)
w e use t o p e r f o r m R B F i n t e r p o l a t i o n i n 2 D is v e r y s i m i l a r t o t h e earlier s c r i p t
D i s t a n c e M a t r i x F i t .m. I t also m a k e s use o f t h e s u b r o u t i n e D i s t a n c e M a t r i x . m .
W h i l e i t is easy t o w r i t e a v e r s i o n o f t h e i n t e r p o l a t i o n s c r i p t t h a t w o r k s for a n y
space d i m e n s i o n s ( j u s t as w e d i d i n D i s t a n c e M a t r i x F i t .m) w e w i l l s t i c k w i t h a
basic 2 D v e r s i o n here.
I n l i n e 1 we define t h e G a u s s i a n R B F as a M A T L A B a n o n y m o u s f u n c t i o n t h a t
accepts a m a t r i x a r g u m e n t ( n a m e l y t h e o u t p u t f r o m D i s t a n c e M a t r i x ) a l o n g w i t h
its shape p a r a m e t e r . N o t e t h a t t h i s f e a t u r e is o n l y a v a i l a b l e since M A T L A B Re-
lease 7. For older M A T L A B versions w e suggest u s i n g a n i n l i n e f u n c t i o n i n s t e a d
(see t h e p r o g r a m s i n t h e f o l d e r M a t l a b 6 o f t h e enclosed C D ) . I f e x e c u t i o n speed
is i m p o r t a n t , t h e n one s h o u l d e x p l i c i t l y p r o v i d e t h e f u n c t i o n ( e i t h e r h a r d c o d e d d i -
r e c t l y w h e r e needed, o r as a n M - f i l e ) . T h i s l a t t e r a p p r o a c h w i l l a l w a y s be m o r e
efficient t h a n t h e i n l i n e o r e v e n a n o n y m o u s f u n c t i o n a p p r o a c h . H o w e v e r , t h e n t h e
i n t e r p o l a t i o n p r o g r a m is n o l o n g e r as generic.
W e c a n replace t h e d e f i n i t i o n o f t h e G a u s s i a n o n l i n e 1 b y t h e d e f i n i t i o n o f t h e
l i n e a r f u n c t i o n <p(r) = r o r a n y o t h e r a d m i s s i b l e R B F w e w i l l e n c o u n t e r l a t e r . In
lines 2 - 6 w e define a t e s t f u n c t i o n t h a t w e w i l l s a m p l e s i m i l a r l y t o t h e function
f
s used i n t h e i n t r o d u c t o r y e x a m p l e . H e r e ( a n d i n m a n y l a t e r e x a m p l e s ) w e use
Franke's function
ffay) = ^ - l / 4 ( ( 9 . r - 2 ) + (9s/-2) )
e
2 2
+ 3 _(l/49)(9a +l) -(l/10)(9 /+l)
e ;
2
!
2
2 2 2 2
+ I - l / 4 ( ( 9 x - 7 ) + (9j/-3) ) _ l _ (
e e 9 a ; -4) -(9y-7)
2 5
w h i c h is a s t a n d a r d t e s t f u n c t i o n f o r 2 D s c a t t e r e d d a t a fitting. N o t e t h a t we used
2
(x,y) t o d e n o t e t h e t w o c o m p o n e n t s o f a; G M . T h e g r a p h o f Franke's function
over t h e u n i t s q u a r e is s h o w n i n F i g u r e 2.2.
Fig. 2.2 Franke's test function.
For m a n y o f o u r e x a m p l e s w e use d a t a l o c a t i o n s t h a t h a v e b e e n saved i n files

0
n a m e d Data2D_/ d / s w h e r e t h e n u m b e r o f p o i n t s (%d) is t a k e n f r o m t h e p r o g r e s s i o n
0 0
f c 2
{(2 + l) } = { 9 , 2 5 , 8 1 , 2 8 9 , 1 0 8 9 , 4 2 2 5 , . . . } . T h e characters u or h ( i n place o f
/ s) are used t o d e n o t e e i t h e r u n i f o r m l y s p a c e d p o i n t s , o r H a l t o n p o i n t s i n t h e u n i t
0
s q u a r e . T h e set o f d a t a p o i n t s is d e f i n e d a n d l o a d e d i n lines 7 a n d 8. A s i n t h e e a r l i e r
e x a m p l e w e consider here o n l y t h e case w h e r e t h e centers f o r t h e R B F s c o i n c i d e
w i t h the d a t a locations (line 9 ) .
A g r i d o f e v a l u a t i o n p o i n t s used t o e v a l u a t e o u r i n t e r p o l a n t for t h e p u r p o s e s
o f r e n d e r i n g a n d e r r o r c o m p u t a t i o n is d e f i n e d i n l i n e s 10 a n d 1 1 . T h e t e s t data
( r i g h t - h a n d side o f t h e i n t e r p o l a t i o n e q u a t i o n s ) are c o m p u t e d o n l i n e 12 w h e r e t h e
t e s t f u n c t i o n is s a m p l e d a t t h e d a t a sites.
T h e m a i n p a r t o f t h e code is g i v e n b y lines 1 3 - 1 7 . N o t e t h a t t h i s p a r t is v e r y
similar t o t h e corresponding segment ( l i n e s 7 - 9 ) i n D i s t a n c e M a t r i x F i t .m. The
o n l y difference is t h a t w e n o w a p p l y t h e basic f u n c t i o n <p t o t h e e n t i r e distance
matrices i n order to o b t a i n the i n t e r p o l a t i o n a n d evaluation matrices.
2. Radial Basis Function Interpolation in MATLAB 21
P r o g r a m 2.1. RBFInterpolation2D.m
% RBFInterpolation2D
S c r i p t t h a t performs b a s i c 2D RBF i n t e r p o l a t i o n
% C a l l s on: D i s t a n c e M a t r i x
/ Define t h e G a u s s i a n RBF and shape parameter
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; ep = 21.1;
% Define Franke's f u n c t i o n as t e s t f u n c t i o n
2 f l = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x - 2 ) . ~ 2 + ( 9 * y - 2 ) . " 2 ) / 4 ) ;
3 f 2 = <3(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x + l ) . ~ 2 / 4 9 + ( 9 * y + l ) . ~ 2 / 1 0 ) ) ;
4 f 3 = @(x,y) 0 . 5 * e x p ( - ( ( 9 * x - 7 ) . ~ 2 + ( 9 * y - 3 ) . ~ 2 ) / 4 ) ;
5 f 4 = @(x,y) 0 . 2 * e x p ( - ( ( 9 * x - 4 ) . ~ 2 + ( 9 * y - 7 ) . " 2 ) ) ;
6 t e s t f u n c t i o n = @(x,y) f l ( x , y ) + f 2 ( x , y ) + f 3 ( x , y ) - f 4 ( x , y ) ;
7 N = 1089; g r i d t y p e = 'h';
% Load d a t a p o i n t s
0
8 name = s p r i n t f ( 'Data2D_y d / s' ,N,gridtype) ; load(name)
o 0
9 ctrs = dsites;
10 n e v a l = 40; g r i d = l i n s p a c e ( 0 , 1 , n e v a l ) ;
11 [xe,ye] = m e s h g r i d ( g r i d ) ; e p o i n t s = [ x e ( : ) y e ( : ) ] ;
% Evaluate the t e s t function a t the data points
12 r h s = t e s t f u n c t i o n ( d s i t e s ( : , 1 ) , d s i t e s ( : , 2 ) ) ;
% Compute d i s t a n c e m a t r i x between t h e d a t a s i t e s and c e n t e r s
13 DM_data = D i s t a n c e M a t r i x ( d s i t e s , c t r s ) ;
% Compute i n t e r p o l a t i o n m a t r i x
14 IM = rbf(ep,DM_data);
% Compute d i s t a n c e m a t r i x between e v a l u a t i o n p o i n t s and c e n t e r s
15 DM_eval = D i s t a n c e M a t r i x ( e p o i n t s , c t r s ) ;
% Compute e v a l u a t i o n m a t r i x
16 EM = rbf(ep,DM_eval);
% Compute RBF i n t e r p o l a n t
% ( e v a l u a t i o n m a t r i x * s o l u t i o n of i n t e r p o l a t i o n system)
17 Pf = EM * ( I M \ r h s ) ;
% Compute exact s o l u t i o n , i . e . ,
% e v a l u a t e t e s t f u n c t i o n on e v a l u a t i o n p o i n t s
18 exact = t e s t f u n c t i o n ( e p o i n t s ( : , 1 ) , e p o i n t s ( : , 2 ) ) ;
0/ Compute e r r o r s on e v a l u a t i o n g r i d
20 rms.err = n o r m ( P f - e x a c t ) / n e v a l ;
21 f p r i n t f ( ' R M S e r r o r : /,e\n', r m s . e r r )
22 f p r i n t f ('Maximum e r r o r : /e\n', maxerr)
0
23 fview = [160,20]; % f o r Franke's f u n c t i o n

24 P l o t S u r f ( x e , y e , P f , n e v a l , e x a c t , m a x e r r , f v i e w ) ;
I n T a b l e 2.1 w e r e p o r t t h e results o f a series o f e x p e r i m e n t s i n w h i c h w e c o m p u t e

G a u s s i a n R B F a n d d i s t a n c e m a t r i x i n t e r p o l a n t s t o i n c r e a s i n g l y l a r g e r sets o f d a t a .
W e use one fixed v a l u e o f e for a l l o f t h e e x p e r i m e n t s w i t h t h e G a u s s i a n s . This
t y p e o f a p p r o x i m a t i o n is k n o w n as non-stationary approximation. Its counterpart
is k n o w n as stationary approximation.
E v e n t h o u g h we do n o t p e r f o r m s t a t i o n a r y i n t e r p o l a t i o n i n t h i s e x p e r i m e n t we
t a k e a m i n u t e t o e x p l a i n t h e essential difference b e t w e e n t h e t w o a p p r o a c h e s . I n t h e
s t a t i o n a r y s e t t i n g w e w o u l d scale t h e s h a p e p a r a m e t e r e a c c o r d i n g t o t h e fill distance
(or meshsize) h so t h a t we e n d u p u s i n g " p e a k e d " basis f u n c t i o n s for densely spaced
d a t a a n d " f l a t " basis f u n c t i o n s for coarsely s p a c e d d a t a . W e w i l l use t h e fill d i s t a n c e
as a m e a s u r e o f t h e d a t a d i s t r i b u t i o n . T h e fill d i s t a n c e is u s u a l l y d e f i n e d as
h = h ,n x s u p m i n ||aj (2.3)
a n d i t i n d i c a t e s h o w w e l l t h e d a t a i n t h e set X fill o u t t h e d o m a i n Q. A g e o m e t r i c
i n t e r p r e t a t i o n o f t h e fill d i s t a n c e is g i v e n b y t h e r a d i u s o f t h e largest possible e m p t y
b a l l t h a t c a n be p l a c e d a m o n g t h e d a t a l o c a t i o n s i n s i d e fl (see F i g u r e 2 . 3 ) . Some-
t i m e s t h e s y n o n y m covering radius is used. I n o u r M A T L A B code w e c a n estimate
t h e fill d i s t a n c e v i a
hX = m a x C m i n C D M - e v a l ' ) ) (2.4)
w h e r e DM_eval is t h e m a t r i x c o n s i s t i n g o f p a i r w i s e d i s t a n c e s b e t w e e n t h e e v a l u a t i o n
p o i n t s ( p l a c e d o n a fine u n i f o r m g r i d i n Q) a n d t h e d a t a sites X (c.f. l i n e 15 o f
P r o g r a m 2 . 1 ) . N o t e t h a t we t r a n s p o s e t h e n o n - s y m m e t r i c e v a l u a t i o n m a t r i x . T h i s
corresponds t o finding for each e v a l u a t i o n p o i n t t h e d i s t a n c e t o t h e c o r r e -
s p o n d i n g closest d a t a site, a n d t h e n s e t t i n g hx,n as t h e w o r s t o f t h o s e d i s t a n c e s .
F i g u r e 2.3 i l l u s t r a t e s t h e fill d i s t a n c e for a set o f 25 H a l t o n p o i n t s . Note that in
t h i s case t h e largest "hole" i n t h e d a t a is n e a r t h e b o u n d a r y .
0.8
0.6
0.4
0.2
0 0.2 0.4 0.6 0.8 1

X
Fig. 2.3 T h e fill distance for N = 25 Halton points (h ,n. x 0.2667).
We w i l l take a closer l o o k a t t h e differences between stationary and non-

stationary interpolation i n later chapters of this book.
In t h e f o l l o w i n g e x a m p l e s w e w i l l c l e a r l y see t h e effects t h e s h a p e parame-

t e r has o n t h e c o n d i t i o n n u m b e r o f t h e i n t e r p o l a t i o n m a t r i x ( a n d t h e r e f o r e the
numerical stability) of our computations. I n o r d e r t o be able t o use o u r s c r i p t
R B F I n t e r p o l a t i o n 2 D . m i n c o n j u n c t i o n w i t h G a u s s i a n s t o p r o d u c e a m e a n i n g f u l se-
quence o f n o n - s t a t i o n a r y e x p e r i m e n t s , i.e., w i t h a f i x e d v a l u e o f t h e s h a p e p a r a m e t e r
s, we are r e q u i r e d t o t a k e t h e f a i r l y l a r g e v a l u e e 21.1. Otherwise computation
w i t h t h e r e l a t i v e l y densely spaced p o i n t set o f N = 4225 H a l t o n p o i n t s r e s u l t s i n
M A T L A B warnings of ill-conditioning. T h i s m e a n s t h a t for t h e n o n - s t a t i o n a r y
a p p r o a c h t h e basis f u n c t i o n s are t o o l o c a l i z e d o n t h e s m a l l e r p o i n t sets, a n d t h e
a p p r o x i m a t i o n is v e r y p o o r (see F i g u r e 2.4).
T h e t e s t r e s u l t s for a n o n - s t a t i o n a r y i n t e r p o l a t i o n e x p e r i m e n t u s i n g G a u s s i a n s
a n d E u c l i d e a n d i s t a n c e m a t r i c e s for F r a n k e ' s f u n c t i o n are s h o w n i n T a b l e 2.1. As
j u s t p o i n t e d o u t , w e n o t e t h a t a fit w i t h G a u s s i a n s a n d a s m a l l shape p a r a m e t e r
s u c h as s = 1 w o u l d q u i c k l y l e a d t o a n u m e r i c a l b r e a k d o w n . F o r as few as N = 25
d a t a p o i n t s a n d e = 1 M A T L A B issues a " m a t r i x close t o s i n g u l a r " w a r n i n g w i t h a n
e s t i m a t e d r e c i p r o c a l c o n d i t i o n n u m b e r o f RCOND=3.986027e-020.
Table 2.1 Non-stationary R B F interpolation to Franke's function using

Gaussians (e 21.1) and Euclidean distance matrices.
Gaussian distance matrix
k N RMS-error max-error RMS-error max-error
1 9 3.647169e-001 1.039682e+000 1.323106e-001 4.578028e-001

2 25 3.203404e-001 9.670980e-001 6.400558e-002 2.767871e-001
3 81 2.152222e-001 8.455161e-001 1.343780e-002 6.733130e-002
4 289 7.431729e-002 7.219253e-001 3.707360e-003 3.057540e-002
5 1089 1.398297e-002 3.857234e-001 1.143589e-003 1.451950e-002
6 4225 4.890709e-004 1.940675e-002 4.002749e-004 8.022336e-003
Fig. 2.4 Gaussian R B F interpolant with e = 21.1 at N = 289 (left) and at N = 1089 Halton
points (right).
I f we l o o k a t t h e e n t r i e s i n T a b l e 2 . 1 t h e n w e see t h a t c o n t r a r y t o w h a t w e
a n n o u n c e d e a r l i e r t h e r e s u l t s b a s e d o n t h e d i s t a n c e m a t r i x f i t are m o r e a c c u r a t e
t h a n those o b t a i n e d w i t h Gaussians. T h i s w i l l change, however, i f we t r y t o o p t i m i z e
our choice o f t h e shape p a r a m e t e r e (see t h e r e s u l t s o f t h e n e x t e x p e r i m e n t i n
Table 2.2).
I n a second e x p e r i m e n t w e c o n s i d e r t h e same t e s t f u n c t i o n a n d G a u s s i a n basis
functions. N o w , h o w e v e r , w e w a n t t o s t u d y t h e effects o f t h e s h a p e parameter.
T h e r e f o r e , i n F i g u r e 2.5 w e d i s p l a y b o t h t h e m a x i m u m a n d R M S e r r o r s as a f u n c t i o n
o f t h e s h a p e p a r a m e t e r e f o r four f i x e d d a t a sets ( 8 1 , 289, 1089 a n d 4225 H a l t o n
points). T h e s e curves r e v e a l s o m e o f t h e p r o b l e m s a s s o c i a t e d w i t h r a d i a l basis
function i n t e r p o l a t i o n especially w h e n w o r k i n g w i t h globally s u p p o r t e d basis
f u n c t i o n s , i.e., dense m a t r i c e s . W e see t h a t t h e e r r o r s decrease w i t h d e c r e a s i n g e
(of course, t h e y also decrease w i t h d e c r e a s i n g f i l l d i s t a n c e b u t t h a t is n o t w h a t
we are c o n c e r n e d w i t h n o w ) . H o w e v e r , t h e e r r o r c u r v e s are n o t m o n o t o n i c . W e c a n
i d e n t i f y a n o p t i m a l v a l u e o f e for w h i c h b o t h e r r o r s are m i n i m a l ( t h e m i n i m a o f
t h e t w o e r r o r c u r v e s o c c u r a t a l m o s t t h e s a m e p l a c e ) . M o r e o v e r , t h e r e is a v a l u e o f
e at w h i c h t h e c o m p u t a t i o n a l results become u n p r e d i c t a b l e , a n d t h e error curves
b e c o m e e r r a t i c . T h i s p o i n t is a s s o c i a t e d w i t h severe i l l - c o n d i t i o n i n g o f t h e s y s t e m
m a t r i x . Since M A T L A B issues a w a r n i n g w h e n a t t e m p t i n g t o solve a n i l l - c o n d i t i o n e d
l i n e a r s y s t e m , we refer t o t h e s m a l l e s t v a l u e o f e for w h i c h w e d o n o t see a M A T L A B
w a r n i n g as t h e "safe" v a l u e o f e ( f o r a g i v e n set X o f d a t a sites a n d basic f u n c t i o n
). T h e i n t e r e s t i n g fact a b o u t t h e f o u r p l o t s d i s p l a y e d i n F i g u r e 2.5 is t h a t for t h e
s m a l l e r d a t a sets (N = 81 and N = 289) t h e m i n i m u m e r r o r s are o b t a i n e d f o r a
"safe" e, w h i l e for t h e l a r g e r sets ( i V = 1089 a n d N = 4225) the m i n i m u m errors
are o b t a i n e d i n t h e "unsafe" r a n g e . T h e r e f o r e , w e are c o m p u t i n g i n a c e r t a i n " g r a y
zone". W e are o b t a i n i n g h i g h l y a c c u r a t e s o l u t i o n s f r o m severely i l l - c o n d i t i o n e d
l i n e a r systems. W e w i l l c o m e b a c k l a t e r t o t h i s i n t e r e s t i n g f e a t u r e o f r a d i a l basis
f u n c t i o n i n t e r p o l a t i o n ( c a l l e d uncertainty o r trade-off principle). I t is c o n c e i v a b l e
(and i n fact possible [ F o r n b e r g a n d W r i g h t ( 2 0 0 4 ) ] ) t o o b t a i n even m o r e a c c u r a t e
r e s u l t s b y u s i n g a m o r e s t a b l e w a y t o e v a l u a t e t h e r a d i a l basis f u n c t i o n i n t e r p o l a n t
(see t h e d i s c u s s i o n i n C h a p t e r s 16 a n d 1 7 ) .
I n T a b l e 2.2 w e l i s t t h e "best p o s s i b l e " r e s u l t s f o r s t a t i o n a r y G a u s s i a n i n t e r p o -

l a t i o n . W e v i e w "best" i n t w o d i f f e r e n t w a y s . F o r t h e r e s u l t s p r e s e n t e d i n c o l u m n s
3 - 5 w e select for e a c h choice o f N t h e s m a l l e s t p o s s i b l e v a l u e o f e s u c h t h a t M A T L A B
does n o t issue a w a r n i n g . W e refer t o t h i s case as t h e "safe" e case i n T a b l e 2.2.
M o s t o f these e r r o r s are n o w c o m p a r a b l e ( o r s m a l l e r ) t h a n t h o s e for s i m p l e d i s t a n c e
m a t r i x i n t e r p o l a t i o n (c.f. T a b l e 2 . 1 ) .
These results, however, do n o t always represent t h e smallest achievable error.
T h e r e f o r e , we p r e s e n t ( i n c o l u m n s 6 - 8 ) r e s u l t s for t h o s e " o p t i m a l " values o f e w h i c h
y i e l d t h e s m a l l e s t R M S - e r r o r . T h e s e r e s u l t s are o b t a i n e d i n t h e " g r a y zone" m e n -
t i o n e d a b o v e . F o r e x a m p l e , i f w e use N = 1089 H a l t o n p o i n t s a n d s h a p e p a r a m e t e r
e = 6.2 t h e n M A T L A B issues a w a r n i n g w i t h RC0ND = 2 . 6 8 3 5 2 7 e - 0 2 0 . However,
10
10 jl
10
10"
10"
10
10 15 20
Fig. 2.5 Maximum (dashed/top curve) and R M S (solid/bottom curve) errors vs. e for 81 (top
left), 289 (top right), 1089 (bottom left), and 4225 Halton points (bottom right).
Table 2.2 "Optimal" R B F interpolation to Franke's function using Gaussians.
smallest "safe" e smallest RMS-error
k N RMS-error max-error RMS-error max-error
1 9 0.02 3.658421e-001 1.580259e+000 2.23 1.118026e-001 3.450275e-001

2 25 0.32 3.629342e-001 2.845554e+000 3.64 4.032550e-002 2.996488e-001
3 81 1.64 1.743059e-001 2.398284e+000 4.28 1.090601e-002 1.579465e-001
4 289 4.73 2.785388e-003 5.472502e-002 5.46 4.610079e-004 7.978283e-003
5 1089 10.5 4.945428e-004 1.812246e-002 6.2 2.498848e-006 8.779119e-005
6 4225 21.1 4.890709e-004 1.940675e-002 6.3 4.269292e-008 8.889552e-007
as we c a n see i n T a b l e 2.2 a n d F i g u r e 2.5, t h e c o r r e s p o n d i n g e r r o r s are n o w much

s m a l l e r t h a n t h o s e p r e v i o u s l y o b t a i n e d . M o r e o v e r , t h e e r r o r s decrease a t a r a t e t h a t
is faster t h a n t h e 0(h) w e o b s e r v e d e a r l i e r for t h e d i s t a n c e m a t r i x fit e x a m p l e .
O f course, i f t h e d a t a w e are t r y i n g t o fit are n o t s a m p l e d f r o m a k n o w n t e s t
f u n c t i o n t h e n w e w i l l n o t be a b l e t o choose a n " o p t i m a l " s h a p e p a r a m e t e r b y m o n -
itoring the R M S error. T h e a s s o c i a t e d issues o f i l l - c o n d i t i o n i n g , p r e c o n d i t i o n i n g ,
o p t i m a l shape p a r a m e t e r selection, a n d a l t e r n a t e stable e v a l u a t i o n m e t h o d s v i a a
C o n t o u r - P a d e a l g o r i t h m are s t u d i e d l a t e r i n C h a p t e r s 16 a n d 17.
Chapter 3
Positive Definite Functions
We noted i n the previous chapters t h a t the solution of the scattered d a t a interpola-

t i o n p r o b l e m w i t h R B F s boils d o w n t o the solution of a system of linear equations
Ac = y,
w h e r e t h e s y s t e m m a t r i x A has e n t r i e s <p(||Cj JCfclh), j , k = 1 , . . . , N. We know

f r o m linear a l g e b r a t h a t t h i s s y s t e m w i l l h a v e a u n i q u e s o l u t i o n w h e n e v e r t h e m a t r i x
A is n o n - s i n g u l a r . W h i l e n o one has y e t succeeded i n c h a r a c t e r i z i n g t h e class o f a l l
basic f u n c t i o n s <p t h a t g e n e r a t e a n o n - s i n g u l a r s y s t e m m a t r i x for a n y set X =
{xi,..., XN} o f d i s t i n c t d a t a sites, t h e s i t u a t i o n is m u c h b e t t e r i f we consider
positive definite matrices. *
I n t h i s c h a p t e r we p r e s e n t t h e m a i n t h e o r e t i c a l r e s u l t s u n d e r l y i n g t h i s a p p r o a c h
a l o n g w i t h some o f t h e i r p r o o f s . A series o f e x a m p l e s are p r e s e n t e d i n t h e next
c h a p t e r . A c o m p r e h e n s i v e t r e a t m e n t o f t h e m a t h e m a t i c a l t h e o r y needed f o r scat-
t e r e d d a t a i n t e r p o l a t i o n w i t h s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s (see Def. 3.2 b e l o w )
is p r e s e n t e d i n t h e recent m o n o g r a p h [ W e n d l a n d ( 2 0 0 5 a ) ] .
3.1 Positive Definite M a t r i c e s a n d Functions
D e f i n i t i o n 3.1. A r e a l s y m m e t r i c m a t r i x A is c a l l e d positive semi-definite i f its

associated q u a d r a t i c f o r m is n o n - n e g a t i v e , i.e.,
N N
Y^CjC A >Q
k jk (3.1)
3= 1 k=l
T N
for c = [ c i , . . .,c ] N G R .
I f t h e q u a d r a t i c f o r m ( 3 . 1 ) is zero o n l y for c = 0, t h e n A is c a l l e d positive
definite.
A n i m p o r t a n t p r o p e r t y o f p o s i t i v e d e f i n i t e m a t r i c e s is t h a t a l l t h e i r eigenvalues
are p o s i t i v e , a n d t h e r e f o r e a p o s i t i v e d e f i n i t e m a t r i x is n o n - s i n g u l a r ( b u t c e r t a i n l y
n o t vice v e r s a ) .
27
I f we therefore h a d basis f u n c t i o n s B k i n the expansion (1.1) t h a t generate a

p o s i t i v e d e f i n i t e i n t e r p o l a t i o n m a t r i x , we w o u l d a l w a y s have a w e l l - p o s e d i n t e r p o -
l a t i o n p r o b l e m . T o t h i s e n d we i n t r o d u c e t h e c o n c e p t o f a positive definite function
f r o m classical analysis.
P o s i t i v e d e f i n i t e f u n c t i o n s w e r e first c o n s i d e r e d i n classical a n a l y s i s e a r l y i n t h e
2 0 t h century. [ M a t h i a s (1923)] seems t o h a v e b e e n t h e first t o define a n d study
positive definite functions. A n overview of the development of positive definite
f u n c t i o n s u p t o t h e m i d 1970s c a n b e f o u n d i n [ S t e w a r t ( 1 9 7 6 ) ] . H o w e v e r , as w e
see f r o m t h e d e f i n i t i o n b e l o w , p o s i t i v e d e f i n i t e f u n c t i o n s w e r e u n f o r t u n a t e l y
defined i n a n a l o g y t o p o s i t i v e s e m i - d e f i n i t e m a t r i c e s . Therefore, i n order to meet
o u r g o a l o f h a v i n g a w e l l - p o s e d i n t e r p o l a t i o n p r o b l e m , i t is necessary t o s h a r p e n t h e
classical n o t i o n o f a p o s i t i v e d e f i n i t e f u n c t i o n t o t h a t o f a strictly positive definite
one. T h i s c o n c e p t does n o t seem t o h a v e b e e n s t u d i e d u n t i l [ M i c c h e l l i (1986)] m a d e
the connection between scattered d a t a i n t e r p o l a t i o n and positive definite functions.
T h i s leads t o a n u n f o r t u n a t e difference i n t e r m i n o l o g y used i n t h e c o n t e x t o f m a t r i -
ces a n d f u n c t i o n s . Instead of r e w r i t i n g h i s t o r y we w i l l adhere t o this t e r m i n o l o g y
here. W e w o u l d l i k e t o p o i n t o u t t h a t w h e n r e a d i n g recent a r t i c l e s (especially i n
t h e r a d i a l basis f u n c t i o n l i t e r a t u r e ) d e a l i n g w i t h ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s
one has t o be a w a r e o f t h e fact t h a t some a u t h o r s h a v e t r i e d t o " c o r r e c t " h i s t o r y ,
a n d n o w refer t o s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s as p o s i t i v e d e f i n i t e f u n c t i o n s .
D e f i n i t i o n 3.2. A c o m p l e x - v a l u e d c o n t i n u o u s f u n c t i o n <fr : R s
- C is c a l l e d positive
definite on K. i fs
N N
(3.2)
3= 1 k=l
XN &
s N
for a n y N p a i r w i s e different p o i n t s x i , . . . , R , and c = [ c i , . . . , CJV] T
<C .
s
T h e f u n c t i o n <E> is c a l l e d strictly positive definite on M. i f t h e q u a d r a t i c form
(3.2) is zero o n l y for c = 0 .
W e n o t e t h a t even t h o u g h we are i n t e r e s t e d i n problems w i t h real data and

r e a l coefficients, a n e x t e n s i o n o f t h e n o t i o n o f p o s i t i v e deffniteness t o cover c o m p l e x
coefficients c a n d c o m p l e x - v a l u e d f u n c t i o n s <E> as d o n e i n D e f i n i t i o n 3.2 w i l l be h e l p -
f u l w h e n d e r i v i n g some p r o p e r t i e s o f ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s l a t e r o n .
M o r e o v e r , t h e c e l e b r a t e d Bochner's theorem (see T h e o r e m 3.3) c h a r a c t e r i z e s e x a c t l y
t h e p o s i t i v e d e f i n i t e f u n c t i o n s o f D e f i n i t i o n 3.2. I n a l l p r a c t i c a l c i r c u m s t a n c e s , h o w -
ever, we w i l l be c o n c e r n e d w i t h r e a l - v a l u e d f u n c t i o n s o n l y , a n d a characterization
o f such f u n c t i o n s a p p e a r s b e l o w as T h e o r e m 3.2. I t s h o u l d also be n o t e d t h a t Def-
i n i t i o n 3.2 i m p l i e s t h a t o n l y f u n c t i o n s w h o s e q u a d r a t i c f o r m is r e a l are c a n d i d a t e s
for ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s .
E x a m p l e 3.1. H e r e , a n d t h r o u g h o u t t h i s b o o k , we w i l l d e n o t e t h e s t a n d a r d i n n e r
t x y
product of x and y i n R s
b y x y. W i t h t h i s n o t a t i o n t h e f u n c t i o n <fr(cc) = e ,
for y G R s
fixed, is p o s i t i v e d e f i n i t e o n R s
since t h e q u a d r a t i c f o r m i n D e f i n i t i o n 3.2
becomes
N N N N
j=l k=l j=l k=l

N N
3=1 k=l
2
N
> 0.
3= 1
D e f i n i t i o n 3.2 a n d t h e discussion p r e c e d i n g i t suggest t h a t we s h o u l d use s t r i c t l y

p o s i t i v e d e f i n i t e f u n c t i o n s as basis f u n c t i o n s i n ( 1 . 1 ) , i.e., Bk(x) $>(x X k ) , or
N
Xk), a; G R . (3.3)
fc=i
N o t e t h a t at t h i s p o i n t w e d o n o t r e q u i r e $ t o be a r a d i a l f u n c t i o n . I n fact,
t h e f u n c t i o n Vf o f (3.3) w i l l y i e l d a n i n t e r p o l a n t t h a t is translation invariant, i.e.,
t h e i n t e r p o l a n t t o t r a n s l a t e d d a t a is t h e same as t h e t r a n s l a t e d i n t e r p o l a n t t o t h e
o r i g i n a l d a t a . I n o r d e r t o o b t a i n i n v a r i a n c e also u n d e r r o t a t i o n s a n d reflections we
w i l l l a t e r specialize t o s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s t h a t are also r a d i a l o n R . s
W e w i l l n o w discuss some o f t h e m o s t i m p o r t a n t p r o p e r t i e s a n d characterizations

of ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s . F o r t h e sake o f completeness we present a l i s t
of some basic p r o p e r t i e s o f ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s a n d some e x a m p l e s .
T h e o r e m 3.1. Some basic properties of positive definite functions are
(1) Non-negative finite linear combinations of positive definite functions are

positive definite. i/$i,...,$ n are positive definite on R s
and Cj > 0,
j 1 , . . . , n, then
n
$(as) = ^2cj$j(x), xGR , s
3= 1
is also positive definite. Moreover, if at least one of the <&j is strictly positive
definite and the corresponding c - > 0, then $ is strictly
3 positive definite.
(2) $ ( 0 ) > 0.
(3) $ ( - J T ) - $(aj).
(4) Any positive definite function is bounded. In fact,
\(x)\ < $(0).
(5) If is positive definite with $ ( 0 ) = 0 then $ = 0.

(6) The product of (strictly) positive definite functions is (strictly) positive def-
inite.
Proof. P r o p e r t i e s (1) a n d (2) f o l l o w i m m e d i a t e l y f r o m D e f i n i t i o n 3.2.

T o s h o w (3) w e l e t N = 2, x\ = 0, x 2 = x, a n d choose c\ = 1 a n d c 2 = c. T h e n
t h e q u a d r a t i c f o r m i n D e f i n i t i o n 3.2 becomes
2 2
2
^ cjcTîxj - x) k = (1 + | c | ) 3 > ( 0 ) + c&(x) + c$(-x) > 0
i = i fc=i
for e v e r y c G C. T a k i n g c = 1 a n d c = z ( w h e r e i = y/ 1 ) , r e s p e c t i v e l y , w e c a n see
t h a t b o t h <3?(a;) + <&(ic) a n d z ( $ ( x ) <&(cc)) m u s t be r e a l . T h i s , h o w e v e r , is o n l y
possible i f <E>(x) = <fr(x).
F o r t h e p r o o f o f (4) w e l e t N = 2, X\ 0, x 2 = x, a n d choose c\ = | $ ( a ; ) | a n d
c 2 = Q(x). T h e n t h e q u a d r a t i c f o r m i n D e f i n i t i o n 3.2 is
2 2
^ r ^ T c j - c ^ ^ - x) k = 2<Z>(0)\$(x)\ 2 2
- $ ( - a j ) $ ( x ) | $ ( x ) | - $ (x)\(x)\ > 0.
j=ik=i
Since <&(a:) = &(x) b y P r o p e r t y ( 3 ) , t h i s gives
2 3
2$(0)\$(x)\ - 2\(x)\ > 0.
2
I f | $ ( x ) | > 0, w e d i v i d e b y |<3?(;E)| a n d t h e s t a t e m e n t f o l l o w s i m m e d i a t e l y . I n case
|<E>(a?)| = 0 t h e s t a t e m e n t h o l d s t r i v i a l l y .
P r o p e r t y (5) f o l l o w s i m m e d i a t e l y f r o m ( 4 ) , a n d P r o p e r t y (6) is a consequence o f a
t h e o r e m b y Schur i n t h e field o f l i n e a r a l g e b r a w h i c h s t a t e s t h a t t h e e l e m e n t w i s e (or
H a d a m a r d ) p r o d u c t o f p o s i t i v e ( s e m i - ) d e f i n i t e m a t r i c e s is p o s i t i v e ( s e m i - ) d e f i n i t e .
For m o r e d e t a i l s w e refer t h e r e a d e r t o [ C h e n e y a n d L i g h t (1999)] o r [ W e n d l a n d
(2005a)].
E x a m p l e 3.2. T h e cosine f u n c t i o n is p o s i t i v e d e f i n i t e o n K since, for x G R, w e

xx lx
h a v e cos a; = ^ (e + e~ ). N o w P r o p e r t y (1) a n d E x a m p l e 3 . 1 c a n be i n v o k e d .
P r o p e r t y (3) shows t h a t a n y r e a l - v a l u e d ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n has

t o be even. H o w e v e r , i t is also possible t o c h a r a c t e r i z e r e a l - v a l u e d ( s t r i c t l y ) p o s i t i v e
d e f i n i t e f u n c t i o n s u s i n g o n l y real coefficients (see [ W e n d l a n d (2005a)] for d e t a i l s ) ,
i.e.,
T h e o r e m 3.2. A real-valued continuous function <fr is positive definite on R s

if and
only if it is even and
N N
c c
^2 j k<&(xj - x k ) > 0 (3.4)
3 = 1 fc=l
s T N
for any N pairwise different points Xi,..., x^ G M. , and c = [ c i , . . . , C j v ] G M. .
The function $ is strictly positive definite on R s
if the quadratic form (3.4) is
zero only for c = 0.
3.2 I n t e g r a l C h a r a c t e r i z a t i o n s for ( S t r i c t l y ) P o s i t i v e Definite

Functions
W e w i l l n o w s u m m a r i z e some facts a b o u t i n t e g r a l c h a r a c t e r i z a t i o n s o f p o s i t i v e def-

inite functions. T h e y were e s t a b l i s h e d i n t h e 1930s b y B o c h n e r a n d S c h o e n b e r g .
H o w e v e r , we w i l l also m e n t i o n t h e m o r e recent e x t e n s i o n s t o s t r i c t l y p o s i t i v e defi-
n i t e a n d s t r i c t l y c o m p l e t e l y / m u l t i p l y m o n o t o n e f u n c t i o n s t h a t are essential t o t h e
application of the theory t o the scattered d a t a i n t e r p o l a t i o n problem. A much
m o r e d e t a i l e d discussion o f t h i s m a t e r i a l is p r e s e n t e d i n t h e recent b o o k [ W e n d -
l a n d (2005a)]. Some f r e q u e n t l y used i n t e g r a l t r a n s f o r m s are l i s t e d i n A p p e n d i x B .
I n t e g r a l c h a r a c t e r i z a t i o n s o f t h e closely r e l a t e d c o m p l e t e l y a n d m u l t i p l y monotone
f u n c t i o n s are presented i n C h a p t e r 5.
3.2.1 Bochner''s Theorem
O n e o f t h e m o s t c e l e b r a t e d r e s u l t s o n p o s i t i v e d e f i n i t e f u n c t i o n s is t h e i r c h a r a c t e r -
i z a t i o n i n t e r m s o f F o u r i e r t r a n s f o r m s e s t a b l i s h e d b y B o c h n e r i n 1932 (for s = 1)
a n d 1933 (for g e n e r a l s).
S
T h e o r e m 3 . 3 ( B o c h n e r ) . A (complex-valued) function <E> e C ( I R ) is positive def-
s
inite on R if and only if it is the Fourier transform of a finite non-negative Borel
s
measure fi on R , i.e.
<2>(cc) = jl(x)
1
J ,ixy x e
Proof. T h e r e are m a n y p r o o f s o f t h i s t h e o r e m . B o c h n e r ' s o r i g i n a l p r o o f c a n be

f o u n d i n [ B o c h n e r (1933)]. O t h e r p r o o f s c a n be f o u n d , e.g., i n t h e b o o k s [ C u p p e n s
(1975)] or [ G e l ' f a n d a n d V i l e n k i n ( 1 9 6 4 ) ] . A p r o o f u s i n g t h e Riesz r e p r e s e n t a t i o n
t h e o r e m t o i n t e r p r e t t h e B o r e l m e a s u r e as a d i s t r i b u t i o n , a n d t h e n t a k i n g a d v a n t a g e
of d i s t r i b u t i o n a l F o u r i e r t r a n s f o r m s c a n be f o u n d i n t h e b o o k [ W e n d l a n d ( 2 0 0 5 a ) ] .
W e w i l l p r o v e o n l y t h e one (easy) d i r e c t i o n . I t is t h i s p a r t o f t h e s t a t e m e n t t h a t
is i m p o r t a n t for t h e a p p l i c a t i o n t o s c a t t e r e d d a t a i n t e r p o l a t i o n . W e assume <fr is
t h e F o u r i e r t r a n s f o r m o f a f i n i t e n o n - n e g a t i v e B o r e l m e a s u r e a n d show <E> is p o s i t i v e
definite. Thus,
N N N N
^^TcjCTQiXj
1=1 k=l
- x)
k =
:
EE
j=l k=X
CjC k J ,-i(xj-x )-y
k
d^(y)
N N
1
J E Cj e -zxj-y
E
fc=i
Cite
ix y
k
dLi(y)
Jut*
J=l
J E N
Cje
-txj-y dfj,(y) > 0.
T h e last i n e q u a l i t y h o l d s because o f t h e c o n d i t i o n s i m p o s e d o n t h e m e a s u r e \x.
t x y
R e m a r k 3 . 1 . W e c a n see f r o m T h e o r e m 3.3 t h a t t h e f u n c t i o n $(x) = e of
E x a m p l e 3.1 c a n be c o n s i d e r e d as t h e fundamental positive definite function since
a l l o t h e r p o s i t i v e d e f i n i t e f u n c t i o n s are o b t a i n e d as ( i n f i n i t e ) l i n e a r c o m b i n a t i o n s o f
t h i s f u n c t i o n . W h i l e P r o p e r t y (1) o f T h e o r e m 3.1 i m p l i e s t h a t l i n e a r c o m b i n a t i o n s
o f <& w i l l a g a i n b e p o s i t i v e d e f i n i t e , t h e r e m a r k a b l e c o n t e n t o f B o c h n e r ' s Theorem
is t h e fact t h a t i n d e e d all p o s i t i v e d e f i n i t e f u n c t i o n s are g e n e r a t e d b y
3.2.2 Extensions to Strictly Positive Definite Functions
I n order t o accomplish our goal o f guaranteeing a well-posed i n t e r p o l a t i o n p r o b l e m

we have t o e x t e n d ( i f possible) B o c h n e r ' s c h a r a c t e r i z a t i o n t o strictly positive definite
functions.
W e b e g i n w i t h a sufficient c o n d i t i o n for a f u n c t i o n t o be s t r i c t l y p o s i t i v e d e f i n i t e
s
on R .
F o r t h i s r e s u l t we r e q u i r e t h e n o t i o n o f t h e carrier of a (non-negative) Borel
m e a s u r e d e f i n e d o n some t o p o l o g i c a l space X (see also A p p e n d i x B ) . T h i s set is
given by
X \ \ J { 0 : O is o p e n a n d fi(0) = 0}.
s
T h e o r e m 3 . 4 . Let /J, be a non-negative finite Borel measure on R whose carrier
is a set of nonzero Lebesgue measure. Then the Fourier transform of /j, is strictly
s
positive definite on R .
Proof. A s i n the p r o o f o f Bochner's t h e o r e m we have

N N N N
d^{y)
3= 1 k=l VW) j = 1 k =i
N N
= _ L _ / c e i X j y e i X k dfi(y)
j 2 3 ~ Y l ^
3=1 k=l
2
\/(2jr) A t
d(y) > 0.
3= 1
N o w let v/(2ir) h
AT
c e i X i v
g{y) = Y. i ~ ' '
3= 1
a n d assume t h a t t h e p o i n t s xj are a l l d i s t i n c t a n d c ^ O . I n t h i s case t h e f u n c t i o n s

lXj y
y i* e~ ' are l i n e a r l y i n d e p e n d e n t so t h a t g ^ 0. Since g is a n e n t i r e f u n c t i o n i t s
s
zero set, i.e., {y R : g(y) = 0 } can have no accumulation p o i n t and therefore
i t has Lebesgue m e a s u r e zero (see, e.g., [ C h e n e y a n d L i g h t ( 1 9 9 9 ) ] ) . N o w , t h e o n l y
r e m a i n i n g w a y t o m a k e t h e a b o v e i n e q u a l i t y a n e q u a l i t y is i f t h e c a r r i e r o f fi is
c o n t a i n e d i n t h e zero set o f g, i.e., has Lebesgue m e a s u r e zero. T h i s , however, is
ruled out i n the hypothesis o f the theorem.
W o r k t o w a r d a n a n a l o g o f B o c h n e r ' s t h e o r e m , i.e., a c o m p l e t e i n t e g r a l charac-

s
t e r i z a t i o n o f f u n c t i o n s t h a t are s t r i c t l y p o s i t i v e d e f i n i t e o n Wt , is g i v e n i n [ C h a n g
(1996)] for t h e case s = 1.
T h e f o l l o w i n g c o r o l l a r y gives us a w a y t o construct s t r i c t l y positive definite
functions.
s
C o r o l l a r y 3 . 1 . Let f be a continuous non-negative function in Li(R ) which is not
s
identically zero. Then the Fourier transform of f is strictly positive definite on M. .
Proof. T h i s is a special case o f t h e p r e v i o u s t h e o r e m i n w h i c h t h e m e a s u r e fj, has

Lebesgue d e n s i t y / . T h u s , w e use t h e m e a s u r e \i defined for a n y B o r e l set B b y
//() = ( f(x)dx.
JB
T h e n t h e c a r r i e r o f fi is e q u a l t o t h e (closed) s u p p o r t o f / . H o w e v e r , since / is
n o n - n e g a t i v e a n d n o t i d e n t i c a l l y e q u a l t o zero, i t s s u p p o r t has p o s i t i v e L e b e s g u e
measure, a n d hence t h e F o u r i e r t r a n s f o r m o f / is s t r i c t l y p o s i t i v e d e f i n i t e b y t h e
preceding theorem.
F i n a l l y , a c r i t e r i o n t o check w h e t h e r a g i v e n f u n c t i o n is s t r i c t l y p o s i t i v e d e f i n i t e
is g i v e n i n [ W e n d l a n d ( 2 0 0 5 a ) ] .
s
T h e o r e m 3.5. Let <& be a continuous function in L i ( R ) . <& is strictly positive
definite if and only if $ is bounded and its Fourier transform is non-negative and
not identically equal to zero.
T h e o r e m 3.5 is o f f u n d a m e n t a l i m p o r t a n c e a n d we w i l l c o m e b a c k t o t h i s t h e o r e m
several t i m e s l a t e r o n . I n f a c t , t h e p r o o f o f T h e o r e m 3.5 i n [ W e n d l a n d (2005a)] shows
t h a t i f <& ^ 0 ( w h i c h i m p l i e s t h a t t h e n also <E ^ 0) w e need t o ensure o n l y
t h a t <l> be n o n - n e g a t i v e i n o r d e r for <3> t o be s t r i c t l y p o s i t i v e d e f i n i t e .
3.3 Positive Definite R a d i a l Functions
W e n o w t u r n o u r a t t e n t i o n t o p o s i t i v e d e f i n i t e radial functions. Recall t h a t Def-

i n i t i o n 3.2 characterizes ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s in terms of multi-
v a r i a t e f u n c t i o n s <. H o w e v e r , w h e n we are d e a l i n g w i t h r a d i a l f u n c t i o n s , i.e.,
<&(x) = (p(\\x\\), t h e n i t w i l l be c o n v e n i e n t t o also refer t o t h e u n i v a r i a t e f u n c t i o n
<p as a positive definite radial function. W h i l e t h i s does present a s l i g h t abuse o f
our t e r m i n o l o g y for p o s i t i v e d e f i n i t e f u n c t i o n s t h i s is w h a t is c o m m o n l y d o n e i n t h e
literature.
A n i m m e d i a t e consequence o f t h i s n o t a t i o n a l c o n v e n t i o n is
L e m m a 3 . 1 . If ^ p(\\ ||) is (strictly) positive definite and radial on R s

then <fr
CT
is also (strictly) positive definite and radial on R for any a < s.
We now r e t u r n to integral characterizations and begin w i t h a theorem due t o

Schoenberg (see, e.g., [Schoenberg ( 1 9 3 8 a ) ] , p . 8 1 6 , o r [ W e l l s a n d W i l l i a m s ( 1 9 7 5 ) ] ,
p.27).
T h e o r e m 3 . 6 . A continuous function ip : [0, oo) * 3R is positive definite and radial

s
on R if and only if it is the Bessel transform of a finite non-negative Borel measure
/ i on [0, o o ) , i.e.
Here
for s = 1,
for s>2,
5
and J ( _ 2 ) / 2 ^ the classical
s Bessel function of the first kind of order (s 2 ) / 2 .
A s above, n o w t h e f u n c t i o n <E>(a;) = cos(a:) f r o m E x a m p l e 3.2 c a n be v i e w e d

as t h e f u n d a m e n t a l positive definite r a d i a l f u n c t i o n o n R . W e w i l l see b e l o w ( i n
E x a m p l e 3 o f C h a p t e r 4) t h a t t h e c h a r a c t e r i z a t i o n o f T h e o r e m 3.6 immediately
suggests a class o f (even s t r i c t l y ) p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s . A s for t h e basic
I D e x a m p l e , t h e m e a s u r e fi w i l l s i m p l y be a p o i n t e v a l u a t i o n m e a s u r e .
A Fourier t r a n s f o r m characterization of s t r i c t l y positive definite r a d i a l functions
s
on R c a n be f o u n d i n [ W e n d l a n d ( 2 0 0 5 a ) ] . I t is e s s e n t i a l l y a c o m b i n a t i o n o f T h e o -
r e m 3.5 a n d t h e f o r m u l a i n T h e o r e m B . l o f A p p e n d i x B for t h e F o u r i e r t r a n s f o r m
of a radial function:
T h e o r e m 3.7. A continuous function r ~ cp(r) s 1
e
s
L\[0, oo) is strictly positive definite and radial on R if and only if the s-dimensional
Fourier transform
is non-negative and not identically equal to zero.
Since L e m m a 3.1 states t h a t a n y f u n c t i o n t h a t is ( s t r i c t l y ) p o s i t i v e d e f i n i t e a n d

s CT
radial on R is also ( s t r i c t l y ) p o s i t i v e d e f i n i t e a n d r a d i a l o n R for a n y a < s,
s
t h o s e f u n c t i o n s w h i c h are ( s t r i c t l y ) p o s i t i v e d e f i n i t e a n d r a d i a l o n R for all s are
s
of p a r t i c u l a r i n t e r e s t . T h e class o f f u n c t i o n s t h a t are p o s i t i v e d e f i n i t e o n R for a l l
s was also c h a r a c t e r i z e d b y S c h o e n b e r g ( [ S c h o e n b e r g (1938a)], pp. 817-821). An
e x t e n s i o n t o t h e s t r i c t l y p o s i t i v e d e f i n i t e case c a n be f o u n d i n [ M i c c h e l l i ( 1 9 8 6 ) ] :
T h e o r e m 3.8 ( S c h o e n b e r g ) . A continuous function ip : [ 0 , o o ) R is strictly

S
positive definite and radial on R for all s if and only if it is of the form
Jo
where fi is a finite non-negative Borel measure on [0, oo) not concentrated at the
origin.
A s suggested for T h e o r e m 3.6 above, l e t t i n g fi be a p o i n t e v a l u a t i o n m e a s u r e i n

T h e o r e m 3.8 we o b t a i n t h a t t h e G a u s s i a n is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n
S
R for a l l s (c.f Example 1 of Chapter 4).
T h e Schoenberg c h a r a c t e r i z a t i o n o f ( s t r i c t l y ) p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s o n
S
R for a l l s ( T h e o r e m 3.8) i m p l i e s t h a t we have a f i n i t e n o n - n e g a t i v e B o r e l m e a s u r e
o n [0, oo) such t h a t
I f we w a n t t o find a zero r n o f cp t h e n we have t o solve
Since t h e e x p o n e n t i a l f u n c t i o n is p o s i t i v e a n d t h e m e a s u r e is n o n - n e g a t i v e , i t f o l l o w s
t h a t /J, m u s t be t h e zero m e a s u r e . H o w e v e r , t h e n cp is i d e n t i c a l l y e q u a l t o zero.
S
T h e r e f o r e , a n o n - t r i v i a l f u n c t i o n cp t h a t is p o s i t i v e d e f i n i t e a n d r a d i a l o n R for a l l
s c a n have n o zeros. T h i s i m p l i e s i n p a r t i c u l a r t h a t
T h e o r e m 3.9. There are no oscillatory univariate continuous functions that are

S
strictly positive definite and radial on R for all s. Moreover, there are no com-
pactly supported univariate continuous functions that are strictly positive definite
S
and radial on R for all s.
A n e q u i v a l e n t a r g u m e n t for t h e o s c i l l a t o r y case is g i v e n i n T h e o r e m 2.3 o f [ F o r n -

b e r g et al. ( 2 0 0 4 ) ] .
Chapter 4
Examples of Strictly Positive Definite

Radial Functions
W e n o w present a n u m b e r o f f u n c t i o n s t h a t are covered b y t h e t h e o r y p r e s e n t e d

t h u s far. W h i l e i t is possible t o i n c l u d e a shape parameter e for a l l o f t h e f u n c t i o n s
presented i n the examples below b y rescaling x t o ex, we a v o i d i t s use i n t h e
f o r m u l a t i o n o f a l l b u t t h e G a u s s i a n e x a m p l e t o keep t h e f o r m u l a s as s i m p l e as
possible. W e d o , however, use a shape p a r a m e t e r w h e n p l o t t i n g some o f t h e basis
functions.
O u r use o f t h e shape p a r a m e t e r does n o t a l w a y s m a t c h i t s " t r a d i t i o n a l " use.
F o r e x a m p l e , H a r d y i n t r o d u c e d his inverse m u l t i q u a d r i c s (see E x a m p l e 5 b e l o w )
2 2
i n t h e f o r m <&(||a:||) = l/\/c + \\x\\ w i t h shape p a r a m e t e r c. I t is, o f course,
straightforward to transform this representation t o t h e one suggested above, i.e.,
x = 2 2 2 2
^Kll II) + ||cc|| , by setting c = 1/e a n d scaling the result by
O u r use o f t h e shape p a r a m e t e r as a f a c t o r a p p l i e d d i r e c t l y t o x has t h e a d v a n -
t a g e o f p r o v i d i n g a u n i f i e d t r e a t m e n t i n w h i c h a decrease o f t h e shape p a r a m e t e r
always has t h e effect o f p r o d u c i n g " f i a t " basis f u n c t i o n s , w h i l e i n c r e a s i n g e leads t o
m o r e p e a k e d (or l o c a l i z e d ) basis f u n c t i o n s .
4.1 E x a m p l e 1: Gaussians
W e c a n n o w show t h a t t h e Gaussian
2 x | 1 2
&(x) = e- H , > 0 , (4.1)
is s t r i c t l y p o s i t i v e d e f i n i t e ( a n d r a d i a l ) o n K s
for a n y s. T h i s is d u e t o t h e fact t h a t
t h e F o u r i e r t r a n s f o r m o f a G a u s s i a n is essentially a G a u s s i a n . I n fact,
, 1
a n d t h i s is p o s i t i v e i n d e p e n d e n t o f t h e space d i m e n s i o n s. I n p a r t i c u l a r , for e =
w e have <E> = P l o t s o f G a u s s i a n R B F s were p r e s e n t e d i n F i g . 2 . 1 . C l e a r l y , t h e
Gaussians are i n f i n i t e l y d i f f e r e n t i a b l e . S o m e o f i t s d e r i v a t i v e s (as w e l l as t h o s e o f
m a n y o t h e r R B F s ) are c o l l e c t e d i n A p p e n d i x D .
37
A n o t h e r a r g u m e n t t o s h o w t h a t G a u s s i a n s are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l
S
on R for a n y s t h a t avoids d e a l i n g w i t h F o u r i e r t r a n s f o r m s w i l l b e c o m e a v a i l a b l e
l a t e r . I t w i l l m a k e use o f c o m p l e t e l y m o n o t o n e f u n c t i o n s .
R e c a l l t h a t P r o p e r t y ( 1 ) o f T h e o r e m 3.1 shows t h a t a n y finite n o n - n e g a t i v e l i n -
ear c o m b i n a t i o n o f ( s t r i c t l y ) p o s i t i v e d e f i n i t e f u n c t i o n s is a g a i n ( s t r i c t l y ) p o s i t i v e
d e f i n i t e . M o r e o v e r , w e j u s t saw t h a t G a u s s i a n s are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a -
S
d i a l o n a l l R . N o w , t h e S c h o e n b e r g c h a r a c t e r i z a t i o n o f f u n c t i o n s t h a t are ( s t r i c t l y )
S
p o s i t i v e d e f i n i t e a n d r a d i a l o n a n y R , T h e o r e m 3.8, s t a t e s t h a t all s u c h f u n c t i o n s
are g i v e n as i n f i n i t e l i n e a r c o m b i n a t i o n s o f G a u s s i a n s . T h e r e f o r e , t h e G a u s s i a n s c a n
be v i e w e d as t h e f u n d a m e n t a l m e m b e r o f t h e f a m i l y o f f u n c t i o n s t h a t are s t r i c t l y
S
positive definite and radial on R for a l l s.
Since Gaussians p l a y a c e n t r a l r o l e i n s t a t i s t i c s t h i s is a g o o d p l a c e t o m e n t i o n
t h a t p o s i t i v e d e f i n i t e f u n c t i o n s are u p t o a n o r m a l i z a t i o n $ ( 0 ) = 1 i d e n t i c a l
w i t h characteristic functions of d i s t r i b u t i o n functions i n statistics.
4.2 E x a m p l e 2: L a g u e r r e - G a u s s i a n s
I n o r d e r t o o b t a i n a g e n e r a l i z a t i o n o f G a u s s i a n s w e s t a r t w i t h t h e generalized La-
2
guerre polynomials L % / o f degree n a n d o r d e r s/2 d e f i n e d b y t h e i r R o d r i g u e s f o r m u l a
(see, e.g., f o r m u l a (6.2.1) i n [ A n d r e w s et al. ( 1 9 9 9 ) ] )
tj. s/2 JTl . .
L / 2 ( i ) = i e < n + / 2
V ^ ( " ' ' ) ' "
A n e x p l i c i t f o r m u l a for t h e g e n e r a l i z e d L a g u e r r e p o l y n o m i a l s is
W e t h e n define t h e Laguerre- Gaussians

2
= c-H-II'L^dlasH ), (4.2)
a n d list t h e i r F o u r i e r t r a n s f o r m s as
_ H"ll 2
n n | | 2 l '
<KuO = ^ V M ^ > 0 . (4.3)
N o t e t h a t t h e d e f i n i t i o n o f t h e L a g u e r r e - G a u s s i a n s d e p e n d s o n t h e space d i m e n s i o n
S
s. T h e r e f o r e t h e y are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R ( a n d b y L e m m a 3.1
C T
also o n R for a n y a < s).
Laguerre-Gaussian f u n c t i o n s for some s p e c i a l choices o f s a n d n are l i s t e d i n
T a b l e 4 . 1 . F i g u r e 4 . 1 shows a L a g u e r r e - G a u s s i a n for s l , n = 2, a n d for s =
2 , n = 2 d i s p l a y e d w i t h a shape p a r a m e t e r e = 3 a n d scaled so t h a t <&(0) = 1.
M o r e o v e r , t h e L a g u e r r e - G a u s s i a n s are i n f i n i t e l y s m o o t h for a l l choices o f n a n d s.
N o t e t h a t t h e L a g u e r r e - G a u s s i a n s ( w h i l e b e i n g s t r i c t l y p o s i t i v e d e f i n i t e func-
t i o n s ) are n o t p o s i t i v e . Since t h e L a g u e r r e - G a u s s i a n s are o s c i l l a t o r y f u n c t i o n s w e
^ 1
4- Examples of Strictly Positive Definite Radial Functions 39
k n o w f r o m T h e o r e m 3.9 t h a t t h e y c a n n o t be s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n
s
R for a l l s. W e w i l l e n c o u n t e r these f u n c t i o n s l a t e r i n t h e c o n t e x t o f a p p r o x i m a t e
m o v i n g least squares a p p r o x i m a t i o n (c.f. Chapter 26).
Table 4.1 Laguerre-Gaussians for various choices of s and n.
s n = 1 n = 2
Fig. 4.1 Laguerre-Gaussians with s = l,n = 2 (left) and s = 2, n = 2 (right) centered at the
origin.
4.3 E x a m p l e 3: P o i s s o n R a d i a l Functions
A n o t h e r class o f o s c i l l a t o r y f u n c t i o n s t h a t are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l
s CT
on R (and all R for a < s) w e r e r e c e n t l y s t u d i e d b y F o r n b e r g a n d c o - w o r k e r s (see
[ F o r n b e r g et al. (2004)] a n d also [ F l y e r ( 2 0 0 6 ) ] ) . T h e s e f u n c t i o n s are o f t h e f o r m
*(*> = ^ P . (*<)
where J u is t h e Bessel f u n c t i o n o f t h e first k i n d o f o r d e r v. W h i l e these f u n c t i o n s

are n o t defined a t t h e o r i g i n t h e y c a n be e x t e n d e d t o be i n f i n i t e l y d i f f e r e n t i a b l e i n
s
allofR .
T h e f u n c t i o n s (4.4) were a l r e a d y s t u d i e d b y S c h o e n b e r g (see t h e d i s c u s s i o n s u r -
r o u n d i n g T h e o r e m 3.6) w h o s u g g e s t e d c a l l i n g t h e m Poisson functions. I n fact, t h e
s _ 2 2
f u n c t i o n s i n (4.4) are ( u p t o t h e scale f a c t o r 2 ^ ^ r ( s / 2 ) ) t h e f u n c t i o n s fl s of
^1. t
T h e o r e m 3.6 a n d t h e r e f o r e c a n be v i e w e d as t h e f u n d a m e n t a l m e m b e r o f t h e f a m i l y
S
o f f u n c t i o n s t h a t are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R for f i x e d s.
S c h o e n b e r g s h o w e d t h a t t h e f u n c t i o n s f 2 are g i v e n b y
s
Q (x) s = _L_ f e dcr,ixa
1 S
w h e r e co -i
s denotes t h e area o f t h e u n i t s p h e r e S**" i n R , a n d da d e n o t e s t h e
s - 1
usual measure o n S ' .
T h e Poisson f u n c t i o n s are a n o t h e r g e n e r a l i z a t i o n o f G a u s s i a n s ( t h e f u n d a m e n t a l
S
s t r i c t l y positive definite radial function o n R for a l l s) since t h e f o l l o w i n g l i m i t -
i n g r e l a t i o n due t o J o h n v o n N e u m a n n h o l d s (see t h e d i s c u s s i o n i n [Schoenberg
(1938a)]):
- 7 - 2
l i m n (rV2s)
s = e .
s>oo
Since t h e P o i s s o n r a d i a l f u n c t i o n s are d e f i n e d i n t e r m s o f Bessel f u n c t i o n s t h e y are
also band-limited, i.e., t h e i r F o u r i e r t r a n s f o r m has c o m p a c t s u p p o r t . I n fact, t h e
F F
F o u r i e r t r a n s f o r m o f <E> i n R , a < s, is g i v e n b y (see [ F l y e r ( 2 0 0 6 ) ] )
Some o f these P o i s s o n f u n c t i o n s are l i s t e d i n T a b l e 4.2 a n d d i s p l a y e d i n F i g u r e 4.2

( w h e r e a shape p a r a m e t e r e 10 was used for t h e p l o t s ) .
Table 4.2 Poisson functions for various choices of s.
s = 2 s = 3 s = 4 s = 5
r2 s i n (11*11) MINI) /TsindlccH) - ll^ll c o s M )

Jo(IMI)
V n | X \\ 11*11 V 7T \\ \\
x 3
2
Fig. 4.2 Poisson functions with s = 2 (left) and s 3 (right) centered at the origin in R .
4.4 E x a m p l e 4: M a t e r n Functions
A f o u r t h e x a m p l e o f s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s is g i v e n b y t h e class o f
Matern functions w h i c h are q u i t e c o m m o n i n t h e s t a t i s t i c s l i t e r a t u r e (see, e.g.,
[ M a t e r n (1986)] or [Stein ( 1 9 9 9 ) ] )
Here K v is t h e modified Bessel function of the second kind ( s o m e t i m e s also c a l l e d

m o d i f i e d Bessel f u n c t i o n o f t h e t h i r d k i n d , or M a c D o n a l d ' s f u n c t i o n ) of order v.
T h e F o u r i e r t r a n s f o r m o f t h e M a t e r n f u n c t i o n s is g i v e n b y t h e Bessel kernels
2
<l(uO = ( i + M ) ^ > o.
S
T h e r e f o r e t h e M a t e r n f u n c t i o n s are s t r i c t l y p o s i t i v e d e f i n i t e o n IR for a l l s < 23.
Schaback calls these f u n c t i o n s Sobolev splines (see, e.g., [Schaback (1995a)] o r his
earlier discussion i n [Schaback ( 1 9 9 3 ) ] ) since t h e y are n a t u r a l l y r e l a t e d t o S o b o l e v
spaces (see C h a p t e r 13). T h e s e f u n c t i o n s are also discussed i n t h e r e l a t i v e l y e a r l y
paper [ D i x and Ogden (1994)].
Some s i m p l e r e p r e s e n t a t i v e s o f t h e f a m i l y o f M a t e r n f u n c t i o n s are l i s t e d ( u p t o
a d i m e n s i o n - d e p e n d e n t scale f a c t o r ) i n T a b l e 4.3. N o t e t h a t t h e scaled f u n c t i o n s
l i s t e d i n T a b l e 4.3 d o n o t d e p e n d o n s. Since t h e m o d i f i e d Bessel f u n c t i o n s are
p o s i t i v e , so are t h e M a t e r n f u n c t i o n s . T w o e x a m p l e s are d i s p l a y e d i n F i g u r e 4.3.
T h e f u n c t i o n o n t h e left is d i s p l a y e d u s i n g a shape p a r a m e t e r e 3. The plot
o n t h e r i g h t is scaled so t h a t t h e value a t t h e o r i g i n equals one a n d uses a shape
1
p a r a m e t e r e = 10. N o t e t h a t t h e f u n c t i o n o n t h e left ( c o r r e s p o n d i n g t o 3 = ^ r )
2
is n o t d i f f e r e n t i a b l e a t t h e o r i g i n . T h e M a t e r n f u n c t i o n for 3 = is C smooth,
4 S
a n d t h a t for 3 = is i n C ( R ) .
Table 4.3 Matern functions for various choices of (3.
P=sk (3= its 0 = *
2
(1+11*11)6-11*11 (3 +311*11+ ||a || )e3
4.5 E x a m p l e 5: G e n e r a l i z e d I n v e r s e Multiquadrics
Since b o t h $ a n d <E> i n t h e p r e v i o u s e x a m p l e are p o s i t i v e r a d i a l f u n c t i o n s we c a n

use t h e H a n k e l i n v e r s i o n t h e o r e m (see A p p e n d i x B ) t o reverse t h e i r roles a n d see
t h a t t h e so-called generalized inverse multiquadrics
2 S
*(x) = (1 + Hxll )-^, 3 > - , (4.6)
s
are s t r i c t l y p o s i t i v e d e f i n i t e o n R for s < 23. G e n e r a l i z e d inverse m u l t i q u a d r i c s are
i n f i n i t e l y d i f f e r e n t i a b l e . B y u s i n g a n o t h e r a r g u m e n t based o n c o m p l e t e l y monotone
2
Fig. 4.3 Matern functions with B = i (left) and B = ^5 (right) centered at the origin in R .
f u n c t i o n s w e w i l l be able t o s h o w t h a t i n f a c t w e n e e d t o r e q u i r e o n l y 8 > 0, a n d
S
t h e r e f o r e t h e g e n e r a l i z e d inverse m u l t i q u a d r i c s are s t r i c t l y p o s i t i v e d e f i n i t e o n ]R
for a n y s.
T h e " o r i g i n a l " inverse m u l t i q u a d r i c was i n t r o d u c e d b y H a r d y i n t h e e a r l y 1970s
a n d c o r r e s p o n d s t o t h e value 8 1/2. T h e s p e c i a l choice 8=1 was r e f e r r e d t o as
inverse quadratic i n v a r i o u s p a p e r s o f F o r n b e r g a n d c o - w o r k e r s (see, e.g., [Fornberg
a n d W r i g h t ( 2 0 0 4 ) ] ) . T h e s e t w o f u n c t i o n s are d i s p l a y e d i n F i g u r e 4.4 u s i n g a s h a p e
p a r a m e t e r e = 5.
1-. r , 1
Fig. 4.4 Inverse multiquadric (/3 = ^, left) and inverse quadratic (8=1, right) centered at the
2
origin in R .
4.6 E x a m p l e 6: T r u n c a t e d P o w e r Functions
W e n o w present a n e x a m p l e o f a f a m i l y o f s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s w i t h
compact support. N o t e t h a t d u e t o t h e o b s e r v a t i o n m a d e i n T h e o r e m 3.9 a t t h e e n d
S
of t h e p r e v i o u s c h a p t e r , t h e y c a n n o t be s t r i c t l y p o s i t i v e d e f i n i t e o n R for a l l s.
T h e truncated power functions
ipi(r) = ( 1 - r ) i+ (4.7)
s
give rise t o s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l f u n c t i o n s o n M p r o v i d e d I satisfies
t > L f J + 1- F i n d i n g t h e F o u r i e r t r a n s f o r m o f t h e t r u n c a t e d p o w e r f u n c t i o n is
r a t h e r i n v o l v e d . F o r d e t a i l s we refer t o [ W e n d l a n d ( 2 0 0 5 a ) ] . W e w i l l l a t e r use a
s i m p l e r t e s t based o n m u l t i p l y m o n o t o n e f u n c t i o n s t o e s t a b l i s h t h e s t r i c t p o s i t i v e
definiteness o f t h e t r u n c a t e d p o w e r f u n c t i o n s . I n (4.7) we used t h e cutoff function
() + w h i c h is defined b y
x, for x > 0,
0, for x < 0.
T h e c u t o f f f u n c t i o n c a n be i m p l e m e n t e d c o n v e n i e n t l y i n M A T L A B u s i n g t h e max
f u n c t i o n , i.e., i f f x is a v e c t o r o f f u n c t i o n values o f / for d i f f e r e n t choices o f x, t h e n
m a x ( f x , 0 ) c o m p u t e s (f(x)) . + W e also p o i n t o u t t h a t t h e expressions o f t h e f o r m
( 1 r) e
+ are t o be i n t e r p r e t e d as ( ( 1 r)+)*, i.e., we first a p p l y t h e c u t o f f f u n c t i o n ,
and then the power.
T w o different t r u n c a t e d p o w e r f u n c t i o n s ( w i t h i = 2 , 4 ) are d i s p l a y e d i n F i g -
u r e 4.5. W h i l e n o n e o f t h e t r u n c a t e d p o w e r f u n c t i o n s are d i f f e r e n t i a b l e a t t h e o r i g i n ,
t h e s m o o t h n e s s a t t h e b o u n d a r y o f t h e s u p p o r t increases w i t h I .
Fig. 4.5 Truncated power function with i = 2 (left) and = 4 (right) centered at the origin in
4.7 E x a m p l e 7: P o t e n t i a l s a n d W h i t t a k e r R a d i a l F u n c t i o n s
L e t / G C[0, oo) be n o n - n e g a t i v e a n d n o t i d e n t i c a l l y e q u a l t o zero, a n d define t h e

f u n c t i o n cp b y
(4.8)
^1 it.
s
T h e n <& = ip(\\ ||) is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n E p r o v i d e d k > |_|J + 2
(see also T h e o r e m 5.5 b e l o w ) . T h i s c a n be v e r i f i e d b y c o n s i d e r i n g t h e quadratic
form
N N oo N N
- 1
EEw^ *!^ /
3= 1 k=l J
3= 1
c
^2^2 jCkVk-i(t\\xj
k=l
-x \\)f(t)dt
k
w h i c h is n o n - n e g a t i v e since t h e t r u n c a t e d p o w e r f u n c t i o n ^ _ i ( | | | | ) is s t r i c t l y
p o s i t i v e d e f i n i t e b y E x a m p l e 6, a n d / is n o n - n e g a t i v e . Since / is also a s s u m e d t o
be n o t i d e n t i c a l l y e q u a l t o zero, t h e o n l y w a y for t h e q u a d r a t i c f o r m t o e q u a l zero
is i f c = 0 , a n d therefore ip is s t r i c t l y p o s i t i v e d e f i n i t e .
For e x a m p l e , i f we t a k e f(t) = t@, 8 > 0, t h e n we get
- r(fc)r(/3 + 1 )
{ X } ( 4 9 )
~ T{k + 8 + l)\\x\\^- -
W h i l e these f u n c t i o n s are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l t h e y are also s i n g u l a r
a t t h e o r i g i n a n d therefore n o t useful for o u r p u r p o s e s . H o w e v e r , these f u n c t i o n s
are u p t o s c a l i n g g e n e r a l i z a t i o n s o f t h e Coulomb potential (for 3 = 0 ) , a n d
c a n therefore be g i v e n a p h y s i c a l i n t e r p r e t a t i o n .
a 1
A n o t h e r p o s s i b i l i t y is t o t a k e f(t) = t e~^ , a > 0,/5 > 0. T h e n we get
$ f x = |Ml t f c Q )
- ^r(i+a)r fc) -^ ( e x u 1 Q )
* W /3i+(fc+)/2r(fc+<*+2) e x
i *-- ; L U
(kM ^ ,(k+a+l)/2
{a k)/2 (|Tij) + ( l + o ) M i _ ( f c _ )/
a 2 i ( f c + Q + i)/2 (|NT)) "
Here M M ) I / is t h e W h i t t a k e r - M f u n c t i o n , a c o n f l u e n t h y p e r g e o m e t r i c f u n c t i o n (see,
e.g., Chapter 13 o f [ A b r a m o w i t z a n d S t e g u n (1972)]). W h e n v is a h a l f - i n t e g e r
( w h i c h is, e.g., t h e case for integer k a n d a) f o r m u l a (4.10) s i m p l i f i e s s i g n i f i c a n t l y .
E x a m p l e s for v a r i o u s integer values o f k a n d a are l i s t e d i n T a b l e 4.4. N o t e t h a t
these f u n c t i o n s are n o t defined a t t h e o r i g i n . H o w e v e r , t h e y c a n be m a d e ( o n l y )
c o n t i n u o u s a t t h e o r i g i n . P l o t s o f t w o o f these f u n c t i o n s are p r o v i d e d i n F i g u r e 4.6.
N o t e t h a t o n l y t h e f u n c t i o n s for k > 3 are g u a r a n t e e d t o be s t r i c t l y p o s i t i v e d e f i n i t e
3
and r a d i a l on IR .
Table 4.4 Whittaker radial functions for various choices of k and a.
a fc = 2 fc = 3
2 2
/ ? - |*|| + | | * | | e ~ A -2/3||* | +2||*|| - 2||*|| e~T^f

0
3
P
P- 2||a;| + (/? + 2 | | * | | ) e ~ T W 2
P - 4/3j *|| + 6 | x\\
2
- (2/3\\x\\ + 6 | | * | | ) e " T i r 2
1
3 i
P P
Equation (4.8) a m o u n t s t o another integral transform of / (not listed i n

A p p e n d i x B ) w i t h t h e c o m p a c t l y s u p p o r t e d t r u n c a t e d p o w e r f u n c t i o n as i n t e g r a -
t i o n k e r n e l . W e w i l l t a k e a n o t h e r l o o k a t these f u n c t i o n s i n t h e c o n t e x t o f m u l t i p l y
monotone functions below.
Fig. 4.6 Whittaker radial functions for a = 0 and 3 = 1 with k = 2 (left) and k = 3 (right)
2
centered at the origin in R .
4.8 E x a m p l e 8: I n t e g r a t i o n A g a i n s t S t r i c t l y P o s i t i v e
Definite K e r n e l s
I n fact, i n [ W e n d l a n d (2005a)] i t is s h o w n t h a t i n t e g r a t i o n o f a n y non-negative

f u n c t i o n / t h a t is n o t i d e n t i c a l l y e q u a l t o zero against a f u n c t i o n K(t,-) t h a t is
S
s t r i c t l y p o s i t i v e definite o n R leads t o a n o t h e r f u n c t i o n t h a t is s t r i c t l y p o s i t i v e
S
definite o n R , i.e.,
<p{r)= / K(t,r)f(t)dt
Jo
S
gives rise t o $ = tp(\\ ||) b e i n g s t r i c t l y p o s i t i v e definite o n R . B y c h o o s i n g / and
K a p p r o p r i a t e l y we can o b t a i n b o t h g l o b a l l y s u p p o r t e d a n d c o m p a c t l y supported
functions.
For example, t h e m u l t i p l y m o n o t o n e f u n c t i o n s i n W i l l i a m s o n ' s c h a r a c t e r i z a t i o n
1
T h e o r e m 5.4 are covered b y t h i s general t h e o r e m b y t a k i n g K(t, r) = (l rt)^T and
/ a n a r b i t r a r y p o s i t i v e f u n c t i o n i n L \ so t h a t dfi(t) = f(t)dt. Also, functions t h a t
S
are s t r i c t l y p o s i t i v e definite a n d r a d i a l o n R for a l l s (or e q u i v a l e n t l y c o m p l e t e l y
rt
m o n o t o n e f u n c t i o n s ) are covered b y c h o o s i n g K(t,r) = e~ .
4.9 Summary
T o s u m m a r i z e t h e t h e o r y s u r v e y e d t h u s far w e c a n say t h a t a n y m u l t i v a r i a t e ( r a d i a l )
f u n c t i o n $ whose F o u r i e r t r a n s f o r m is n o n - n e g a t i v e can be used t o g e n e r a t e a basis
for t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m b y s h i f t i n g i t t o t h e d a t a sites. The
f u n c t i o n $ can be p o s i t i v e , o s c i l l a t o r y , o r have c o m p a c t s u p p o r t . H o w e v e r , i f $ has
S
any zeros t h e n i t c a n n o t be s t r i c t l y p o s i t i v e d e f i n i t e o n R for a l l choices of s.
Chapter 5
Completely Monotone and

Multiply Monotone Functions
Since F o u r i e r t r a n s f o r m s are n o t always easy t o c o m p u t e , we n o w present t w o alter-

n a t i v e c r i t e r i a t h a t a l l o w us t o decide w h e t h e r a f u n c t i o n is s t r i c t l y p o s i t i v e d e f i n i t e
s
and radial o n M (one for t h e case o f a l l s, a n d one for o n l y l i m i t e d choices o f s ) .
5.1 Completely Monotone Functions
W e b e g i n w i t h t h e f o r m e r case. T o t h i s e n d we n o w i n t r o d u c e a class o f f u n c t i o n s
t h a t is v e r y closely r e l a t e d t o p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s a n d leads t o a s i m p l e
c h a r a c t e r i z a t i o n o f such f u n c t i o n s .
Definition 5 . 1 . A f u n c t i o n M t h a t is i n C [ 0 , o o ) D C ( 0 , o o ) and

satisfies
)
(-l)V (r) > 0 , r >0, = 0,1,2,...,
is called completely monotone on [0, o o ) .
E x a m p l e 5 . 1 . T h e f u n c t i o n cp(r) = e, s > 0, is c o m p l e t e l y m o n o t o n e o n [0, o o ) .
_ e r
E x a m p l e 5 . 2 . T h e f u n c t i o n </?(r) = e , e > 0, is c o m p l e t e l y m o n o t o n e o n [ 0 , o o )
since
W r
(-l)V ( r ) = e e- > 0, t = 0,1,2,....
E x a m p l e 5 . 3 . T h e f u n c t i o n <p(r) = t j ~ ~ y g > /5 > 0, is c o m p l e t e l y m o n o t o n e o n
[0, oo) since

2 1 3 1
( - l ) y < > ( ) = ( - l ) ' / 3 ( / 3 + 1) (3 + i - 1 ) ( 1 + r y
r - > 0, = 0,1, 2,....
Some p r o p e r t i e s o f c o m p l e t e l y m o n o t o n e f u n c t i o n s t h a t c a n be f o u n d i n [Cheney
a n d L i g h t (1999); Feller ( 1 9 6 6 ) ; W i d d e r (1941)] are:
(1) A non-negative f i n i t e l i n e a r c o m b i n a t i o n o f c o m p l e t e l y m o n o t o n e f u n c t i o n s is
completely monotone.
(2) T h e p r o d u c t o f t w o c o m p l e t e l y m o n o t o n e f u n c t i o n s is c o m p l e t e l y m o n o t o n e .
47
(3) I f 0 for a l l

i > 0 ) , t h e n ip o (p is c o m p l e t e l y m o n o t o n e .
(4) I f <p is c o m p l e t e l y m o n o t o n e a n d ip is a p o s i t i v e f u n c t i o n such t h a t i t s d e r i v a t i v e
is c o m p l e t e l y m o n o t o n e , t h e n <p o ip is c o m p l e t e l y m o n o t o n e .
N o t e t h a t t h e f u n c t i o n s i n t h e second a n d t h i r d e x a m p l e above are, except for

2
a v a r i a b l e s u b s t i t u t i o n r t> r , s i m i l a r t o t h e G a u s s i a n a n d inverse m u l t i q u a d r i c s
m e n t i o n e d earlier. I n o r d e r t o see h o w c o m p l e t e l y m o n o t o n e f u n c t i o n s are r e l a t e d
t o s t r i c t l y p o s i t i v e definite r a d i a l f u n c t i o n s w e r e q u i r e a n i n t e g r a l c h a r a c t e r i z a t i o n
of completely monotone functions.
T h e o r e m 5.1 ( H a u s d o r f f - B e r n s t e i n - W i d d e r ) . A function ip : [0, oo) > R is

completely monotone on [0, oo) if and only if it is the Laplace transform of a finite
non-negative Borel measure \x on [ 0 , o o ) i.e., <p is of the
; form
rOO
rt
<p(r) = /x(r) = / e~ d{t).
Jo
Proof. W i d d e r ' s p r o o f o f t h i s t h e o r e m c a n be f o u n d i n [ W i d d e r ( 1 9 4 1 ) ] , p . 160,

where he reduces t h e p r o o f o f t h i s t h e o r e m t o a n o t h e r t h e o r e m b y H a u s d o r f f o n
c o m p l e t e l y m o n o t o n e sequences. A d e t a i l e d p r o o f c a n also be f o u n d i n t h e b o o k s
[Cheney a n d L i g h t (1999); W e n d l a n d (2005a)].
T h e o r e m 5.1 shows t h a t , i n t h e s p i r i t o f o u r earlier r e m a r k s , t h e f u n c t i o n p(r) =

er
e~ c a n be v i e w e d as t h e f u n d a m e n t a l c o m p l e t e l y m o n o t o n e f u n c t i o n .
T h e following connection between positive definite radial a n d completely mono-
t o n e f u n c t i o n s was first p o i n t e d o u t b y S c h o e n b e r g i n 1938.
T h e o r e m 5.2. A function =
2 s
<p(\\ \\ ) is positive definite and radial on R for all s.
N o t e t h a t t h e f u n c t i o n <3> is n o w defined v i a t h e square o f t h e n o r m . T h i s differs

f r o m o u r d e f i n i t i o n o f r a d i a l f u n c t i o n s (see D e f i n i t i o n 2 . 1 ) .
Proof. O n e p o s s i b i l i t y is t o use a change o f v a r i a b l e s t o c o m b i n e Schoenberg's

s
c h a r a c t e r i z a t i o n o f f u n c t i o n s t h a t are p o s i t i v e d e f i n i t e a n d r a d i a l o n a n y R , T h e o -
r e m 3.8, w i t h t h e H a u s d o r f f - B e r n s t e i n - W i d d e r c h a r a c t e r i z a t i o n o f c o m p l e t e l y m o n o -
t o n e f u n c t i o n s . T o get m o r e i n s i g h t w e present a n a l t e r n a t i v e p r o o f o f t h e c l a i m t h a t
t h e c o m p l e t e l y m o n o t o n e f u n c t i o n <p gives rise t o a $ t h a t is p o s i t i v e d e f i n i t e a n d
s
r a d i a l o n a n y R . D e t a i l s for t h e o t h e r d i r e c t i o n c a n be f o u n d , e.g., i n [ W e n d l a n d
(2005a)].
T h e H a u s d o r f f - B e r n s t e i n - W i d d e r t h e o r e m i m p l i e s t h a t we c a n w r i t e <p as
/OO
r t
tp(r) = / c- d/i(0
Jo
5. Completely Monotone and Multiply Monotone Functions 49
2
w i t h a finite n o n - n e g a t i v e B o r e l m e a s u r e / i . T h e r e f o r e , <&(x) = <^(||cc|j ) has the
representation
/OO
Jo
s
T o see t h a t t h i s f u n c t i o n is p o s i t i v e d e f i n i t e o n a n y R w e consider t h e q u a d r a t i c
form
N N r oo N N
j=l k=l J
j=l k=l
Since we saw earlier t h a t t h e Gaussians are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n

s
any R i t follows t h a t t h e q u a d r a t i c f o r m is n o n - n e g a t i v e .
W e c a n see f r o m t h e p r e v i o u s p r o o f t h a t i f t h e measure LL is n o t c o n c e n t r a t e d
s
at t h e o r i g i n , t h e n <fr is even s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n a n y R . This
c o n d i t i o n o n t h e measure is e q u i v a l e n t w i t h ip n o t b e i n g c o n s t a n t . W i t h t h i s a d d i -
t i o n a l r e s t r i c t i o n o n <p we c a n a p p l y t h e n o t i o n o f a c o m p l e t e l y m o n o t o n e f u n c t i o n
t o t h e scattered d a t a i n t e r p o l a t i o n p r o b l e m . T h e f o l l o w i n g interpolation theorem
o r i g i n a t e s i n t h e w o r k o f Schoenberg ([Schoenberg (1938a)], p . 823) w h o s h o w e d
t h a t c o m p l e t e m o n o t o n i c i t y i m p l i e s s t r i c t p o s i t i v e definiteness, t h u s p r o v i d i n g a
v e r y s i m p l e test for v e r i f y i n g t h e well-posedness o f m a n y s c a t t e r e d d a t a i n t e r p o -
l a t i o n p r o b l e m s . A p r o o f t h a t t h e converse also h o l d s c a n be f o u n d i n [ W e n d l a n d
(2005a)].
T h e o r e m 5.3. A function R is completely monotone but not constant
2 s
if and only if <p{\\ || ) is strictly positive definite and radial on R for any s.
_ e r
E x a m p l e 5.4. Since we showed above t h a t t h e f u n c t i o n s <p(r) = e , s > 0, a n d
p(r) = 1 / ( 1 + r)@, 8 > 0, are c o m p l e t e l y m o n o t o n e o n [0, o o ) , a n d since t h e y are
2
also n o t c o n s t a n t we k n o w f r o m T h e o r e m 5.3 t h a t t h e Gaussians <&(cc) = <>(||cc|| ) =
2 2
e - ^ I M I ^ e > 0, a n d inverse m u l t i q u a d r i c s $(cc) = ^ ( | | a ; | | ) = 1 / ( 1 + | | c c | | ) ^ , 8 > 0,
s
are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R. for a l l s. N o t o n l y is t h e test for
c o m p l e t e m o n o t o n i c i t y a s i m p l e r one t h a n c a l c u l a t i o n o f t h e F o u r i e r t r a n s f o r m s ,
b u t we also are able t o v e r i f y s t r i c t p o s i t i v e definiteness o f t h e inverse m u l t i q u a d r i c s
w i t h o u t a n y dependence o f s o n 8.
5.2 Multiply Monotone Functions
W e can also use m o n o t o n i c i t y t o test for s t r i c t p o s i t i v e definiteness o f r a d i a l func-

S
t i o n s o n 1R for some fixed v a l u e o f s. T o t h i s e n d we i n t r o d u c e t h e concept o f a
multiply monotone function.
D e f i n i t i o n 5 . 2 . A f u n c t i o n p : (0, oo) R w h i c h is i n C ~ ( 0 , o o ) , k > 2, a n d for

f c 2
w h i c h ( !)'(/?(') ( r ) is n o n - n e g a t i v e , n o n - i n c r e a s i n g , a n d convex for I = 0 , 1 , 2 , . . . , k

2 is c a l l e d k-times monotone on ( 0 , o o ) . I n case fc = 1 w e o n l y r e q u i r e p C ( 0 , oo)

t o be n o n - n e g a t i v e a n d n o n - i n c r e a s i n g .
r r 2 r i r
Since c o n v e x i t y o f tp m e a n s t h a t < ^ ( i + ) < ^ ( ) + ^ ( 2 ) ; o r s i p l y p"(r)
m > 0 if
ip" exists, a m u l t i p l y m o n o t o n e f u n c t i o n is i n essence j u s t a c o m p l e t e l y m o n o t o n e
f u n c t i o n whose m o n o t o n i c i t y is " t r u n c a t e d " .
E x a m p l e 5 . 5 . T h e t r u n c a t e d p o w e r f u n c t i o n (c.f. (4.7))
e
<Pi(r) = (1 - r) +
is ^-times m o n o t o n e for a n y since
(-l)Vf(r) = { - ! ) . . . { - 1 + 1)(1 - rf~ l

> 0, I = 0,1,2,..., L
W e saw i n S e c t i o n 4.6 t h a t t h e t r u n c a t e d p o w e r f u n c t i o n s l e a d t o r a d i a l f u n c t i o n s
s
t h a t are s t r i c t l y p o s i t i v e d e f i n i t e o n R p r o v i d e d > [s/2\ + 1.
E x a m p l e 5 . 6 . I f w e define t h e i n t e g r a l o p e r a t o r I b y
/OO
(//)(r)= / f(t)dt, r > 0 , (5.1)

Jr
a n d / is ^-times m o n o t o n e , t h e n If is + 1-times m o n o t o n e . T h i s follows i m m e d i -
a t e l y f r o m t h e f u n d a m e n t a l t h e o r e m o f c a l c u l u s . A s w e w i l l see l a t e r , t h e o p e r a t o r
/ p l a y s a n i m p o r t a n t r o l e i n t h e c o n s t r u c t i o n o f c o m p a c t l y s u p p o r t e d r a d i a l basis
functions.
To make the connection t o s t r i c t l y positive definite r a d i a l functions we require

a n i n t e g r a l r e p r e s e n t a t i o n for t h e class o f m u l t i p l y m o n o t o n e f u n c t i o n s . T h i s was
g i v e n i n [ W i l l i a m s o n (1956)] b u t a p p a r e n t l y a l r e a d y k n o w n t o S c h o e n b e r g i n 1940.
T h e o r e m 5 . 4 ( W i l l i a m s o n ) . A continuous function ip : ( 0 , o o ) R is k-times

monotone on ( 0 , oo) if and only if it is of the form
/OO
1
p(r)= / (l-r*)*- ^*), (5.2)
where IL is a non-negative Borel measure on ( 0 , o o ) .

Proof. T o see t h a t a f u n c t i o n o f t h e f o r m (5.2) is i n d e e d m u l t i p l y m o n o t o n e we
j u s t need t o d i f f e r e n t i a t e u n d e r t h e i n t e g r a l (since d e r i v a t i v e s u p t o o r d e r k 2
_ 1
of (1 r ) k are c o n t i n u o u s a n d b o u n d e d ) . T h e o t h e r d i r e c t i o n c a n be f o u n d i n
[ W i l l i a m s o n (1956)].
W i l l i a m s o n ' s c h a r a c t e r i z a t i o n shows us t h a t j u s t like t h e t r u n c a t e d p o w e r

f u n c t i o n s t h e W h i t t a k e r r a d i a l f u n c t i o n s ( 4 . 1 0 ) i n S e c t i o n 4.7 are b a s e d o n m u l -
t i p l y monotone functions.
For k > oo t h e W i l l i a m s o n c h a r a c t e r i z a t i o n c o r r e s p o n d s t o t h e Hausdorff-
B e r n s t e i n - W i d d e r c h a r a c t e r i z a t i o n T h e o r e m 5.1 o f c o m p l e t e l y m o n o t o n e f u n c t i o n s
5. Completely Monotone and Multiply Monotone Functions 51
( a n d is e q u i v a l e n t p r o v i d e d we e x t e n d W i l l i a m s o n ' s w o r k t o i n c l u d e c o n t i n u i t y a t
the origin).
W e c a n see f r o m Sections 4.6 a n d 4.7 t h a t m u l t i p l y m o n o t o n e f u n c t i o n s give rise
to positive definite r a d i a l functions. S u c h a c o n n e c t i o n was first n o t e d i n [ A s k e y
(1973)] ( a n d i n t h e o n e - d i m e n s i o n a l case b y P o l y a ) u s i n g t h e t r u n c a t e d p o w e r func-
t i o n s o f S e c t i o n 4.6.
I n t h e R B F l i t e r a t u r e t h e f o l l o w i n g t h e o r e m was s t a t e d i n [ M i c c h e l l i ( 1 9 8 6 ) ] ,
a n d t h e n refined i n [ B u h m a n n (1993a)]:
T h e o r e m 5.5 ( M i c c h e l l i ) . Let k = [s/2\+2 be a positive integer. If p : [0, oo)

R , p (E C [ 0 , o o ) , is k-times monotone on (0, oo) but not constant, then ip is strictly
positive definite and radial on R s
for any s such that \_s/2\ < k 2 .
We w o u l d like t o m e n t i o n t h a t several versions o f T h e o r e m 5.5 c o n t a i n m i s -

p r i n t s i n t h e l i t e r a t u r e . T h e c o r r e c t f o r m s h o u l d be as s t a t e d above (c.f. also t h e
g e n e r a l i z a t i o n for s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s , T h e o r e m 9.3).
U s i n g T h e o r e m 5.5 we c a n n o w v e r i f y t h e s t r i c t p o s i t i v e definiteness of the
t r u n c a t e d p o w e r f u n c t i o n s a n d W h i t t a k e r r a d i a l f u n c t i o n s o f Sections 4.6 a n d 4.7
w i t h o u t t h e use o f F o u r i e r t r a n s f o r m s . A g a i n , as for Gaussians a n d t h e Poisson
r a d i a l f u n c t i o n s , w e can v i e w t h e t r u n c a t e d p o w e r f u n c t i o n as t h e fundamental
c o m p a c t l y s u p p o r t e d s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n since i t is o b t a i n e d
u s i n g t h e p o i n t e v a l u a t i o n measure i n W i l l i a m s o n ' s c h a r a c t e r i z a t i o n o f a m u l t i p l y
monotone function.
I t is i n t e r e s t i n g t o observe a c e r t a i n l a c k o f s y m m e t r y i n t h e t h e o r y for c o m p l e t e l y
monotone and m u l t i p l y monotone functions. First, i n the completely monotone
case we c a n use T h e o r e m 5.3 t o c o n c l u d e t h a t i f (p is c o m p l e t e l y m o n o t o n e a n d n o t
2 s
c o n s t a n t t h e n ip{ ) is s t r i c t l y p o s i t i v e d e f i n i t e o n R for a n y s. I n the m u l t i p l y
m o n o t o n e case (see T h e o r e m 5.5) t h e square is m i s s i n g . N o w i t is clear t h a t w e
c a n n o t expect t h e s t a t e m e n t w i t h a square t o be t r u e i n t h e m u l t i p l y monotone
case. T o see t h i s w e consider t h e t r u n c a t e d p o w e r f u n c t i o n ip{r) = ( 1 r ) + ( w h i c h
we k n o w a c c o r d i n g t o E x a m p l e 5.1 above t o be ^-times m u l t i p l y m o n o t o n e for
2 e
a n y ) . However, t h e f u n c t i o n tp(r) = ( 1 r ) + is n o t s t r i c t l y p o s i t i v e d e f i n i t e a n d
s
radial on R for a n y s since i t is n o t even s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R
( a n d therefore even m u c h less so o n a n y h i g h e r - d i m e n s i o n a l space). W e c a n see t h i s
f r o m t h e u n i v a r i a t e r a d i a l F o u r i e r t r a n s f o r m o f ijj (see T h e o r e m B . l o f A p p e n d i x B
w i t h s = 1)
2 e
F^(r) = - 7 = f / (1 - t ) t^J_ (rt)dt
+ 1/2
vi Jo 1
= \/f J
1
(1 - t )2 e
cos(rt)dt
Here we used t h e c o m p a c t s u p p o r t o f ip a n d t h e fact t h a t J_i/ (r) 2 = y/2/ivr cosr.

T h e f u n c t i o n T\ip is o s c i l l a t o r y , a n d therefore ip c a n n o t be s t r i c t l y p o s i t i v e d e f i n i t e
(c.f. T h e o r e m 3.5). I n fact, t h e F o u r i e r t r a n s f o r m T\ip is closely r e l a t e d t o t h e
Poisson r a d i a l f u n c t i o n s o f S e c t i o n 4.3.
M o r e o v e r , i n t h e c o m p l e t e l y m o n o t o n e case w e have a n equivalence b e t w e e n
c o m p l e t e l y m o n o t o n e a n d s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s t h a t are r a d i a l o n a n y
s
R (see T h e o r e m 5.3). A g a i n , we c a n n o t e x p e c t such a n equivalence t o h o l d i n t h e
m u l t i p l y m o n o t o n e case, i.e., t h e converse o f T h e o r e m 5.5 c a n n o t be t r u e . This
is clear since w e have a l r e a d y seen a n u m b e r o f f u n c t i o n s t h a t are s t r i c t l y p o s i t i v e
definite a n d r a d i a l , b u t n o t m o n o t o n e a t a l l n a m e l y t h e o s c i l l a t o r y L a g u e r r e -
Gaussians o f S e c t i o n 4.2 a n d t h e Poisson r a d i a l f u n c t i o n s o f S e c t i o n 4.3.
However, i t is i n t e r e s t i n g t o c o m b i n e t h e Schoenberg T h e o r e m 5.3 a n d T h e -
o r e m 5.5 based o n W i l l i a m s o n ' s c h a r a c t e r i z a t i o n . I f one s t a r t s w i t h t h e s t r i c t l y
2 2
e r
p o s i t i v e d e f i n i t e r a d i a l Gaussian <p(r) = e~ , t h e n T h e o r e m 5.3 tells us t h a t
2
r
4>{r) = <p(y/r) = e~~ is c o m p l e t e l y m o n o t o n e . N o w , a n y f u n c t i o n t h a t is c o m -
p l e t e l y m o n o t o n e is also m u l t i p l y m o n o t o n e o f a n y o r d e r , so t h a t we c a n use T h e o -
2
r
r e m 5.5 a n d c o n c l u d e t h a t t h e f u n c t i o n (p{r) = e~ is also s t r i c t l y p o s i t i v e d e f i n i t e
s
a n d r a d i a l o n M for a l l s. O f course, n o w we c a n r e p e a t t h e a r g u m e n t a n d c o n c l u d e
2 /
y / r s
t h a t ip(r) = e~ is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R for a l l s, a n d so o n
(see [ W e n d l a n d (2005c)]). T h i s r e s u l t was a l r e a d y k n o w n t o Schoenberg (at least
s
i n t h e n o n - s t r i c t case).
As a f i n a l r e m a r k i n t h i s c h a p t e r we m e n t i o n t h a t we are a l o n g w a y f r o m
h a v i n g a c o m p l e t e c h a r a c t e r i z a t i o n o f ( r a d i a l ) f u n c t i o n s for w h i c h t h e scattered
d a t a i n t e r p o l a t i o n p r o b l e m has a u n i q u e s o l u t i o n . A s we w i l l see l a t e r , such a n (as
of n o w u n k n o w n ) c h a r a c t e r i z a t i o n w i l l i n v o l v e also f u n c t i o n s w h i c h are n o t s t r i c t l y
positive definite. For e x a m p l e , we w i l l m e n t i o n a r e s u l t o f M i c c h e l l i ' s a c c o r d i n g
t o w h i c h conditionally p o s i t i v e d e f i n i t e f u n c t i o n s o f o r d e r one c a n be used for t h e
s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m . F u r t h e r m o r e , a l l o f t h e results d e a l t w i t h so
far i n v o l v e r a d i a l basis f u n c t i o n s t h a t are centered a t t h e g i v e n d a t a sites. There
are o n l y l i m i t e d results addressing t h e s i t u a t i o n i n w h i c h t h e centers for t h e basis
f u n c t i o n s a n d t h e d a t a sites m a y differ.
Chapter 6
Scattered Data Interpolation with

Polynomial Precision
6.1 Interpolation with Multivariate Polynomials
As we m e n t i o n e d i n t h e i n t r o d u c t i o n i t is n o t a n easy m a t t e r t o use p o l y n o m i a l s
t o p e r f o r m m u l t i v a r i a t e s c a t t e r e d d a t a i n t e r p o l a t i o n . O n l y i f t h e d a t a sites are i n
c e r t a i n special l o c a t i o n s c a n we g u a r a n t e e well-posedness o f m u l t i v a r i a t e p o l y n o m i a l
i n t e r p o l a t i o n . W e n o w address t h i s p r o b l e m .
S
D e f i n i t i o n 6 . 1 . W e c a l l a set o f p o i n t s X = {x\,... ,x^} C R m-unisolvent if
t h e o n l y p o l y n o m i a l o f t o t a l degree at m o s t m i n t e r p o l a t i n g zero d a t a o n X is t h e
zero p o l y n o m i a l .
T h i s d e f i n i t i o n guarantees a u n i q u e s o l u t i o n for i n t e r p o l a t i o n t o g i v e n d a t a at a
subset o f c a r d i n a l i t y M = ( * ) m s
f t n e
p o i n t s x \ , . . . , XN b y a p o l y n o m i a l o f degree
m. Here M is t h e d i m e n s i o n o f t h e linear space o f p o l y n o m i a l s o f t o t a l degree
less t h a n or equal t o m i n s variables.
S
For p o l y n o m i a l i n t e r p o l a t i o n a t N d i s t i n c t d a t a sites i n R t o be a w e l l - p o s e d
p r o b l e m , t h e p o l y n o m i a l degree needs t o be chosen a c c o r d i n g l y , i.e., we need M =
N, a n d t h e d a t a sites need t o f o r m a n m - u n i s o l v e n t set. T h i s is r a t h e r r e s t r i c t i v e .
2
For example, t h i s i m p l i e s t h a t p o l y n o m i a l i n t e r p o l a t i o n at N = 7 points i n R
can n o t be done i n a u n i q u e w a y since we c o u l d either a t t e m p t t o use b i v a r i a t e
q u a d r a t i c p o l y n o m i a l s (for w h i c h M = 6 ) , or b i v a r i a t e c u b i c p o l y n o m i a l s ( w i t h
M = 10). T h e r e exists n o space o f b i v a r i a t e p o l y n o m i a l s for w h i c h M = 7.
W e w i l l see i n t h e n e x t c h a p t e r t h a t m - u n i s o l v e n t sets p l a y a n i m p o r t a n t role i n
t h e c o n t e x t o f c o n d i t i o n a l l y p o s i t i v e definite f u n c t i o n s . T h e r e , however, even t h o u g h
we w i l l be interested i n i n t e r p o l a t i n g N pieces o f d a t a , t h e p o l y n o m i a l degree w i l l
be s m a l l ( u s u a l l y m = 1, 2, 3 ) , a n d t h e r e s t r i c t i o n s i m p o s e d o n t h e l o c a t i o n s o f t h e
d a t a sites b y t h e u n i s o l v e n c y c o n d i t i o n s w i l l be r a t h e r m i l d .
A sufficient c o n d i t i o n ( t o be f o u n d i n [ C h u i (1988)], C h . 9) o n t h e points
X \ , . . . , xpj t o f o r m a n m - u n i s o l v e n t set i n R 2
is
2
T h e o r e m 6 . 1 . Suppose {Lo,..., L m } is a set o / m + 1 distinct lines in R , and that
U = . . . , UM} is a set of M = ( m + l ) ( m + 2 ) / 2 distinct points such that the
53
first point lies on L Q , the next two points lie on L \ but not on L Q , and so on, so that
the last m + 1 points lie on L m but not on any of the previous lines LQ, . . . , L m _ i .
Then there exists a unique interpolation polynomial of total degree at most m to
arbitrary data given at the points in U. Furthermore, if the data sites {x,..., x^}
2
contain U as a subset then they form an m-unisolvent set on R .
Proof. W e use i n d u c t i o n o n m . F o r m = 0 t h e r e s u l t is t r i v i a l . T a k e R t o be t h e
m a t r i x a r i s i n g f r o m p o l y n o m i a l i n t e r p o l a t i o n a t t h e p o i n t s i n IA, i.e.,
Rjk=Pk(uj), j,k = l,...,M,
w h e r e t h e pk f o r m a basis o f LT^. W e w a n t t o s h o w t h a t t h e o n l y possible s o l u t i o n

t o Rc = 0 is c = 0. T h i s is e q u i v a l e n t t o s h o w i n g t h a t i f p TL^ satisfies
p(t*i)=0, i = l,...,M,
t h e n p is t h e zero p o l y n o m i a l .
For each i = 1 , . . . , m, let t h e e q u a t i o n o f t h e l i n e Li be g i v e n b y
OHX + fay = 7 i ,
2
w h e r e x = (x, y) R .
Suppose n o w t h a t p i n t e r p o l a t e s zero d a t a at a l l t h e p o i n t s U i as s t a t e d above.
Since p reduces t o a u n i v a r i a t e p o l y n o m i a l o f degree m o n L m w h i c h vanishes a t
m + 1 distinct points on L m , i t follows t h a t p vanishes i d e n t i c a l l y o n L m , a n d so
p(x, y) = (a xm + 3y
m - ~/m)q(x, y),
w h e r e q is a p o l y n o m i a l o f degree m 1. B u t n o w q satisfies t h e h y p o t h e s i s o f t h e
x
t h e o r e m w i t h m r e p l a c e d b y m 1 a n d U r e p l a c e d b y U c o n s i s t i n g o f t h e first ( " ^ )
p o i n t s o f U. B y i n d u c t i o n , therefore q = 0, a n d t h u s p = 0. T h i s establishes t h e
uniqueness o f t h e i n t e r p o l a t i o n p o l y n o m i a l . T h e l a s t s t a t e m e n t o f t h e t h e o r e m is
obvious.
A s i m i l a r t h e o r e m was a l r e a d y p r o v e d i n [ C h u n g a n d Y a o ( 1 9 7 7 ) ] . T h e o r e m 6.1
S
c a n be generalized t o R b y using hyperplanes. T h e p r o o f is c o n s t r u c t e d w i t h t h e
h e l p o f a n a d d i t i o n a l i n d u c t i o n o n s. C h u i also gives a n e x p l i c i t expression for t h e
d e t e r m i n a n t o f t h e m a t r i x associated w i t h ( p o l y n o m i a l ) i n t e r p o l a t i o n a t t h e set o f
p o i n t s U.
R e m a r k 6.1. F o r l a t e r reference w e n o t e t h a t ( m l ) - u n i s o l v e n c y o f t h e p o i n t s
x\,..., XN is e q u i v a l e n t t o t h e fact t h a t t h e m a t r i x P w i t h
Pji = Pi{xj), j = l,...,N, I = 1,..., M,
has f u l l ( c o l u m n - ) r a n k . F o r N = M t h i s is t h e p o l y n o m i a l i n t e r p o l a t i o n m a t r i x .
2
Example 6.1. A s can easily be v e r i f i e d , t h r e e c o l l i n e a r p o i n t s i n R are n o t 1-
u n i s o l v e n t , since a linear i n t e r p o l a n t , i.e., a p l a n e t h r o u g h t h r e e a r b i t r a r y h e i g h t s
a t these t h r e e c o l l i n e a r p o i n t s is n o t u n i q u e l y d e t e r m i n e d . O n t h e o t h e r h a n d , i f a
2
set o f p o i n t s i n R c o n t a i n s t h r e e n o n - c o l l i n e a r p o i n t s , t h e n i t is 1-unisolvent.
x 4-
6. Scattered Data Interpolation with Polynomial Precision 55
W e used t h e difficulties associated w i t h m u l t i v a r i a t e p o l y n o m i a l i n t e r p o l a t i o n

as one o f t h e m o t i v a t i o n s for t h e use o f r a d i a l basis f u n c t i o n s . H o w e v e r , s o m e t i m e s
i t is desirable t o have a n i n t e r p o l a n t t h a t e x a c t l y reproduces c e r t a i n t y p e s o f func-
t i o n s . For e x a m p l e , i f t h e d a t a are c o n s t a n t , or come f r o m a l i n e a r f u n c t i o n , t h e n
i t w o u l d be nice i f o u r i n t e r p o l a n t were also c o n s t a n t or linear, respectively. U n f o r -
t u n a t e l y , t h e m e t h o d s we have presented t h u s far (except for t h e d i s t a n c e m a t r i x
fit i n t h e s = 1 case) do n o t r e p r o d u c e these s i m p l e p o l y n o m i a l f u n c t i o n s . More-
over, later o n we w i l l be i n t e r e s t e d i n a p p l y i n g o u r i n t e r p o l a t i o n m e t h o d s t o t h e
n u m e r i c a l s o l u t i o n o f p a r t i a l d i f f e r e n t i a l equations, a n d p r a c t i t i o n e r s (especially o f
finite element m e t h o d s ) o f t e n j u d g e a n i n t e r p o l a t i o n m e t h o d b y i t s a b i l i t y t o pass
t h e so-called patch test. A n i n t e r p o l a t i o n m e t h o d passes t h e s t a n d a r d p a t c h test i f
i t can reproduce linear f u n c t i o n s . I n engineering a p p l i c a t i o n s t h i s t r a n s l a t e s i n t o
exact c a l c u l a t i o n o f c o n s t a n t stress a n d s t r a i n . W e w i l l see later t h a t i n o r d e r t o
prove e r r o r estimates for meshfree a p p r o x i m a t i o n m e t h o d s i t is n o t necessary t o
be able t o r e p r o d u c e p o l y n o m i a l s g l o b a l l y ( b u t l o c a l p o l y n o m i a l r e p r o d u c t i o n is a n
essential i n g r e d i e n t ) . T h u s , i f we are o n l y concerned w i t h t h e a p p r o x i m a t i o n p o w e r
of a n u m e r i c a l m e t h o d t h e r e is r e a l l y n o need for t h e s t a n d a r d p a t c h test t o h o l d .
6.2 E x a m p l e : R e p r o d u c t i o n of L i n e a r F u n c t i o n s U s i n g
Gaussian RBFs
I f we do insist o n r e p r o d u c t i o n o f l i n e a r f u n c t i o n s t h e n t h e t o p p a r t o f F i g u r e 6.1
shows a Gaussian R B F i n t e r p o l a n t (e = 6) t o t h e b i v a r i a t e l i n e a r f u n c t i o n f(x,y) =
(x + y)/2 based o n 1089 u n i f o r m l y spaced p o i n t s i n t h e u n i t square a l o n g w i t h
t h e absolute error. C l e a r l y t h e i n t e r p o l a n t is n o t c o m p l e t e l y p l a n a r n o t even t o
m a c h i n e precision.
F o r t u n a t e l y , t h e r e is a s i m p l e r e m e d y for t h i s p r o b l e m . A l l we need t o d o
is a d d t h e p o l y n o m i a l f u n c t i o n s x i> 1, x i x, and x t> y t o the- basis
_ e - X l _ e - X J V
{e H' H ,...,e H' H } we have t h u s far been u s i n g t o o b t a i n o u r i n t e r -
polant. However, n o w we have N + 3 u n k n o w n s , n a m e l y t h e coefficients c,
k
k = 1 , . . . , N + 3, i n t h e e x p a n s i o n
N
e 2 x 2 2
V (x)
f = ^ 2 c e - k ^ - ^ + c N + 1 + c N + 2 x + c N + 3 y, x = {x,y) e M ,
fc=i
a n d we have o n l y N c o n d i t i o n s t o d e t e r m i n e t h e m , n a m e l y t h e i n t e r p o l a t i o n con-
ditions
V {x )
S 3 = f( ) Xj = ( Xj + )/2,Vj j = 1 , . . . , N.
W h a t can we do t o o b t a i n a ( n o n - s i n g u l a r ) square system? A s we w i l l see b e l o w ,
we can a d d t h e f o l l o w i n g t h r e e c o n d i t i o n s :
N
^ c f c = 0,
fc=i
^CkXk = 0,
fc=l
N
y^c ?/fc = o.
fc
fc=i
How do we have t o m o d i f y o u r e x i s t i n g M A T L A B p r o g r a m for s c a t t e r e d data
i n t e r p o l a t i o n t o i n c o r p o r a t e these m o d i f i c a t i o n s ? I f we p r e v i o u s l y d e a l t w i t h t h e
solution of
Ac = y,
T
with A jk = e -e*\\ - \\^
Xj Xk j k = i , . . . c = [ C l ,... ) C 7 v] , and y =
t
[ / ( c c i ) , . . . , / ( C J V ) ] , t h e n we n o w have t o solve t h e augmented system
' A P~ c V
(6.1)
P T
O d 0
w h e r e A, c, a n d y are as before, a n d Pji = pi(xj), j = 1,..., N, I = 1,..., 3, w i t h

Pi(x) = 1, p {x) 2 = x, a n d pz{x) = y. M o r e o v e r , 0 is a zero v e c t o r o f l e n g t h 3, a n d
O is a zero m a t r i x o f size 3 x 3 .
The MATLAB s c r i p t R B F I n t e r p o l a t i o n 2 D l i n e a r .m shows a n i m p l e m e n t a t i o n
o f t h i s a p p r o a c h for Gaussians ( a l t h o u g h t h e y c a n easily be r e p l a c e d b y a n y o t h e r
RBF) a n d test f u n c t i o n f(x,y) = (x + y)/2. T h e resulting interpolant using iV = 9
e q u a l l y spaced d a t a p o i n t s a n d s = 6 is s h o w n i n t h e b o t t o m p a r t o f F i g u r e 6.1.
Now, w h i l e s t i l l n o t p e r f e c t l y l i n e a r , t h e e r r o r is o n t h e level o f m a c h i n e accuracy.
P r o g r a m 6 . 1 . R B F I n t e r p o l a t i o n 2 D l i n e a r .m
/ R B F I n t e r p o l a t i o n 2 D l i n e a r
0
% S c r i p t t h a t performs 2D RBF i n t e r p o l a t i o n w i t h r e p r o d u c t i o n of
% l i n e a r functions
% Define t h e Gaussian RBF and shape parameter
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; ep = 6;
% Define l i n e a r t e s t f u n c t i o n
2 t e s t f u n c t i o n = @(x,y) (x+y)/2;
% Number and type of d a t a p o i n t s
5
3 N = 9; g r i d t y p e = ' u ;
5
4 name = s p r i n t f ( Data2D_y d%s' ,N, g r i d t y p e ) ; l o a d (name)
o
5 ctrs = dsites;
6 neval = 4 0 ; M = neval~2; g r i d = l i n s p a c e ( 0 , 1 , n e v a l ) ;
7 [xe.ye] = m e s h g r i d ( g r i d ) ; e p o i n t s = [ x e ( : ) y e ( : ) ] ;
% Evaluate the t e s t function a t the data p o i n t s .
8 rhs = testfunction(dsites(:,1).dsites(:,2));
/ Add z e r o s f o r l i n e a r (2D) r e p r o d u c t i o n
0
9 rhs = [rhs; z e r o s ( 3 , l ) ] ;
% Compute d i s t a n c e m a t r i x between t h e d a t a s i t e s and c e n t e r s
% Define 3-column m a t r i x P f o r l i n e a r r e p r o d u c t i o n
12 PM = [ones(N.l) d s i t e s ] ;
% Augment i n t e r p o l a t i o n m a t r i x
J
13 IM = [IM PM; [PM z e r o s ( 3 , 3 ) ] ] ;
% Add column f o r constant r e p r o d u c t i o n
16 PM = [ones(M,l) e p o i n t s ] ; EM = [EM PM];
/ Compute RBF i n t e r p o l a n t
0
% ( e v a l u a t i o n matrix * s o l u t i o n of i n t e r p o l a t i o n system)
17 Pf = EM * ( I M \ r h s ) ;
/ Compute maximum e r r o r on e v a l u a t i o n g r i d
0
20 rms_err = norm(Pf-exact)/neval;
21 fprintf('RMS e r r o r : %e\n', rms_err)
5
22 fprintf('Maximum e r r o r : % e \ n , maxerr)
23 fview = [-30,30];
24 plotsurf(xe,ye,Pf,neval,exact,maxerr,fview);
25 ploterror2D(xe,ye,Pf,exact,maxerr,neval,fview);
N o t e t h a t P r o g r a m 6.1 is a l m o s t t h e same as P r o g r a m 2.1. T h e o n l y difference

are lines 9, 12, 13, a n d 16 t h a t have been a d d e d t o d e a l w i t h t h e a u g m e n t e d p r o b l e m .
I n P r o g r a m 6.1 we also m o d i f i e d t h e d e f i n i t i o n o f t h e test f u n c t i o n .
6.3 Scattered D a t a Interpolation with More General Polynomial

Precision
As we j u s t saw for a specific e x a m p l e , we m a y w a n t t o m o d i f y t h e a s s u m p t i o n on

the f o r m (1.1) o f t h e s o l u t i o n t o t h e s c a t t e r e d d a t a i n t e r p o l a t i o n P r o b l e m 1.1 b y
a d d i n g c e r t a i n p o l y n o m i a l s t o t h e e x p a n s i o n , i.e., Vf is n o w assumed t o be o f t h e
form
N M
(6.2)
Fig. 6.1 Top: Gaussian interpolant to bivariate linear function with N = 1089 (left) and as-
sociated abolute error (right). Bottom: Interpolant based on linearly augmented Gaussians to
bivariate linear function with N = 9 (left) and associated abolute error (right).
w h e r e pi,... ,PM f o r m a basis for t h e M = ( " ^ " l " } " ) - d i m e n s i o n a l l i n e a r space

5
H _i
m
of p o l y n o m i a l s o f t o t a l degree less t h a n o r e q u a l t o m 1 i n s v a r i a b l e s . I t seems

a w k w a r d t o formulate t h i s setup w i t h p o l y n o m i a l s i n n ^ _ l 1 i n s t e a d o f degree m
p o l y n o m i a l s . H o w e v e r , i n l i g h t o f o u r discussion o f c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
f u n c t i o n s i n t h e n e x t c h a p t e r t h i s choice is q u i t e n a t u r a l .
Since e n f o r c i n g t h e i n t e r p o l a t i o n c o n d i t i o n s Vf(xj) = f{xj), j = 1, , N, leads
t o a system of N linear equations i n the N + M unknowns c k a n d di one u s u a l l y
adds t h e M a d d i t i o n a l c o n d i t i o n s
N
c
J2 kPi(xk) = 0, Z = 1,...,M,
fc=i
t o ensure a u n i q u e s o l u t i o n . T h e e x a m p l e i n t h e p r e v i o u s s e c t i o n represents t h e
p a r t i c u l a r case s = m = 2.
W h i l e t h e use o f p o l y n o m i a l s is s o m e w h a t a r b i t r a r y ( a n y o t h e r set o f M l i n e a r l y
i n d e p e n d e n t f u n c t i o n s c o u l d also be u s e d ) , i t is o b v i o u s t h a t t h e a d d i t i o n o f p o l y -
n o m i a l s o f t o t a l degree at m o s t m 1 g u a r a n t e e s p o l y n o m i a l p r e c i s i o n p r o v i d e d t h e
p o i n t s i n X f o r m a n ( m l ) - u n i s o l v e n t set. I n o t h e r w o r d s , i f t h e d a t a c o m e f r o m a
p o l y n o m i a l o f t o t a l degree less t h a n o r e q u a l t o m 1, t h e n t h e y are f i t t e d e x a c t l y

by the expansion (6.2).
I n general, s o l v i n g t h e i n t e r p o l a t i o n p r o b l e m based o n t h e e x t e n d e d expansion
(6.2) n o w a m o u n t s t o s o l v i n g a s y s t e m o f l i n e a r e q u a t i o n s o f t h e f o r m
" A P~ c y
(6.3)
d 0
where t h e pieces are g i v e n b y Ajk = <p(\\xj - Xk\\), j,k = 1 , . . . , N, Pj\ = pi(xj),

j = 1 , . . . , N, I = 1 , . . . ,M, c = [ c i , . . . ,c ] ,
N
T
d = [di,.. .,d ] ,
M
T
V = [yi, -,VN] ,
T
0 is a zero vector o f l e n g t h M, a n d O is a n M x M zero m a t r i x . B e l o w we w i l l

s t u d y t h e i n v e r t i b i l i t y o f t h i s m a t r i x i n t w o steps. F i r s t for t h e case m = 1 i n
T h e o r e m 6.2, a n d t h e n for t h e case o f general m i n T h e o r e m 7.2.
N o t e t h a t we c a n easily m o d i f y t h e M A T L A B p r o g r a m l i s t e d above t o deal w i t h
r e p r o d u c t i o n o f p o l y n o m i a l s o f o t h e r degrees. F o r e x a m p l e , i f we w a n t t o r e p r o d u c e
constants t h e n we need t o replace lines 9, 12, 13, a n d 16 b y
9 rhs = [rhs; 0 ] ;
12 PM = o n e s ( N , l ) ;
13 IM = [IM PM; [PM' 0 ] ] ;
16 PM = ones(M,l); EM = [EM PM];
a n d for r e p r o d u c t i o n o f b i v a r i a t e q u a d r a t i c p o l y n o m i a l s we c a n use
9 rhs = [rhs; zeros(6,l)];

12a PM = [ones(N,l) d s i t e s d s i t e s ( : , 1 ) . " 2 ...
12b dsites(:,2)."2 dsites(:,1).*dsites(:,2)];
13 IM = [IM PM; [PM' z e r o s ( 6 , 6 ) ] ] ;
16a PM = [ones(M,l) e p o i n t s e p o i n t s ( : , 1 ) . " 2 ...
16b epoints(:,2).~2 epoints(:,1).*epoints(:,2)];
16c EM = [EM PM] ;
O f course, these specific examples w o r k o n l y for t h e case s = 2. T h e generaliza-

t i o n t o higher dimensions, however, is o b v i o u s b u t m o r e c u m b e r s o m e .
6.4 Conditionally Positive Definite Matrices and Reproduction of

Constant Functions
W e n o w need t o i n v e s t i g a t e w h e t h e r t h e a u g m e n t e d s y s t e m m a t r i x i n (6.3) is n o n -
singular. T h e special case m = 1 ( i n a n y space d i m e n s i o n s), i.e., r e p r o d u c t i o n o f
constants, is covered b y s t a n d a r d results f r o m l i n e a r algebra, a n d w e discuss i t f i r s t .
D e f i n i t i o n 6 . 2 . A r e a l s y m m e t r i c m a t r i x A is c a l l e d conditionally positive semi-

definite of order one i f its associated q u a d r a t i c f o r m is n o n - n e g a t i v e , i.e.

N N
^2^2c j C k A j k >0 (6.4)
3= 1 k=l
for a l l c = [ c i , . . . , C N ] T
R N
t h a t satisfy
3= 1
I f c ^ 0 i m p l i e s s t r i c t i n e q u a l i t y i n (6.4) t h e n A is c a l l e d conditionally positive

definite of order one.
I n t h e l i n e a r algebra l i t e r a t u r e t h e d e f i n i t i o n u s u a l l y is f o r m u l a t e d u s i n g " < "
i n ( 6 . 4 ) , a n d t h e n A is referred t o as ( c o n d i t i o n a l l y or a l m o s t ) negative definite.
O b v i o u s l y , c o n d i t i o n a l l y p o s i t i v e d e f i n i t e m a t r i c e s o f o r d e r one exist o n l y for N > 1.
W e can i n t e r p r e t a m a t r i x A t h a t is c o n d i t i o n a l l y p o s i t i v e definite o f o r d e r one
as one t h a t is p o s i t i v e d e f i n i t e o n t h e space o f vectors c s u c h t h a t
3= 1
T h u s , i n t h i s sense, A is p o s i t i v e d e f i n i t e o n t h e space o f v e c t o r s c " p e r p e n d i c u l a r "

t o c o n s t a n t functions.
Now we are r e a d y t o f o r m u l a t e a n d p r o v e
Theorem 6.2. Let A be a real symmetric N x N matrix that is conditionally
T
positive definite of order one, and let P = [ 1 , . . . , 1 ] be an N x 1 matrix (column
vector). Then the system of linear equations
' A P~ c y~
T
P 0 d 0
is uniquely solvable.
T
Proof. A s s u m e [c, d] is a s o l u t i o n o f t h e homogeneous l i n e a r system, i.e., with
T T
y = 0 . W e show t h a t [c, d] = 0 is t h e o n l y possible s o l u t i o n .
T
M u l t i p l i c a t i o n o f t h e t o p b l o c k o f t h e (homogeneous) l i n e a r s y s t e m b y c yields
T T
c Ac + dc P = 0.
T T
F r o m t h e b o t t o m b l o c k o f t h e s y s t e m we k n o w P c = cP = 0, a n d therefore
T
c Ac = 0.
Since t h e m a t r i x A is c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r one b y a s s u m p t i o n we
get t h a t c = 0 . F i n a l l y , t h e t o p b l o c k o f t h e h o m o g e n e o u s linear s y s t e m u n d e r
c o n s i d e r a t i o n states t h a t
Ac + dP = 0,
so t h a t c = 0 a n d t h e fact t h a t P is a v e c t o r o f ones i m p l y d = 0.
Since Gaussians ( a n d a n y o t h e r s t r i c t l y p o s i t i v e definite r a d i a l f u n c t i o n ) give rise

t o p o s i t i v e d e f i n i t e m a t r i c e s , a n d since p o s i t i v e d e f i n i t e m a t r i c e s are also c o n d i t i o n -
a l l y p o s i t i v e definite o f o r d e r one, T h e o r e m 6.2 establishes t h e n o n s i n g u l a r i t y o f t h e
( a u g m e n t e d ) r a d i a l basis f u n c t i o n i n t e r p o l a t i o n m a t r i x for c o n s t a n t r e p r o d u c t i o n .
I n order t o cover r a d i a l basis f u n c t i o n i n t e r p o l a t i o n w i t h r e p r o d u c t i o n o f h i g h e r -
order p o l y n o m i a l s we w i l l n o w i n t r o d u c e ( s t r i c t l y ) c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
functions o f o r d e r m .
I
i
Chapter 7
Conditionally Positive Definite Functions
7.1 Conditionally Positive Definite Functions Defined
I n a n a l o g y t o o u r earlier discussion o f i n t e r p o l a t i o n w i t h p o s i t i v e definite f u n c t i o n s

we w i l l n o w i n t r o d u c e c o n d i t i o n a l l y p o s i t i v e d e f i n i t e a n d s t r i c t l y c o n d i t i o n a l l y pos-
i t i v e definite f u n c t i o n s o f o r d e r m. W e w i l l realize t h a t these f u n c t i o n s p r o v i d e t h e
n a t u r a l g e n e r a l i z a t i o n o f R B F i n t e r p o l a t i o n w i t h p o l y n o m i a l r e p r o d u c t i o n discussed
i n the previous chapter. Examples of s t r i c t l y c o n d i t i o n a l l y positive definite (radial)
f u n c t i o n s are p r e s e n t e d i n t h e n e x t c h a p t e r .
Definition 7 . 1 . A c o m p l e x - v a l u e d c o n t i n u o u s f u n c t i o n <& is c a l l e d conditionally

S
positive definite of order m on R if
N N
E ^CjCk&ixj - x k ) > 0 (7.1)

3= 1 k=l
for a n y N p a i r w i s e d i s t i n c t p o i n t s x \ , . . . , XN R , a n d c = [ c i , . . . , c y v ]
S T
& N
satisfying
N
^2cjP(xj) = 0,
3= 1
for any c o m p l e x - v a l u e d p o l y n o m i a l p o f degree at m o s t m 1. T h e f u n c t i o n <E> is
S
called strictly conditionally positive definite of order m on 1R. i f t h e q u a d r a t i c f o r m
(7.1) is zero o n l y for c = 0.
A n i m m e d i a t e o b s e r v a t i o n is
L e m m a 7 . 1 . A function that is (strictly) conditionally positive definite of order

S
m on R is also (strictly) conditionally positive definite of any higher order. In
particular, a (strictly) positive definite function is always (strictly) conditionally
positive definite of any order.
Proof. T h e first s t a t e m e n t f o l l o w s i m m e d i a t e l y f r o m D e f i n i t i o n 7 . 1 . T h e second

s t a t e m e n t is t r u e since t h e case m = 0 y i e l d s t h e class o f ( s t r i c t l y ) p o s i t i v e d e f i n i t e
f u n c t i o n s , i.e., ( s t r i c t l y ) c o n d i t i o n a l l y p o s i t i v e definite f u n c t i o n s o f o r d e r zero are
(strictly) positive definite.
63
A s for p o s i t i v e definite f u n c t i o n s earlier, w e c a n r e s t r i c t ourselves t o r e a l - v a l u e d ,

even f u n c t i o n s <fr a n d real coefficients. A d e t a i l e d discussion is p r e s e n t e d i n [ W e n d -
l a n d (2005a)].
Theorem 7 . 1 . A real-valued continuous even function <3> is called conditionally

s
p o s i t i v e definite o f order m o n M if
N N
E ^2cjC ^(xj k - x k ) > 0 (7.2)
j=i k=i
for any N pairwise distinct points x \ , . . . , x^ R , s

and c = [ c i , . . . , C N ] T
R N
satisfying
N
^r p( ) Cj Xj = 0,
3= 1
for any real-valued polynomial p of degree at most m 1. The function <fr is called
s
s t r i c t l y conditionally positive definite of order m on M if the quadratic form (7.2)
is zero only for c = 0.
T h e m a t r i x A w i t h entries A j k = {xj x) k c o r r e s p o n d i n g t o a r e a l a n d even

s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n $ o f o r d e r m c a n also be i n t e r p r e t e d
as b e i n g p o s i t i v e d e f i n i t e o n t h e space o f v e c t o r s c s u c h t h a t
N
8
^2c (x )jP j = o, p e u ^ .
3= 1
T h u s , i n t h i s sense, A is p o s i t i v e d e f i n i t e o n t h e space o f v e c t o r s c "perpendicular"

t o s-variate p o l y n o m i a l s o f degree a t m o s t m 1.
W e c a n n o w generalize t h e i n t e r p o l a t i o n T h e o r e m G.2 t o t h e case o f g e n e r a l
polynomial reproduction:
Theorem 7 . 2 . If the real-valued even function $ is strictly conditionally positive

definite of order m s
on M. and the points x,..., XN form an ( m l)-unisolvent
set, then the system of linear equations (6.3) is uniquely solvable.
T
Proof. T h e p r o o f is a l m o s t i d e n t i c a l t o t h e p r o o f o f T h e o r e m 6.2. A s s u m e [c, d]
is a s o l u t i o n o f t h e homogeneous l i n e a r s y s t e m , i.e., w i t h y = 0. W e show that
T
[c, d] = 0 is t h e o n l y possible s o l u t i o n .
T
Multiplication of the top block by c yields
T T
c Ac + c Pd = 0.
T T T
F r o m t h e b o t t o m b l o c k o f (6.3) we k n o w P c 0. T h i s implies c P = 0 , and
therefore
T
c Ac = 0. (7.3)
7. Conditionally Positive Definite Functions 65
Since t h e f u n c t i o n $ is s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m b y a s s u m p -
T
t i o n we k n o w t h a t t h e q u a d r a t i c f o r m o f A ( w i t h coefficients such t h a t P c = 0)
above is zero o n l y for c = 0. T h e r e f o r e (7.3) tells us t h a t c = 0. T h e u n i s o l v e n c y o f
t h e d a t a sites, i.e., t h e l i n e a r independence o f t h e c o l u m n s o f P (c.f. R e m a r k 6.1),
a n d t h e fact t h a t c = 0 g u a r a n t e e d = 0 f r o m t h e t o p b l o c k
Ac + Pd = 0
of (6.3).
7.2 C o n d i t i o n a l l y Positive Definite Functions and Generalized

Fourier Transforms
As before, i n t e g r a l c h a r a c t e r i z a t i o n s h e l p us i d e n t i f y f u n c t i o n s t h a t are s t r i c t l y con-

s
d i t i o n a l l y p o s i t i v e definite o f o r d e r m o n I . A n integral characterization o f con-
d i t i o n a l l y p o s i t i v e definite f u n c t i o n s o f o r d e r m, i.e., a g e n e r a l i z a t i o n o f B o c h n e r ' s
t h e o r e m , can be f o u n d i n t h e p a p e r [Sun ( 1 9 9 3 b ) ] . However, since t h e s u b j e c t m a t -
ter is r a t h e r c o m p l i c a t e d , a n d since i t does n o t r e a l l y h e l p us solve t h e s c a t t e r e d
d a t a i n t e r p o l a t i o n p r o b l e m , we d o n o t m e n t i o n a n y details here.
T h e Fourier transform characterization o f s t r i c t l y conditionally positive definite
s
functions o f order m o n M. also makes use o f some a d v a n c e d t o o l s f r o m analy-
sis. However, since t h i s c h a r a c t e r i z a t i o n is relevant for o u r purposes we s t a t e t h e
result (due t o [iske (1994)]) a n d collect some o f t h e m o s t relevant concepts f r o m
distribution theory in Appendix B .
T h i s d i s t r i b u t i o n a l approach originated i n the manuscript [ M a d y c h and Nelson
(1983)]. M a n y m o r e details c a n be f o u n d i n t h e o r i g i n a l papers m e n t i o n e d above as
w e l l as i n t h e b o o k [ W e n d l a n d (2005a)].
T h e o r e m 7.3. Suppose the complex-valued function <& G B possesses a generalized

s
Fourier transform $ of order m which is continuous on M. \ { 0 } . Then $ is strictly
conditionally positive definite of order m if and only if $ is non-negative and non-
vanishing.
T h e o r e m 7.3 states t h a t s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s o n W
are characterized b y t h e o r d e r o f t h e s i n g u l a r i t y o f t h e i r generalized F o u r i e r t r a n s -
f o r m at t h e o r i g i n , p r o v i d e d t h a t t h i s generalized F o u r i e r t r a n s f o r m is n o n - n e g a t i v e
a n d non-zero.
Since i n t e g r a l c h a r a c t e r i z a t i o n s s i m i l a r t o Schoenberg's T h e o r e m s 3.6 a n d 3.8
are so c o m p l i c a t e d i n t h e c o n d i t i o n a l l y p o s i t i v e d e f i n i t e case we do n o t p u r s u e t h e
concept o f a c o n d i t i o n a l l y p o s i t i v e definite r a d i a l f u n c t i o n here. The interested
reader is referred t o [ G u o et al. (1993a)] for details. W e w i l l discuss some e x a m -
ples o f r a d i a l f u n c t i o n s v i a t h e F o u r i e r t r a n s f o r m a p p r o a c h i n t h e n e x t c h a p t e r ,
a n d i n C h a p t e r 9 we w i l l e x p l o r e t h e c o n n e c t i o n between c o m p l e t e l y a n d m u l t i p l y
m o n o t o n e f u n c t i o n s a n d c o n d i t i o n a l l y p o s i t i v e definite r a d i a l f u n c t i o n s .
1
Chapter 8
Examples of Conditionally Positive

Definite Functions
We n o w present a n u m b e r o f s t r i c t l y c o n d i t i o n a l l y p o s i t i v e definite ( r a d i a l ) func-

t i o n s t h a t are covered b y t h e F o u r i e r t r a n s f o r m c h a r a c t e r i z a t i o n T h e o r e m 7.3. T h e
generalized F o u r i e r t r a n s f o r m s for these examples are e x p l i c i t l y c o m p u t e d i n [ W e n d -
l a n d (2005a)]. W e w i l l e s t a b l i s h t h e s t r i c t c o n d i t i o n a l p o s i t i v e definiteness o f these
functions a g a i n i n d e t a i l i n t h e n e x t c h a p t e r w i t h t h e h e l p o f c o m p l e t e l y m o n o t o n e
functions. I n c l u d e d i n t h e examples b e l o w are several o f t h e best k n o w n r a d i a l basic
functions such as t h e m u l t i q u a d r i c due t o [ H a r d y (1971)] a n d t h e t h i n p l a t e spline
due t o [ D u c h o n (1976)].
8.1 E x a m p l e 1: G e n e r a l i z e d M u l t i q u a d r i c s
T h e generalized multiquadrics
2 0 s
$(x) = ( l + WxW ) , x e R, 3 e R \ N ,
0 (8.1)
have generalized F o u r i e r t r a n s f o r m s
9I+/3
/ 3 s / 2
<&M = f 7zîiîr - ^ / 3 + s /2(iiîi)> # ,
of order m = m a x ( 0 , [/?]), w h e r e \3~\ denotes t h e smallest integer greater t h a n o r

equal t o 3. H e r e t h e K v are a g a i n t h e m o d i f i e d Bessel f u n c t i o n s o f t h e second k i n d
o f order v (c.f. Section 4 . 5 ) . N o t e t h a t we need t o exclude p o s i t i v e integer values
o f 3 since t h i s w o u l d lead t o p o l y n o m i a l s o f even degree (see t h e r e l a t e d discussion
in Example 2 below).
Since t h e generalized F o u r i e r t r a n s f o r m s are p o s i t i v e w i t h a s i n g u l a r i t y o f o r d e r
m at t h e o r i g i n , T h e o r e m 7.3 tells us t h a t t h e f u n c t i o n s
f / 3 ] 2
$(x) = (-l) ( 1 + \\x\\ f, 0 < 3 N,
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m \8~\ ( a n d h i g h e r ) .
For 3 < 0 t h e F o u r i e r t r a n s f o r m is a classical one a n d we are b a c k t o t h e
generalized inverse m u l t i q u a d r i c s o f S e c t i o n 4.5. These f u n c t i o n s are a g a i n s h o w n
t o be s t r i c t l y c o n d i t i o n a l l y p o s i t i v e definite o f o r d e r m = 0, i.e., s t r i c t l y p o s i t i v e
definite.
67
Fig. 8.1 Hardy's multiquadric with 3 = | (left) and a generalized multiquadric with 3 = |
2
(right) centered at the origin in R .
F i g u r e 8.1 shows H a r d y ' s " o r i g i n a l " m u l t i q u a d r i c ( w i t h 3 = 1/2, i.e., s t r i c t l y

conditionally positive definite of order 1) a n d a generalized multiquadric with
3 = 5 / 2 (i.e., s t r i c t l y conditionally positive definite of order 3). Note that the
generalized m u l t i q u a d r i c s are no longer " b u m p " f u n c t i o n s (as m o s t o f t h e s t r i c t l y
p o s i t i v e definite f u n c t i o n s w e r e ) , b u t f u n c t i o n s t h a t g r o w w i t h t h e d i s t a n c e f r o m
the origin.
T h e a r g u m e n t s above t o g e t h e r w i t h T h e o r e m 7.2 s h o w t h a t we c a n use H a r d y ' s
multiquadrics i n the form
N
C k 1 + x X f c 2 + d s
V {x)
f = Y^ ^ W - H > x e R ,
k=l
together w i l l the constraint
N
c
E* =
fc=i
t o solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m . The resulting interpolant will
be exact for c o n s t a n t d a t a . A s i n o u r earlier discussions we can scale t h e basis
f u n c t i o n s w i t h a shape p a r a m e t e r e b y r e p l a c i n g ||sr|| b y ||||cc||. T h i s does n o t
affect t h e well-posedness o f t h e i n t e r p o l a t i o n p r o b l e m . H o w e v e r , a s m a l l value o f e
gives rise t o " f l a t " basis f u n c t i o n s , whereas a large value o f e p r o d u c e s v e r y steep
functions. A s before, t h e accuracy o f t h e fit w i l l i m p r o v e w i t h decreasing e w h i l e
t h e s t a b i l i t y w i l l decrease, a n d t h e n u m e r i c a l results w i l l become i n c r e a s i n g l y less
reliable. For F i g u r e 8.1 we used t h e shape p a r a m e t e r e = 1.
B y T h e o r e m 9.7 b e l o w we c a n also solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m
using t h e s i m p l e r e x p a n s i o n
N
2 s
V (x)
f = J2ckV^ + \\x-Xk\\ , x e R .
k=i
T h i s is w h a t H a r d y p r o p o s e d t o d o i n his w o r k i n t h e e a r l y 1970s (see, e.g., [ H a r d y
(1971)]).
8.2 E x a m p l e 2: R a d i a l P o w e r s
T h e radial powers
S
{x) = \\x\f, x e 3R , 0 < 3 2 N , (8.2)
20+S/2Y(S0\
= L_2j.|| ,||-/3-*
a u=AO
{ J 11 11
T(-3/2) ' ^ '
of order m = \3/2~\. Therefore, the functions
$(x) = ( - 1 ) ^ / 2 1 \\xf, 0</32N,
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m = \8/2~\ ( a n d h i g h e r ) .
T h i s shows t h a t t h e basic f u n c t i o n $(cc) = ||a?||2 used for t h e d i s t a n c e m a t r i x
fits i n t h e i n t r o d u c t o r y c h a p t e r are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r
one. A c c o r d i n g t o T h e o r e m 7.2 we s h o u l d have used these basic f u n c t i o n s t o g e t h e r
w i t h an a p p e n d e d c o n s t a n t . H o w e v e r , T h e o r e m 9.7 b e l o w p r o v i d e s t h e j u s t i f i c a t i o n
for t h e i r use as a p u r e distance m a t r i x .
I n F i g u r e 8.2 we show r a d i a l cubics {3 = 3, i.e., s t r i c t l y c o n d i t i o n a l l y p o s i t i v e
definite of o r d e r 2) a n d q u i n t i c s (8 = 5, i.e., s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
of order 3 ) .
N o t e t h a t we h a d t o exclude even powers i n ( 8 . 2 ) . T h i s is clear since a n even
power c o m b i n e d w i t h t h e square r o o t i n t h e d e f i n i t i o n o f t h e E u c l i d e a n n o r m results
i n a p o l y n o m i a l a n d we have a l r e a d y d e c i d e d t h a t p o l y n o m i a l s c a n n o t be used
for i n t e r p o l a t i o n at a r b i t r a r i l y s c a t t e r e d m u l t i v a r i a t e sites.
N o t e t h a t r a d i a l powers are n o t affected b y a s c a l i n g o f t h e i r a r g u m e n t . I n o t h e r
w o r d s , r a d i a l powers are shape parameter free. T h i s has t h e advantage t h a t t h e
user need n o t w o r r y a b o u t f i n d i n g a " g o o d " value o f e. O n t h e o t h e r h a n d , w e w i l l
see below t h a t r a d i a l powers w i l l n o t be able t o achieve t h e s p e c t r a l convergence
rates t h a t are possible w i t h some o f t h e o t h e r basic f u n c t i o n s such as Gaussians a n d
generalized (inverse) m u l t i q u a d r i c s .
2
Fig. 8.2 Radial cubic (left) and quintic (right) centered at the origin in R .
8.3 E x a m p l e 3: T h i n P l a t e Splines
I n t h e p r e v i o u s e x a m p l e we h a d t o r u l e o u t even powers. H o w e v e r , i f t h e even r a d i a l

powers are m u l t i p l i e d b y a l o g t e r m , t h e n w e are b a c k i n business.
D u c h o n ' s thin plate splines (or M e i n g u e t ' s surface splines)
2/3 s
&(x) = ||x|| l o g ||x||, x e R , 3eN, (8.3)
0 + 1 2 l 3 1 + s 2 s 213
= {-l) 2 - / r(B + s/2)8\\\u>\\- -
of o r d e r m = 3 + 1. T h e r e f o r e , t h e f u n c t i o n s
+ 1 2
$(x) = (-l)0 ||x|| 01og||a;||, /?GN,
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m = 3 + 1. I n p a r t i c u l a r , we can
use
N
2 2
Vf(x) = "Y^CkHx - c c | | l o g ||cc - x \\
fc k + di + d x 2 + d y, 3 x = (x,y) G R,
fc=i
together w i l l the constraints
N N N
k=l fc=l k=l

2
t o solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m i n R p r o v i d e d t h e d a t a sites are
not a l l c o l l i n e a r . T h e r e s u l t i n g i n t e r p o l a n t w i l l be e x a c t for d a t a c o m i n g f r o m a
bivariate linear function.
Fig. 8.3 "Classical" thin plate spline (left) and order 3 thin plate spline (right) centered at the
2
origin in R .
F i g u r e 8.3 shows t h e "classical" t h i n p l a t e spline ( w i t h 3 = 1, i.e., strictly

4
c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r 2) a n d t h e o r d e r 3 s p l i n e <>(x) = ||a?|| l o g ||cc||.
N o t e t h a t t h e t h i n p l a t e spline basic f u n c t i o n s are n o t m o n o t o n e . A l s o , b o t h graphs
d i s p l a y e d i n F i g u r e 8.3 c o n t a i n a p o r t i o n w i t h n e g a t i v e f u n c t i o n values.
A s w i t h r a d i a l powers, use o f a shape p a r a m e t e r i n c o n j u n c t i o n w i t h t h i n p l a t e

splines is pointless. F i n a l l y , w e n o t e t h a t t h e families o f r a d i a l powers a n d t h i n p l a t e
splines are o f t e n referred t o c o l l e c t i v e l y as polyharmonic splines.
T h e r e is n o r e s u l t t h a t states t h a t i n t e r p o l a t i o n w i t h t h i n p l a t e splines (or a n y
o t h e r s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n o f o r d e r m > 2) w i t h o u t t h e
a d d i t i o n o f a n a p p r o p r i a t e degree m 1 p o l y n o m i a l is w e l l - p o s e d . T h e o r e m 9.7
q u o t e d several t i m e s before covers o n l y t h e case m = 1.
J
Chapter 9
Conditionally Positive Definite

Radial Functions
A s for s t r i c t l y p o s i t i v e definite r a d i a l f u n c t i o n s , we w i l l be able t o connect s t r i c t l y

c o n d i t i o n a l l y p o s i t i v e definite r a d i a l f u n c t i o n s t o c o m p l e t e l y m o n o t o n e a n d m u l t i p l y
m o n o t o n e functions, a n d t h u s be able t o o b t a i n a c r i t e r i o n for checking c o n d i t i o n a l
p o s i t i v e definiteness o f r a d i a l f u n c t i o n s t h a t is easier t o use t h a n t h e generalized
Fourier t r a n s f o r m i n t h e p r e v i o u s c h a p t e r s .
9.1 Conditionally Positive Definite R a d i a l Functions and

Completely Monotone Functions
I n analogy t o t h e discussion i n S e c t i o n 3.3 we n o w focus o n c o n d i t i o n a l l y p o s i t i v e

S
definite functions t h a t are r a d i a l o n R for a l l s. T h e p a p e r [ G u o et al. (1993a)]
b y G u o , H u a n d Sun c o n t a i n s a n i n t e g r a l c h a r a c t e r i z a t i o n for such f u n c t i o n s . This
c h a r a c t e r i z a t i o n is t o o t e c h n i c a l t o be i n c l u d e d here.
A n o t h e r i m p o r t a n t r e s u l t i n [ G u o et al. (1993a)] is a c h a r a c t e r i z a t i o n o f c o n -
S
d i t i o n a l l y p o s i t i v e definite r a d i a l f u n c t i o n s o n R for a l l s i n t e r m s o f c o m p l e t e l y
monotone functions.
2
T h e o r e m 9 . 1 . Let p> G C [ 0 , oo) n C ( 0 , o o ) . Then the function $ = p(\\ | | ) is
S
conditionally positive definite of order m and radial on R for all s if and only if
m m
(l) <>( ) is completely monotone on ( 0 , o o ) .
Proof. T h e fact t h a t c o m p l e t e m o n o t o n i c i t y i m p l i e s c o n d i t i o n a l p o s i t i v e d e f i n i t e -
ness was p r o v e d i n [ M i c c h e l l i (1986)]. M i c c h e l l i also c o n j e c t u r e d t h a t t h e converse
holds a n d gave a s i m p l e p r o o f for t h i s i n t h e case m = 1. For m = 0 t h i s is Schoen-
S
berg's c h a r a c t e r i z a t i o n o f p o s i t i v e definite r a d i a l functions o n R for a l l s i n t e r m s o f
c o m p l e t e l y m o n o t o n e f u n c t i o n s ( T h e o r e m 5.2). T h e r e m a i n i n g p a r t o f t h e t h e o r e m
is s h o w n i n [Guo et al. (1993a)].
I n order t o get s t r i c t c o n d i t i o n a l p o s i t i v e definiteness we need t o generalize

T h e o r e m 5.3, i.e., t h e fact t h a t ip n o t be c o n s t a n t . T h i s leads t o (see [Wendland
(2005a)])
73
T h e o r e m 9.2. If is strictly conditionally positive definite of order m and radial on M for
all s.
W e c a n n o w m o r e easily v e r i f y t h e c o n d i t i o n a l p o s i t i v e definiteness o f t h e func-

tions listed i n the previous chapter.
E x a m p l e 9.1. T h e f u n c t i o n s
p(r) = ( - 1 ) ^ 1 ( 1 + 7-)", 0 < / 3 N
imply
r / 3 1 e
^)(r) = (-l) W - 1) [8 - I + 1 ) ( 1 + rf-
so t h a t
(_i)r/3i^(r/3i) ( r ) ( / 5 _ ^ + 1 ) ( i + r ) /3-r/3i
is c o m p l e t e l y m o n o t o n e . M o r e o v e r , m \8~\ is t h e smallest possible m s u c h t h a t

m m
( l ) t p ( ) is c o m p l e t e l y m o n o t o n e . Since 8 N we k n o w t h a t o,

S
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m > and radial on R for a l l
values o f s.
E x a m p l e 9.2. T h e f u n c t i o n s
2 2
<p(r) = ( - 1 ) ^ / V ^ , 0<8<2N,
imply
^>(r) = ( - 1 ) ^ 1 f - l ) - . . r ^ 2
^
2
so t h a t ( l ) r ^ / l ^ ( r / 3 / 2 l ) j s c o m p l e t e l y m o n o t o n e a n d m = \B/2~\ is t h e smallest
7 7 1
possible m such t h a t ( l ) ^ ) is c o m p l e t e l y m o n o t o n e . Since /? is n o t a n even
integer ip is n o t a p o l y n o m i a l , a n d therefore, t h e r a d i a l powers (c.f. ( 8 . 2 ) )
W 2 ]
*(||*||) = (-l) \\xf, 3>0, 82N,
S
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m > \8/2\ and radial on R for
a l l s.
E x a m p l e 9.3. T h e t h i n p l a t e splines (c.f. ( 8 . 3 ) )

3 + 1 2 / 3
*(||x||) = ( - l ) ' |||| log||x||, PeN,
S
are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m > 8 + 1 a n d r a d i a l o n R for
a l l s. T o see t h i s we observe t h a t
1 2 3 2
2$(||o ||) = ; (-l)^ || r|| / log(||a || ).
; ;
Therefore, we l e t
<p{r) = (-l^+Vlogr, 8eN,
w h i c h is o b v i o u s l y n o t a p o l y n o m i a l . D i f f e r e n t i a t i n g ip we get
pW(r) +1
= {-lf (3{(3 - l ) . . . ( 8 - e + iy-< logr + p {r),
e 1<<P,
w i t h pe a p o l y n o m i a l o f degree (3 1. T h e r e f o r e , t a k i n g i = (3 we have
<pW(r) = BWogr + C
and
.09+D( ) = ( - l ) / 3 + i [
r >
w h i c h is c o m p l e t e l y m o n o t o n e o n (0, o o ) .
9.2 Conditionally Positive Definite R a d i a l Functions a n d M u l t i p l y

Monotone Functions
F i n a l l y , [ M i c c h e l l i (1986)] p r o v e d a m o r e general v e r s i o n o f T h e o r e m 5.5 r e l a t i n g

s
c o n d i t i o n a l l y p o s i t i v e definite r a d i a l f u n c t i o n s o f o r d e r m o n R ( f o r some fixed
value o f s) a n d m u l t i p l y m o n o t o n e f u n c t i o n s . W e state a stronger version due t o
[ B u h m a n n (1993a)] w h i c h ensures s t r i c t c o n d i t i o n a l p o s i t i v e definiteness.
T h e o r e m 9 . 3 . Let k = [s/2j m + 2 be a positive integer, and suppose ip

m _ 1
C [ 0 , oo) is not a polynomial of degree at most m. If (1) k-times
monotone on (0, oo) but not constant, then ip is strictly conditionally positive definite
s
of order m and radial on M for any s such that [s/2] < k + m 2.
J u s t as we showed earlier t h a t c o m p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s c a n n o t b e
s
s t r i c t l y p o s i t i v e definite o n R for a l l s, i t is i m p o r t a n t t o n o t e t h a t t h e r e are n o
t r u l y c o n d i t i o n a l l y p o s i t i v e definite f u n c t i o n s w i t h c o m p a c t s u p p o r t . M o r e precisely
(see [ W e n d l a n d (2005a)]),
S
T h e o r e m 9 . 4 . Assume that the complex-valued function $ G C ( R ) has compact
support. If & is strictly conditionally positive definite of (minimal) order m, then
m is necessarily zero, i.e., $ is already strictly positive definite.
Proof. T h e hypotheses o n $ ensure t h a t i t is i n t e g r a b l e , a n d therefore i t pos-

sesses a classical F o u r i e r t r a n s f o r m ^> w h i c h is c o n t i n u o u s . F o r i n t e g r a b l e f u n c t i o n s
t h e generalized F o u r i e r t r a n s f o r m coincides w i t h t h e classical F o u r i e r transform.
s
T h e o r e m 7.3 ensures t h a t is n o n - n e g a t i v e o n R \ { 0 } a n d not identically equal
t o zero. B y c o n t i n u i t y we also get <&(0) > 0, a n d T h e o r e m 3.5 shows t h a t $ is
s t r i c t l y p o s i t i v e definite.
T h e o r e m 9.3 t o g e t h e r w i t h T h e o r e m 9.4 i m p l i e s t h a t i f we p e r f o r m ra-fold a n t i -

d i f f e r e n t i a t i o n o n a n o n - c o n s t a n t A;-times m o n o t o n e f u n c t i o n , t h e n we o b t a i n a func-
s
t i o n t h a t is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R for \_s/2\ < k + m 2.
? r r
Example 9.4. T h e f u n c t i o n v fe( ) = ( 1 ~ )+ is fc-times m o n o t o n e (see E x -
a m p l e 5.5 i n S e c t i o n 5.2). To avoid the i n t e g r a t i o n constant for t h e compactly
s u p p o r t e d t r u n c a t e d p o w e r f u n c t i o n we c o m p u t e t h e a n t i - d e r i v a t i v e v i a t h e i n t e g r a l
o p e r a t o r I of E x a m p l e 5.6 i n S e c t i o n 5.2, i.e.,
oo poo / i \fc
/
Mt)dt = j ( l ~ t ) l d t = ) ~ ^ ( l - r ) k
+
+ 1
.
I f we a p p l y m - f o l d a n t i - d i f f e r e n t i a t i o n we get
(
I-Mr) = / / - W ) = ( , + 1 ) ( f c ; 2 > , , ( + m ) ( l- r ) ^ .
T h e r e f o r e , b y T h e o r e m 9.3, t h e f u n c t i o n
<p(r) = ( 1 - r ) * +
s
is s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r m a n d r a d i a l o n R for [s/2\ <
s
k + rn 2, a n d b y T h e o r e m 9.4 i t is even s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n M. .
T h i s was also observed i n E x a m p l e 6 o f C h a p t e r 4. I n fact, we saw t h e r e t h a t <p is
s
s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R. for [s/2j < k + m 1.
W e see t h a t we c a n c o n s t r u c t s t r i c t l y p o s i t i v e d e f i n i t e c o m p a c t l y supported
radial functions by anti-differentiating the t r u n c a t e d power f u n c t i o n . T h i s is es-
sentially the approach taken by Wendland to construct his p o p u l a r compactly
s u p p o r t e d r a d i a l basis f u n c t i o n s . W e p r o v i d e m o r e d e t a i l s o f his c o n s t r u c t i o n i n
Chapter 11.
9.3 Some Special Properties of Conditionally Positive Definite

Functions of O r d e r O n e
Since a n N x N m a t r i x t h a t is c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r one is p o s i t i v e
d e f i n i t e o n a subspace o f d i m e n s i o n N 1 i t has t h e i n t e r e s t i n g p r o p e r t y that
at least N 1 o f i t s eigenvalues are p o s i t i v e . T h i s follows i m m e d i a t e l y f r o m the
C o u r a n t - F i s c h e r t h e o r e m o f l i n e a r a l g e b r a (see e.g., p . 550 o f [ M e y e r ( 2 0 0 0 ) ] ) :
T h e o r e m 9 . 5 ( C o u r a n t - F i s c h e r ) . Let A be a real symmetric N x N matrix with

eigenvalues A i > A2 > > \N, then
T
Afc = max min x Ax
dimV=fc ^GV
II a, 11=1
and
T
Afc = min max x Ax.
dimV=iV-fc+l <*ev
II <= | | = i
W i t h an a d d i t i o n a l a s s u m p t i o n o n A we c a n m a k e a n even s t r o n g e r s t a t e m e n t .
Theorem 9 . 6 . An N x N matrix A which is conditionally positive definite of or-

der one and has a non-positive trace possesses one negative and N 1 positive
eigenvalues.
Proof. L e t A i > A2 > > \ N d e n o t e t h e eigenvalues o f A. F r o m the Courant-

Fischer t h e o r e m we get
XN-I = max m i n x Ax T
> min T
c Ac > 0,
dimV=iV-l =>=V c: E--k=
II a> 11 = 1 l|c||=l
so t h a t A has at least N 1 p o s i t i v e eigenvalues. B u t since tr(A) Ylk=i ^ 0'

A also m u s t have at least one n e g a t i v e eigenvalue.
N o t e t h a t t h e a d d i t i o n a l h y p o t h e s i s o f T h e o r e m 9.6 is satisfied for t h e i n t e r p o -

l a t i o n m a t r i x r e s u l t i n g f r o m ( t h e negative) o f R B F s such as H a r d y ' s multiquadric
or t h e linear r a d i a l f u n c t i o n ip(r) = r since i t s d i a g o n a l elements c o r r e s p o n d t o t h e
value o f t h e basic f u n c t i o n at t h e o r i g i n .
M o r e o v e r , we w i l l n o w use T h e o r e m 9.6 t o conclude t h a t we c a n use radial
functions t h a t are s t r i c t l y c o n d i t i o n a l l y p o s i t i v e definite o f o r d e r one (such as t h e
m u l t i q u a d r i c , 0 < 3 < 1, a n d t h e n o r m basic f u n c t i o n ) without a p p e n d i n g t h e con-
s t a n t t e r m t o solve t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m . T h i s was first p r o v e d
by [ M i c c h e l l i (1986)] a n d m o t i v a t e d b y H a r d y ' s earlier w o r k w i t h multiquadrics
a n d Franke's conjecture t h a t t h e m a t r i x A is n o n - s i n g u l a r i n t h i s case (see [Franke
(1982a)]).
T h e o r e m 9 . 7 ( I n t e r p o l a t i o n ) . Suppose <fr is strictly conditionally positive defi-

nite of order one and that <&(0) < 0. Then for any distinct points x \ , . . . , XN G l s
the matrix A with entries Ajk = &{xj x)k has N 1 positive and one negative
eigenvalue, and is therefore non-singular.
Proof. Clearly, t h e m a t r i x A is c o n d i t i o n a l l y p o s i t i v e definite o f o r d e r one. M o r e -

over, t h e t r a c e o f A is g i v e n b y ti(A) = N<&(0) < 0. Therefore, T h e o r e m 9.6 applies
a n d t h e s t a t e m e n t follows.
A s m e n t i o n e d above, t h i s t h e o r e m covers t h e generalized m u l t i q u a d r i c s <E>(a?) =

(1-f- H ^ l l ) ^ w i t h 0 < 3 < 1 ( w h i c h includes t h e H a r d y m u l t i q u a d r i c ) . T h e t h e o r e m
also covers t h e r a d i a l powers <&(x) = \\x\\P for 0 < 3 < 2 ( i n c l u d i n g t h e E u c l i d e a n
distance f u n c t i o n ) .
A n o t h e r special p r o p e r t y o f a c o n d i t i o n a l l y p o s i t i v e definite f u n c t i o n o f o r d e r
one is
Lemma 9 . 1 . If C is an arbitrary real constant and the real even function $ is

(strictly) conditionally positive definite of order one, then & + C is also (strictly)
conditionally positive definite of order one.
Proof. S i m p l y consider
N N N N N N
Yl J2 j k[$(xj
c c
- x) k + C] = E ^CjCk&ixj - Xk) + E ^CjCkC.
3=1 k=l 3=1 fc=l j=l fc=l
T h e second t e r m o n t h e r i g h t is zero since <3> is c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f
c = a n
o r d e r one, i.e., X ^ y L i j 0' d thus the statement follows.
Chapter 10
Miscellaneous Theory: Other Norms and

Scattered Data Fitting on Manifolds
10.1 Conditionally Positive Definite Functions and p-Norms
I n C h a p t e r 1 we used i n t e r p o l a t i o n w i t h d i s t a n c e m a t r i c e s as a m u l t i v a r i a t e g e n e r a l -
i z a t i o n o f t h e piecewise l i n e a r a p p r o a c h . O u r choice o f t h e distance m a t r i x a p p r o a c h
was m o t i v a t e d b y t h e fact t h a t t h e associated basis f u n c t i o n s , &j{x) = \\x X j \ \
w o u l d satisfy t h e dependence o n t h e d a t a sites i m p o s e d o n a m u l t i v a r i a t e i n t e r p o -
lation m e t h o d by the M a i r h u b e r - C u r t i s theorem. We made the (natural?) choice
o f u s i n g t h e E u c l i d e a n ( 2 - n o r m ) d i s t a n c e f u n c t i o n , a n d t h e n showed i n subsequent
chapters t h a t t h e f u n c t i o n &(x) = \\x\\2 is s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
s
of o r d e r one a n d r a d i a l o n R , a n d t h u s o u r distance m a t r i x a p p r o a c h was i n d e e d
well-posed v i a M i c c h e l l i ' s T h e o r e m 9.7.
W e n o w b r i e f l y consider s o l v i n g t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o b l e m w i t h
r a d i a l f u n c t i o n s based o n o t h e r p - n o r m s . These n o r m s are defined as u s u a l as
1 < p < oo.
T h e c o n t e n t o f t h i s s e c t i o n is m o s t l y t h e s u b j e c t o f t h e p a p e r [ B a x t e r (1991)].
I f we consider o n l y distance m a t r i c e s , i.e., i n t e r p o l a t i o n m a t r i c e s g e n e r a t e d b y
t h e basic f u n c t i o n <&(cc) = ||cc|| , t h e n i t was s h o w n i n [ D y n et al.
p (1989)] t h a t
t h e choice p = 1 leads t o a s i n g u l a r m a t r i x a l r e a d y for v e r y s i m p l e sets o f d i s t i n c t
interpolation points. For e x a m p l e , i f X = { ( 0 , 0 ) , ( 1 , 0 ) , ( 1 , 1 ) , ( 0 , 1 ) } t h e n t h e 1-
n o r m distance m a t r i x is g i v e n b y
"0 1 2 1 "
10 12
2 10 1
12 10
a n d i t is easy t o v e r i f y t h a t t h i s m a t r i x is s i n g u l a r . T h i s r e s u l t has d i s c o u r a g e d
people f r o m u s i n g 1 - n o r m r a d i a l basis f u n c t i o n s .
However, i f we use, e.g., N H a l t o n p o i n t s , t h e n we have never e n c o u n t e r e d a
s i n g u l a r 1-norm distance m a t r i x i n a l l o f o u r n u m e r i c a l e x p e r i m e n t s . I n fact, t h e
79
m a t r i x seems t o have N 1 negative a n d one p o s i t i v e eigenvalue (just as p r e d i c t e d

b y T h e o r e m 9.7 for t h e 2 - n o r m case).
F i g u r e 10.2 shows various i n t e r p o l a n t s t o t h e l i n e a r f u n c t i o n f(x,y) = (x + y)/2
on t h e u n i t square. T h e i n t e r p o l a n t is false c o l o r e d a c c o r d i n g t o t h e m a x i m u m
e r r o r . I n t h e t o p r o w o f t h e figure we used a 1-norm d i s t a n c e m a t r i x based o n 1089
H a l t o n p o i n t s . T h e M A T L A B code for g e n e r a t i n g a p - n o r m distance m a t r i x f i t is
v i r t u a l l y i d e n t i c a l t o o u r earlier code i n P r o g r a m s 1.1 a n d 1.2. T h e o n l y change
r e q u i r e d is t h e replacement o f lines 6 a n d 8 o f P r o g r a m 1.1 b y
6 DM = DM + a b s ( d r - c c ) . " p ;
8 DM = DM."(l/p);
W e can also use t h i s m o d i f i c a t i o n o f P r o g r a m 1.1 t o p r o d u c e m o r e general R B F

i n t e r p o l a n t s (see t h e e x a m p l e w i t h p - n o r m Gaussians i n t h e b o t t o m r o w o f F i g -
u r e 10.2 b e l o w ) .
S i m i l a r t o t h e 1-norm result f r o m [ D y n et al. (1989)] q u o t e d above i t was s h o w n
in [ B a x t e r (1991)] t h a t for p > 2 we c a n n o t i n general g u a r a n t e e n o n - s i n g u l a r
distance m a t r i c e s , either. O n t h e o t h e r h a n d , a n u m b e r o f n u m e r i c a l e x p e r i m e n t s
showed t h e p - n o r m m a t r i c e s t o be n o n - s i n g u l a r p r o v i d e d u n i f o r m l y spaced o r H a l t o n
2
p o i n t s i n [0, l ] were used. T h e second r o w o f F i g u r e 10.2 shows d i s t a n c e m a t r i x
i n t e r p o l a n t s t o f(x,y) = (x + y)/2 o n t h e u n i t square u s i n g a p - n o r m distance
m a t r i x for p = 10 a n d p = 100 based o n 25 u n i f o r m l y spaced p o i n t s .
These examples show t h a t c e r t a i n l y n o t a l l is lost w h e n u s i n g p - n o r m r a d i a l
basis f u n c t i o n s . T h e s i t u a t i o n is s i m i l a r as w i t h t h e use o f K a n s a ' s m e t h o d for t h e
c o l l o c a t i o n s o l u t i o n o f e l l i p t i c P D E s (see C h a p t e r 3 8 ) . T h e r e d o exist c o n f i g u r a t i o n s
of d a t a p o i n t s for w h i c h t h e i n t e r p o l a t i o n m a t r i x becomes s i n g u l a r . H o w e v e r , these
c o n f i g u r a t i o n s m a y be rare, a n d therefore t h e use o f p - n o r m r a d i a l basis f u n c t i o n s
m a y be j u s t i f i e d i n m a n y cases. W e p o i n t o u t t h a t we used n o r m s for p > 2 even
t h o u g h t h e B a x t e r result m e n t i o n e d above guarantees existence o f d a t a sets X for
w h i c h t h e i n t e r p o l a t i o n m a t r i x w i l l be s i n g u l a r . For o u r examples t h e i n t e r p o l a t i o n
m a t r i x was far f r o m s i n g u l a r . U s i n g 25 u n i f o r m l y spaced d a t a sites t h e m a t r i c e s
a g a i n e x h i b i t e d 24 n e g a t i v e a n d one p o s i t i v e eigenvalue. T h i s use o f p - n o r m r a d i a l
basis f u n c t i o n s c e r t a i n l y deserves f u r t h e r i n v e s t i g a t i o n .
T h e case 1 < p < 2, however, is m u c h b e t t e r u n d e r s t o o d . I n [ B a x t e r (1991)] we
find
T h e o r e m 1 0 . 1 . Suppose 1 < p < 2 and let A be the p-norm distance matrix with
entries
Ajk Xk\\p, j,k = l,...,N.
Then the matrix A is conditionally positive definite of order one. Moreover,

it is strictly conditionally positive definite of order one if N > 2 and the points
X i , . . . , XN are distinct.
10. Miscellaneous Theory: Other Norms and Scattered Data Fitting on Manifolds 81
T h i s t h e o r e m is d e r i v e d f r o m a m u c h earlier t h e o r e m b y Schoenberg r e l a t i n g
c o n d i t i o n a l l y p o s i t i v e definite m a t r i c e s o f o r d e r one a n d E u c l i d e a n d i s t a n c e m a t r i -
ces. W h e n Schoenberg first s t u d i e d c o n d i t i o n a l l y p o s i t i v e d e f i n i t e m a t r i c e s o f o r d e r
one t h i s was i n c o n n e c t i o n w i t h i s o m e t r i c e m b e d d i n g s . Based o n earlier w o r k b y
K a r l M e n g e r [Menger (1928)] Schoenberg d e r i v e d t h e f o l l o w i n g result c h a r a c t e r i z i n g
c e r t a i n c o n d i t i o n a l l y p o s i t i v e definite m a t r i c e s as E u c l i d e a n distance m a t r i c e s (see
[Schoenberg (1937)]).
T h e o r e m 10.2 ( S c h o e n b e r g - M e n g e r ) . Let A be a real symmetric N x N ma-

trix with all diagonal entries zero and all other elements positive. Then A is
conditionally positive semi-definite of order one if and only if there exist N points
2/1, , VN R N
for which
Ajk = I I ^ -Vk\\l-
N
These points are the vertices of a simplex in ~R .
I n t h e t h i r d r o w o f F i g u r e 10.2 we d i s p l a y t h e i n t e r p o l a n t s t o t h e test f u n c t i o n
2
f(x, y) = (x+y)/2 o n [0, l ] u s i n g distance m a t r i x i n t e r p o l a t i o n based o n 25 e q u a l l y
spaced p o i n t s a n d p - n o r m s w i t h p = 1.001 a n d p = 2. Since we use a p l a i n d i s t a n c e
i n t e r p o l a n t , i.e., $(x) = ||aj|| p i t is r e m a r k a b l e t h a t t h e e r r o r u s i n g t h e p = 1.001-
n o r m is a b o u t t w o orders o f m a g n i t u d e smaller t h a n t h e n e x t best p - n o r m d i s t a n c e
m a t r i x fit a m o n g o u r e x p e r i m e n t s ( w h i c h we o b t a i n e d for p = 100, c.f. F i g u r e 10.2).
T h e use o f different p - n o r m s for different a p p l i c a t i o n s has n o t been s t u d i e d
carefully i n t h e l i t e r a t u r e .
T w o o t h e r results r e g a r d i n g i n t e r p o l a t i o n w i t h p - n o r m r a d i a l basis f u n c t i o n s
can also be f o u n d i n t h e l i t e r a t u r e . I n [ W e n d l a n d (2005a)] we find a reference t o
[ Z a s t a v n y i (1993)] a c c o r d i n g t o w h i c h for space dimensions s > 3 t h e o n l y
s
f u n c t i o n t h a t is p o s i t i v e d e f i n i t e a n d p - n o r m r a d i a l o n M is t h e zero f u n c t i o n .
A g a i n , s o m e w h a t d i s c o u r a g i n g news. H o w e v e r , t h e r e is also g o o d news. T h e f o l l o w -
i n g r a t h e r p o w e r f u l t h e o r e m comes f r o m [ B a x t e r (1991)]. B a x t e r calls t h e m a t r i x
A o f T h e o r e m 10.2 a n almost negative definite m a t r i x (c.f. the remarks following
D e f i n i t i o n 6.2).
T h e o r e m 1 0 . 3 . Let A be an N x N matrix that is conditionally positive semi-

2
definite of order one with all diagonal entries zero, and let ip( ) be a function that
s
is conditionally positive definite of order one and radial on M . Then the matrix
defined by
B jk =-<p(Ajk), j,k = l,...,N,
is conditionally positive semi-definite of order one. Moreover, if N > 2 and no

off-diagonal elements of A vanish, then B is strictly conditionally positive definite
2
of order one whenever <p( ) is strictly conditionally positive definite of order one.
r a
Proof. B y Schoenberg's T h e o r e m 10.2 w e c a n w r i t e Ajk = \\yj f P"
N 2
p r o p r i a t e p o i n t s yj ~R B y a s s u m p t i o n p{ ) is c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f
o r d e r one a n d r a d i a l , a n d therefore B is c o n d i t i o n a l l y p o s i t i v e s e m i - d e f i n i t e o f o r d e r
one. M o r e o v e r , i f Ajk ^ 0 for a l l o f f - d i a g o n a l elements, t h e n j / i , . . . , ? / A T are d i s t i n c t ,
2
a n d therefore B is s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r one p r o v i d e d (p( )
is s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r one.
Since B a x t e r also shows t h a t i f A is a 1 - n o r m d i s t a n c e m a t r i x , t h e n A is a

c o n d i t i o n a l l y p o s i t i v e s e m i - d e f i n i t e m a t r i x o f o r d e r one, T h e o r e m 10.3 g u a r a n t e e s
t h a t we c a n use m a n y " s t a n d a r d " r a d i a l basic f u n c t i o n s i n c o n j u n c t i o n w i t h t h e 1-
n o r m for R B F i n t e r p o l a t i o n . For e x a m p l e , t h e use o f 1 - n o r m Gaussians is j u s t i f i e d
by T h e o r e m 10.3. I n t h e l i t e r a t u r e one c a n also find a n a n a l o g o f B o c h n e r ' s t h e o r e m
for p o s i t i v e d e f i n i t e 1 - n o r m r a d i a l f u n c t i o n s d u e t o [ C a m b a n i s at al. (1983)] (see
also [ W e n d l a n d ( 2 0 0 5 a ) ] ) .
s x
F i g u r e 10.1 shows p - n o r m Gaussians <3>(:E) = e~ " " p for p = 1 a n d p = 10. A
shape p a r a m e t e r e = 3 was used. I n t e r p o l a n t s t o t h e f u n c t i o n f(x, y) = (x + y)/2 at
2
25 e q u a l l y spaced p o i n t s i n [0, l ] u s i n g these basic f u n c t i o n s w i t h e = 1 are s h o w n
i n t h e b o t t o m r o w o f F i g u r e 10.2.
2
Fig. 10.1 p-norm Gaussians for p = 1 (left) and p = 10 (right) centered at the origin in R .
A n o t h e r , closely r e l a t e d t h e o r e m b y B a x t e r is
2 2
T h e o r e m 1 0 . 4 . Suppose p( ) and ip( ) are functions that are conditionally pos-
S 2
itive definite of order one and radial on ~R with ip(0) = 0. Then ip o ip( ) is also
S 2
conditionally positive definite of order one and radial on 1R . Indeed, if ip(- ) is
strictly conditionally positive definite of order one and radial and tp vanishes only
2
at zero, then ip o ip( ) is strictly conditionally positive definite of order one and
radial.
T h i s t h e o r e m is a g e n e r a l i z a t i o n o f a classical r e s u l t i n l i n e a r a l g e b r a b y Schur
(see, e.g., [ H o r n a n d J o h n s o n ( 1 9 9 1 ) ; M i c c h e l l i ( 1 9 8 6 ) ] , w h e r e Schur's t h e o r e m was
e x t e n d e d t o cover s t r i c t n e s s ) .
10. Miscellaneous Theory: Other Norms and Scattered Data Fitting on Manifolds 83
10.2 Scattered D a t a Fitting on Manifolds
T h e r e exists a sizeable b o d y o f l i t e r a t u r e o n t h e t o p i c o f s c a t t e r e d d a t a i n t e r p o l a t i o n
- 1 s
on m a n i f o l d s , especially t h e sphere S ^ i n M . W e w i l l n o t m e n t i o n a n y specific
results here. I n s t e a d w e refer t h e reader t o t h e b o o k [Freeden et al. (1998)], t h e
survey papers [Cheney (1995a); Fasshauer a n d S c h u m a k e r (1998)], as w e l l as m a n y
o r i g i n a l papers such as [ B a x t e r a n d H u b b e r t (2001); B i n g h a m (1973); Fasshauer
(1995a); Fasshauer (1999b); H u b b e r t a n d M o r t o n (2004a); H u b b e r t a n d M o r t o n
(2004b); Levesley et al. (1999); M e n e g a t t o ( 1 9 9 4 b ) ; N a r c o w i c h a n d W a r d ( 2 0 0 2 ) ;
R a g o z i n a n d Levesley (1996); R o n a n d S u n ( 1 9 9 6 ) ; Schoenberg (1942); Schreiner
(1997); W a h b a (1981); W a h b a (1982); X u a n d C h e n e y ( 1 9 9 2 b ) ] .
R a d i a l basis functions o n m o r e general R i e m a n n i a n m a n i f o l d s are s t u d i e d i n ,
e.g., [ D y n et al. (1997); D y n et al. (1999); Levesley a n d R a g o z i n (2002); N a r c o w i c h
(1995); N a r c o w i c h et al. (2003); S c h i m m i n g a n d Belger (1991)].
T h e r e is also a " p o o r m a n ' s s o l u t i o n " t o i n t e r p o l a t i o n o n m a n i f o l d s , especially
the sphere. O n e can use t h e E u c l i d e a n r a d i a l basis f u n c t i o n m e t h o d s discussed t h u s
far, a n d s i m p l y r e s t r i c t t h e i r e v a l u a t i o n t o t h e m a n i f o l d . T h i s a p p r o a c h is o u t l i n e d
i n Section 6 o f [Fasshauer a n d S c h u m a k e r (1998)].
W e w i l l discuss a n o t h e r , r e l a t e d , i n t e r p o l a t i o n p r o b l e m l a t e r . N a m e l y , i n t e r p o -
3
l a t i o n t o p o i n t c l o u d d a t a i n R . I n t h i s case, t h e u n d e r l y i n g m a n i f o l d is u n k n o w n ,
a n d a n o t h e r a p p r o a c h needs t o be t a k e n . See C h a p t e r 30 for details.
10.3 Remarks
M a n y o f t h e results g i v e n i n t h e p r e v i o u s c h a p t e r s c a n be generalized t o v e c t o r -
v a l u e d or even m a t r i x - v a l u e d f u n c t i o n s . Some results a l o n g these lines c a n be f o u n d
i n [ L o w i t z s c h (2002); L o w i t z s c h (2005); M y e r s (1992); N a r c o w i c h a n d W a r d (1994a);
Schaback (1995a)].
We point out that the approach to solving the interpolation problems taken i n
t h e p r e v i o u s chapters a l w a y s assumes t h a t t h e centers, i.e., the points x , k k =
1,...,N, at w h i c h t h e basis f u n c t i o n s are centered, coincide w i t h t h e d a t a sites.
T h i s is a f a i r l y severe r e s t r i c t i o n , a n d i t has been s h o w n i n examples i n t h e c o n t e x t
o f least squares a p p r o x i m a t i o n o f s c a t t e r e d d a t a (see, e.g., [Franke et al. (1994);
F r a n k e et al. (1995)] or [Fasshauer (1995a)]) t h a t b e t t e r r e s u l t s can be achieved
i f t h e centers are chosen different f r o m t h e d a t a sites. Theoretical results i n this
d i r e c t i o n are v e r y l i m i t e d , a n d are r e p o r t e d i n [ Q u a k et al. (1993)] a n d i n [Sun
(1993a)].
z
0.5
zo.5
2
0.5-
2
Fig. 10.2 p-norm distance matrix fits to f(x, y) = (x + y)/2 on a 5 X 5 grid in [0, l ] unless noted
otherwise. Top: p = 1 (1089 Halton points). 2nd row: p = 10 (left), p = 100 (right). 3rd row:
p = 1.001 (left), p = 2 (right). Bottom: p-norm Gaussian fits for p = 1 (left) and p = 10 (right).
Chapter 11
Compactly Supported
Radial Basis Functions
A s we saw earlier (see T h e o r e m 9.4), c o m p a c t l y s u p p o r t e d f u n c t i o n s <& t h a t are

t r u l y s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f order m > 0 d o n o t exist. The
c o m p a c t s u p p o r t a u t o m a t i c a l l y ensures t h a t <E> is s t r i c t l y p o s i t i v e d e f i n i t e . A n o t h e r
o b s e r v a t i o n (see T h e o r e m 3.9) was t h a t c o m p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s c a n
s
be s t r i c t l y p o s i t i v e definite o n K o n l y for a fixed m a x i m a l s-value. I t is n o t possible
s
for a f u n c t i o n t o be s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R for a l l s a n d also have
a compact support. T h e r e f o r e we focus o u r a t t e n t i o n o n t h e c h a r a c t e r i z a t i o n and
c o n s t r u c t i o n o f f u n c t i o n s t h a t are c o m p a c t l y s u p p o r t e d , s t r i c t l y p o s i t i v e d e f i n i t e
s
a n d r a d i a l o n M for some fixed s.
A c c o r d i n g t o o u r earlier w o r k ( B o c h n e r ' s t h e o r e m a n d g e n e r a l i z a t i o n s t h e r e o f ) , a
s
f u n c t i o n is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R i f i t s s-variate F o u r i e r t r a n s f o r m
is non-negative. T h e o r e m B . l i n t h e A p p e n d i x gives t h e F o u r i e r t r a n s f o r m o f t h e
r a d i a l f u n c t i o n <fr = ip(\\ ||) as a n o t h e r r a d i a l f u n c t i o n
/>oo
2
*(x) = F,<p{\\x\\) = Ija-H-C/ <p(t)r' J - (t\\x\\)dt,
(a 2)/2
Jo
where J v is t h e Bessel f u n c t i o n o f t h e first k i n d o f order v.
11.1 O p e r a t o r s for R a d i a l F u n c t i o n s a n d D i m e n s i o n W a l k s
A certain integral operator a n d i t s inverse d i f f e r e n t i a l o p e r a t o r were defined i n

[Schaback a n d W u (1996)]. I n t h a t p a p e r a n e n t i r e calculus was developed for h o w
these o p e r a t o r s act o n r a d i a l f u n c t i o n s . I n fact, a c c o r d i n g t o [ G n e i t i n g ( 2 0 0 2 ) ] ,
these o p e r a t o r s c a n be t r a c e d b a c k t o JjMatheron (1965)] w h o called t h e i n t e g r a l
o p e r a t o r montee a n d t h e d i f f e r e n t i a l o p e r a t o r descente motivated by an application
related to m i n i n g .
I n t h e f o l l o w i n g we define these o p e r a t o r s a n d show h o w t h e y f a c i l i t a t e t h e
construction of compactly supported r a d i a l functions.
85
Definition 11.1.
(1) L e t (fi be such t h a t t i t<p(t) G L i [ 0 , o o ) . T h e n we define t h e integral operator

X via
(T<p)(r) = / f<p(t)dt, r > 0.

J r
2
(2) For even 0.
I n b o t h cases t h e r e s u l t i n g f u n c t i o n s are t o be i n t e r p r e t e d as even functions

u s i n g even extensions.
N o t e t h a t t h e i n t e g r a l o p e r a t o r X differs f r o m t h e o p e r a t o r / i n t r o d u c e d earlier
(see (5.1)) b y a f a c t o r t i n t h e i n t e g r a n d .
T h e m o s t i m p o r t a n t p r o p e r t i e s o f t h e m o n t e e a n d descente o p e r a t o r s are (see,
e.g., [Schaback a n d W u (1996)] o r [ W e n d l a n d ( 1 9 9 5 ) ] ) :
Theorem 11.1.
(1) Both T> andX preserve compact support, i.e., if if has compact support, then so
do T>p and Xp.
(2) IfpE C(R) and t ^ t(f>(t) G L i [ 0 , o o ) , then VX 3, then F (<p) s = JF _ (X^).
S 2
2 s
(5) Ifpe C (R) is even and 11-+ t <p'(t) G L i [ 0 , o o ) , then F (ip) 8 = T i{pp).
s+
T h e o p e r a t o r s X a n d V a l l o w us t o express s - v a r i a t e F o u r i e r t r a n s f o r m s as (s2)-
o r ( s + 2 ) - v a r i a t e F o u r i e r t r a n s f o r m s , respectively. I n p a r t i c u l a r , a d i r e c t consequence
o f t h e above p r o p e r t i e s a n d t h e c h a r a c t e r i z a t i o n o f s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l
f u n c t i o n s ( T h e o r e m 3.6) is
Theorem 11.2.
s l
(1) Suppose cp G C(R). Ift ^ t ~ p(t) G Z a [ 0 , o o ) and s > 3, then <p is strictly
s
positive definite and radial on R if and only if Xp is strictly positive definite and
s + 2
radial on R .
T h i s a l l o w s us t o c o n s t r u c t n e w s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s from
g i v e n ones b y a " d i m e n s i o n - w a l k " t e c h n i q u e t h a t steps t h r o u g h m u l t i v a r i a t e E u -
c l i d e a n space i n even i n c r e m e n t s . T h e e x a m p l e s p r e s e n t e d i n t h e f o l l o w i n g sections
illustrate this technique.
.1,
11. Compactly Supported Radial Basis Functions 87
11.2 Wendland's Compactly Supported Functions
p r o b a b l y the most popular family of compactly supported r a d i a l functions presently

i n use was c o n s t r u c t e d i n [ W e n d l a n d (1995)]. W e n d l a n d s t a r t s w i t h t h e t r u n c a t e d
e
power f u n c t i o n <pe(r) = ( 1 r) + ( w h i c h we k n o w t o be s t r i c t l y p o s i t i v e d e f i n i t e a n d
s
r a d i a l o n M for > |_J + 1 a c c o r d i n g t o S e c t i o n 4 . 6 ) , a n d t h e n he w a l k s t h r o u g h
dimensions b y r e p e a t e d l y a p p l y i n g t h e o p e r a t o r X.
e
Definition 11.2. W i t h <pe(r) = ( 1 r) + we define
k(
Vs,k = 1 P\_s/2\+k+l-
I t t u r n s o u t t h a t t h e f u n c t i o n s (p k St are a l l s u p p o r t e d o n [0,1] a n d have a p o l y -

n o m i a l r e p r e s e n t a t i o n t h e r e . M o r e precisely,
s
T h e o r e m 1 1 . 3 . The functions (p kSj are strictly positive definite and radial on R
and are of the form
s k K
^ ' ' \0, r > 1,
2 f e
with a univariate polynomial pk 3t of degree [s/2\ + Sk + 1. Moreover, (p k
Sj C (IR)
are unique up to a constant factor, and the polynomial degree is minimal for given
space dimension s and smoothness 2k.
2k
T h i s t h e o r e m states t h a t a n y o t h e r c o m p a c t l y s u p p o r t e d C p o l y n o m i a l func-
t i o n t h a t is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n W w i l l n o t have a smaller p o l y -
n o m i a l degree. O u r o t h e r e x a m p l e s b e l o w ( W u ' s f u n c t i o n s , G n e i t i n g ' s f u n c t i o n s )
i l l u s t r a t e t h i s fact. T h e s t r i c t p o s i t i v e definiteness of Wendland's functions ip k
Sj
s t a r t i n g w i t h non-integer values o f i n D e f i n i t i o n 11.2 was established i n [ G n e i t i n g

(1999)]. N o t e , however, t h a t t h e n t h e f u n c t i o n s are no longer g u a r a n t e e d t o be
polynomials on their support.
W e n d l a n d gave recursive f o r m u l a s for t h e f u n c t i o n s ,,i(r) = ( l - r [(* + l ) r + l ] ,
e 2 2 2
Vs,2{r) = ( 1 - r) + [( + 4 + 3 ) r + (3 + 6 ) r + 3] ,
3 3 2 3 2 2
ip (r)
at3 = (1 - rY+ [{ + 9 + 23 + 1 5 ) r + {U + 3Q + 4 5 ) r
+ (15^ + 4 5 ) r + 1 5 ] ,
where = [s/2\ + k + 1, and the symbol = denotes equality up to a multiplicative

positive constant.
Proof. T h e case k = 0 follows d i r e c t l y f r o m t h e d e f i n i t i o n . Application of the

d e f i n i t i o n for t h e case k = 1 y i e l d s
oo
t<p (t)dt
/
e
oo
- l
t(l -
t) dt +
e
= f t(l-t) dt
J r
+ 1
= ( , + 1 )(, + 2 ) ( i - r ) ' [ ( ^ i ) r + i ] ,
w h e r e t h e c o m p a c t s u p p o r t o f <f reduces t h e i m p r o p e r i n t e g r a l t o a d e f i n i t e i n t e g r a l
w h i c h c a n be e v a l u a t e d u s i n g i n t e g r a t i o n b y p a r t s . T h e o t h e r t w o cases are o b t a i n e d
similarly b y repeated application o f X.
Example 1 1 . 1 . F o r s = 3 we l i s t some o f t h e m o s t c o m m o n l y used f u n c t i o n s i n

s
T a b l e 1 1 . 1 . T h e s e f u n c t i o n s are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n M. for s < 3.
W e also l i s t t h e i r degree o f s m o o t h n e s s 2k. T h e f u n c t i o n s were d e t e r m i n e d using
t h e f o r m u l a s f r o m T h e o r e m 11.4, i.e., f o r k = 1, 2 , 3 t h e y m a t c h D e f i n i t i o n 11.2 o n l y
u p t o a positive constant factor.
For t h e M A T L A B i m p l e m e n t a t i o n i n t h e n e x t c h a p t e r i t is b e t t e r t o express
t h e c o m p a c t l y s u p p o r t e d f u n c t i o n s i n a s h i f t e d f o r m since w e w i l l be u s i n g a m a t r i x
v e r s i o n o f 1er i n place o f t h e code used earlier i n D i s t a n c e M a t r i x . m for r . T h u s w e
also list t h e a p p r o p r i a t e f u n c t i o n s ,/c(l ) so t h a t > fc(l er) =
s s S) p> ,k{^f).
s
For c l a r i f i c a t i o n purposes w e r e i t e r a t e t h a t expressions o f t h e f o r m (x)+ are t o be

e
i n t e r p r e t e d as ((x)+) , i.e., w e first a p p l y t h e c u t o f f f u n c t i o n , a n d t h e n t h e p o w e r .
a n
Table 11.1 Wendland's compactly supported radial functions v's.fc d *?s,k Vs,fc(l " )
for various choices offcand s 3.
k <P3,k( ) r
&3,k(r) smoothness
r C
0 i
2
1 (1 - r)\ (4r + 1) r\ (5 - 4 r ) C
2 2 4
2 (1 - r)\ ( 3 5 r + 18r + 3) r\ (56 - 88r + 3 5 r ) C
3 2 2 3 6
3 (1 - r)\ ( 3 2 r + 2 5 r + 8r + l ) r \ (66 - 154r + 1 2 1 r - 3 2 r ) C
11.3 Wu's Compactly Supported Functions
I n [ W u (1995b)] w e f i n d a n o t h e r w a y t o c o n s t r u c t s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l
functions w i t h compact support. W u starts w i t h t h e f u n c t i o n
2 e
i>(r) = (1 - r ) , + eN,
w h i c h i n itself is n o t p o s i t i v e d e f i n i t e (see t h e discussion at t h e e n d o f C h a p t e r 5 ) .

However, W u t h e n uses c o n v o l u t i o n t o c o n s t r u c t a n o t h e r f u n c t i o n t h a t is s t r i c t l y
p o s i t i v e definite a n d r a d i a l o n R , i.e.,
ipi(r) = (ip * ip)(2r)

2 e 2
( 1 - t ) (l + - (2r - t) Y dt
+
-oo
2
= f ( l - t Y { l - { 2 r - t Y Y + d t .
T h i s f u n c t i o n is s t r i c t l y p o s i t i v e d e f i n i t e since i t s F o u r i e r t r a n s f o r m is essentially
the square o f t h e F o u r i e r t r a n s f o r m o f ip a n d therefore n o n - n e g a t i v e . J u s t like t h e
W e n d l a n d functions, t h i s f u n c t i o n is a p o l y n o m i a l o n i t s s u p p o r t . I n fact, t h e degree
2e
of t h e p o l y n o m i a l is 4 + 1, a n d ip e e C (R).
N o w , a f a m i l y o f s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s is c o n s t r u c t e d b y a
d i m e n s i o n w a l k u s i n g t h e T> o p e r a t o r .
2 2
D e f i n i t i o n 11.3. W i t h ip {r)e = ( ( 1 - - Y+ * ( 1 - - ) + ) ( 2 r ) we define
k
iP t K = V ip . t
s
T h e f u n c t i o n s ipk,e are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R for s < 2k + 1,
2 k
are p o l y n o m i a l s o f degree A 2k+l o n t h e i r s u p p o r t a n d i n C ^~ ^ i n the interior
2 k
of t h e s u p p o r t . O n t h e b o u n d a r y t h e smoothness increases t o C ~.
E x a m p l e 1 1 . 2 . For = 3 we c a n c o m p u t e t h e four f u n c t i o n s
k k 2 2
W ) = V ip {r) 3 = V ((l - -f + * (1 - - ) ) ( 2 r ) , + k = 0,1,2,3.
T h e y are l i s t e d i n T a b l e 11.2 a l o n g w i t h t h e i r smoothness. T h e m a x i m a l space

d i m e n s i o n s for w h i c h these f u n c t i o n s are s t r i c t l y p o s i t i v e definite a n d r a d i a l o n
s
R is also l i s t e d . J u s t as w i t h the W e n d l a n d f u n c t i o n s , t h e f u n c t i o n s i n T a b l e 11.2
m a t c h t h e d e f i n i t i o n o n l y u p t o a p o s i t i v e m u l t i p l i c a t i v e c o n s t a n t . A g a i n , we also list
the functions ip ,i k = ipk,(l ) used i n o u r M A T L A B i m p l e m e n t a t i o n i n C h a p t e r 12.
T h i s r e p r e s e n t a t i o n o f t h e W u f u n c t i o n s is g i v e n i n T a b l e 11.3.
Table 11.2 Wu's compactly supported radial functions ipk,e for various choices of
fc and I = 3.
fc smoothness s
0 2
( l - r ) + ( 5 - r 35r + 101r + 147r + 101r + 3 5 r + 5 r )
3 4 5 6
1
1 2
( 1 - r ) ( 6 + 36r + 8 2 r + 7 2 r + 3 0 r + 5 r )
+
3 4 5
c 4
3
2
2 (1 - r ) + ( 8 + 40r + 4 8 r + 2 5 r + 5 r )
2 3 4
c 5
3 4
(1 - r ) (16 + 29r + 2 0 r + 5 r ) 2 3
c 7
Table 11.3 Shifted version ipk,e of Wu's compactly supported radial functions tpk,e
for various choices of k and = 3.
r
k ^k,zi ) smoothness s
2 3 4 5 6
0 r ( 4 2 9 - 1287r + 1573r - l O O l r + 3 5 1 r - 6 5 r + 5 r )
+ C 6
1
2 3 4 5
1 r ( 2 3 1 - 561r + 528r - 242r + 5 5 r - 5 r )
+ C 4
3
2 3 4
2 r ( 1 2 6 - 231r + 153r - 4 5 r + 5 r )
+ C 2
5
4 2 3
3 r (70 - 84r + 3 5 r - 5 r ) C 7
Fig. 11.1 Plot of Wendland's functions from Example 11.1 (left) and Wu's functions from E x a m -
ple 11.2 (right).
As predicted by Theorem 11.3, for a p r e s c r i b e d smoothness the polynomial

degree o f W e n d l a n d ' s f u n c t i o n s is l o w e r t h a n t h a t o f W u ' s f u n c t i o n s . F o r e x a m p l e ,
4
b o t h W e n d l a n d ' s f u n c t i o n (p 3j2 a n d W u ' s f u n c t i o n ipi j3 are C smooth and strictly
3
p o s i t i v e definite a n d r a d i a l o n I R . H o w e v e r , t h e p o l y n o m i a l degree o f W e n d l a n d ' s
f u n c t i o n is 8, whereas t h a t o f W u ' s f u n c t i o n is 1 1 . A n o t h e r c o m p a r a b l e f u n c t i o n
4
is G n e i t i n g ' s o s c i l l a t o r y f u n c t i o n a 2 (see T a b l e 11.5), w h i c h is a C polynomial of
3
degree 9 t h a t is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R .
W h i l e t h e t w o families o f s t r i c t l y p o s i t i v e d e f i n i t e c o m p a c t l y s u p p o r t e d functions
discussed above are b o t h c o n s t r u c t e d v i a d i m e n s i o n w a l k , W e n d l a n d uses i n t e g r a t i o n
( a n d t h u s o b t a i n s a f a m i l y o f i n c r e a s i n g l y s m o o t h e r f u n c t i o n s ) , whereas W u needs
t o s t a r t w i t h a f u n c t i o n o f sufficient s m o o t h n e s s , a n d t h e n o b t a i n s successively less
s m o o t h functions (via differentiation).
11.4 Oscillatory Compactly Supported Functions
O t h e r s t r i c t l y p o s i t i v e d e f i n i t e c o m p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s have b e e n p r o -
posed b y G n e i t i n g (see, e.g., [ G n e i t i n g ( 2 0 0 2 ) ] ) . H e s h o w e d t h a t a f a m i l y o f o s c i l l a -
t o r y c o m p a c t l y s u p p o r t e d f u n c t i o n s c a n be c o n s t r u c t e d u s i n g t h e so-called turning
bands operator o f [ M a t h e r o n ( 1 9 7 3 ) ] . S t a r t i n g w i t h a f u n c t i o n ip s t h a t is s t r i c t l y
s
p o s i t i v e definite a n d r a d i a l o n M. for s > 3 t h e t u r n i n g b a n d s o p e r a t o r p r o d u c e s
^ - ( r ) = ^ (r) + î^
2 s (11.1)
s _ 2
w h i c h is s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R .
Example 1 1 . 3 . O n e such f a m i l y o f f u n c t i o n s is g e n e r a t e d is we s t a r t w i t h the

W e n d l a n d functions (p +2,i(f) s = ( 1 r ) ^ " [( + l ) r + 1] ( n o n - i n t e g e r
1
allowed).
A p p l i c a t i o n of the t u r n i n g bands operator results i n the functions
( \ fi V (-\ ^ o ( l + l ) ( l + 2 + s) 2 \
s
w h i c h are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n M p r o v i d e d > (see [ G n e i t i n g
(2002)]). Some specific f u n c t i o n s f r o m t h i s f a m i l y are l i s t e d i n T a b l e 11.4. A l l o f
2
t h e f u n c t i o n s are i n C ( 1 R ) . I f we w a n t s m o o t h e r f u n c t i o n s , t h e n we need t o s t a r t
w i t h a s m o o t h e r W e n d l a n d f a m i l y as d e s c r i b e d b e l o w i n E x a m p l e 11.4.
Table 11.4 Gneiting's compactly supported radial

functions r for various choices of and s = 2.
s
T r
^ 2,e( ) smoothness
7 i | 5
7/2 (1 - r) ^ (1 + \r - r 2 )
C 2
2 2
5 (1 - r)\ (1 + 5r - 2 7 r ) C
15/2 ( l - r ^ l + f r - _3|i r 2 ) C 2
12 (i - 0 + ( i + 1 2 r
- 104r )
2
C 2
T h e f u n c t i o n s o f T a b l e 11.4 are s h o w n i n t h e left p l o t o f F i g u r e 11.2 w i t h

increasing f r o m t h e o u t s i d e i n (as v i e w e d near t h e o r i g i n ) .
Fig. 11.2 Oscillatory functions of Table 11.4 (left) and Table 11.5 (right).
92 Meshfree Approximation Methods urith M A T L A B
E x a m p l e 1 1 . 4 . A l t e r n a t i v e l y , we c a n o b t a i n a set o f o s c i l l a t o r y f u n c t i o n s t h a t are
3
s t r i c t l y positive definite and r a d i a l o n M b y applying the t u r n i n g bands operator
t o t h e W e n d l a n d f u n c t i o n s </?5,fc t h a t are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n R 5
for different choices o f A:. T h e n t h e r e s u l t i n g f u n c t i o n s a k w i l l have t h e same degree

o f s m o o t h n e s s 2k as t h e o r i g i n a l f u n c t i o n s a n d t h e y w i l l be s t r i c t l y p o s i t i v e d e f i n i t e
3
a n d r a d i a l o n R . T h e r e s u l t s for k = 1,2,3 are l i s t e d i n T a b l e 11.5 a n d d i s p l a y e d
i n t h e r i g h t p l o t o f F i g u r e 11.2.
Table 11.5 Oscillatory compactly supported functions that are

3
strictly positive definite and radial on R parametrized by smooth-
ness.
k Ok(r) smoothness
1 (1 - r)\ (1 + 4 r - 1 5 r ) 2
C 2
2 3 4
2 (1 - r)\ (3 + 18r + 3 r - 1 9 2 r ) C
2 3 4 6
3 (1 - r ) \ (15 + 120r + 2 1 0 r - 8 4 0 r - 3 4 6 5 r ) C
G n e i t i n g also suggests t h e c o n s t r u c t i o n o f s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l func-

t i o n s b y t a k i n g t h e p r o d u c t o f t h e ( a p p r o p r i a t e l y scaled) Poisson f u n c t i o n s Q, s (see
e i t h e r T h e o r e m 3.6 or S e c t i o n 4.3) w i t h a c e r t a i n c o m p a c t l y s u p p o r t e d non-negative
f u n c t i o n (see [ G n e i t i n g (2002)] for m o r e d e t a i l s ) . B y P r o p e r t y (6) o f T h e o r e m 3.1
t h e r e s u l t i n g f u n c t i o n w i l l be s t r i c t l y p o s i t i v e d e f i n i t e .
11.5 Other Compactly Supported Radial Basis Functions
T h e r e are m a n y o t h e r w a y s i n w h i c h one c a n c o n s t r u c t c o m p a c t l y s u p p o r t e d func-

s
t i o n s t h a t are s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n M . I n [Schaback (1995a)]
several such p o s s i b i l i t i e s are d e s c r i b e d .
E x a m p l e 1 1 . 5 . Euclid's hat f u n c t i o n s are c o n s t r u c t e d i n a n a l o g y t o S - s p l i n e s . I t is

w e l l k n o w n t h a t t h e u n i v a r i a t e f u n c t i o n (3(r) = ( 1 | r | ) + is a second-order jE?-spline
w i t h k n o t s a t 1 , 0 , 1 , a n d i t is o b t a i n e d as t h e c o n v o l u t i o n o f t h e characteristic
f u n c t i o n o f t h e i n t e r v a l [ 1 / 2 , 1 / 2 ] w i t h itself. E u c l i d ' s h a t f u n c t i o n s are n o w o b -
tained by convolving the characteristic function of the s-dimensional Euclidean u n i t
b a l l w i t h itself. T h e r e s u l t i n g f u n c t i o n s c a n be w r i t t e n for r G [ 0 , 1 ] i n t h e f o r m
f 27ry 2 f c _ (2r)-r(l-r )
1
2 f c
r. _ -i o q
^ f c + 1 (2r) = K - 1 , 2 , 6 , . . . ,
12(1-r) fc = 0,
for o d d space d i m e n s i o n s s = 2k + 1, a n d as
[ 2 ry 7 2 f c (2r)-TV(l-r2)(l-r ) 2 f c
, _
>2fc+ (2r) = I
2 ^+2 * - 1, A 6,
2 ( a r c c o s r ry/l r) 2
k = 0,
for even space d i m e n s i o n s s = 2k. N o t e t h a t these f u n c t i o n s are zero o u t s i d e the

interval [0,2].
W e have l i s t e d several o f these f u n c t i o n s i n T a b l e 11.6 w h e r e w e have e m p l o y e d
a s u b s t i t u t i o n 2r > r a n d a n o r m a l i z a t i o n f a c t o r such t h a t t h e f u n c t i o n s a l l have
a v a l u e o f one a t t h e o r i g i n . T h e f u n c t i o n s are also d i s p l a y e d i n t h e left p l o t o f
F i g u r e 11.3.
Table 11.6 Euclid's hat functions (defined for 0 < r < 2) for
different values of s.
s V>s(r) smoothness
1 1_ r C
1
2
2
2 ^ ^4arccos (^) r \ / 4 r ^ C
3 1
- sh (( + * ) ~ ) 4 1 6 r r3 c
4 I arccos ( r ) _ _1_^4 - r (20r + r ) 2 3
c
5 1
- 64^ ( t 1 2
+ 8 7 r
+ 3 2 7 r 2
) r 3
~ ( + 2 ? r
) r 3
) c
Fig. 11.3 Euclid's hat functions (left) of Table 11.6 and Buhmann's function of Example 11.6
(right).
A n o t h e r c o n s t r u c t i o n d e s c r i b e d i n [Schaback (1995a)] is t h e r a d i a l i z a t i o n o f t h e
s-fold tensor p r o d u c t o f u n i v a r i a t e 5 - s p l i n e s o f even o r d e r 2m w i t h u n i f o r m k n o t s .
T h e s e f u n c t i o n s d o n o t seem t o have a s i m p l e r e p r e s e n t a t i o n t h a t lends i t s e l f t o
numerical computations. A s c a n be seen f r o m i t s r a d i a l i z e d F o u r i e r t r a n s f o r m , t h e
5
r a d i a l i z e d S - s p l i n e i t s e l f is n o t s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n a n y R with
s > 1. For s = 1 o n l y t h e S - s p l i n e s o f even o r d e r are s t r i c t l y p o s i t i v e d e f i n i t e (see,
e.g., [ S c h o l k o p f a n d S m o l a ( 2 0 0 2 ) ] ) .
T h e last f a m i l y o f c o m p a c t l y s u p p o r t e d s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s
we w o u l d like t o m e n t i o n is due t o [ B u h m a n n ( 1 9 9 8 ) ] . B u h m a n n ' s f u n c t i o n s c o n t a i n
a l o g a r i t h m i c t e r m i n a d d i t i o n t o a p o l y n o m i a l . H i s f u n c t i o n s have t h e g e n e r a l f o r m
/OO
<p(r)= / (l-r /t)+* (l 2 a
Jo
H e r e 0 < 5 < ^, p > 1, a n d i n o r d e r t o o b t a i n f u n c t i o n s t h a t are s t r i c t l y p o s i t i v e
s
d e f i n i t e a n d r a d i a l o n M for s < 3 t h e c o n s t r a i n t s for t h e r e m a i n i n g p a r a m e t e r s are
A > 0, a n d - 1 < a < =.
Example 11.6. A n example w i t h a = < 5 = | , p = l and A = 2 is l i s t e d i n

[ B u h m a n n (2000)]:
4 4 3 2
(p(r) = 12r logr - 21r + 32r - 12r + 1, 0 < r < 1.
2 S
T h i s f u n c t i o n is i n C ( R ) a n d s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n IR for s < 3.
I t is d i s p l a y e d i n t h e r i g h t p l o t o f F i g u r e 11.3.
W h i l e i t is s t a t e d i n [ B u h m a n n (2000)] t h a t t h e c o n s t r u c t i o n t h e r e encompasses
b o t h W e n d l a n d ' s a n d W u ' s f u n c t i o n s , a n even m o r e g e n e r a l t h e o r e m t h a t shows t h a t
integration of a positive function / L i [ 0 , o o ) against a s t r i c t l y positive definite
k e r n e l K r e s u l t s i n a s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n c a n be f o u n d i n [ W e n d l a n d
(2005a)] (see also S e c t i o n 4 . 8 ) . M o r e specifically,
roc
p(r) = / K(t,r)f(t)dt
Jo
is s t r i c t l y p o s i t i v e d e f i n i t e . B u h m a n n ' s c o n s t r u c t i o n t h e n c o r r e s p o n d s t o c h o o s i n g
a s 2
f(t) = t ( l - t )^ a n d K(t, r) = ( 1 - r /t)\.
Chapter 12
Interpolation with Compactly Supported

R B F s in M A T L A B
We n o w have a n a l t e r n a t i v e w a y t o c o n s t r u c t a n R B F i n t e r p o l a n t t o scattered
s
d a t a i n M. . I f we use t h e c o m p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s o f t h e p r e v i o u s
chapter t h e n t h e m a i n difference t o o u r p r e v i o u s i n t e r p o l a n t s is t h a t n o w t h e i n -
t e r p o l a t i o n m a t r i x c a n be m a d e sparse b y s c a l i n g t h e s u p p o r t o f t h e basic f u n c t i o n
a p p r o p r i a t e l y . T o achieve t h i s w e use as w e d i d earlier t h e basic f u n c t i o n s
(p (r)
e = <p{er). T h u s , a large v a l u e o f e corresponds t o a s m a l l s u p p o r t . I n o t h e r
w o r d s , i f t h e s u p p o r t o f p = 1/e.

Since we k n o w t h a t t h e i n t e r p o l a t i o n m a t r i x w i l l be a sparse m a t r i x , we w a n t
t o w r i t e M A T L A B code t o efficiently assemble t h e m a t r i x . O n c e we have defined a
sparse m a t r i x , M A T L A B w i l l a u t o m a t i c a l l y use s t a t e - o f - t h e - a r t sparse m a t r i x t e c h -
niques t o solve t h e l i n e a r s y s t e m . O b v i o u s l y , we d o n o t w a n t t o c o m p u t e t h e m a t r i x
entries for a l l p a i r s o f p o i n t s since we k n o w a l l o f t h e entries for far away p o i n t s
w i l l be zero. T h e r e f o r e , a n efficient d a t a s t r u c t u r e is needed. W e use A;c?-trees
( i m p l e m e n t e d i n a set o f M A T L A B M E X - f i l e s w r i t t e n b y G u y Shechter t h a t c a n be
d o w n l o a d e d f r o m t h e M A T L A B C e n t r a l F i l e E x c h a n g e , see [ M C F E ] ) . Some i n f o r -
m a t i o n o n kd-tvees is p r o v i d e d i n A p p e n d i x A . D a t a s t r u c t u r e s for t h e use w i t h
meshfree a p p r o x i m a t i o n m e t h o d s are also discussed i n [ W e n d l a n d (2005a)].
12.1 Assembly o f the Sparse I n t e r p o l a t i o n M a t r i x
We have s t r u c t u r e d t h e s c a t t e r e d d a t a i n t e r p o l a t i o n p r o g r a m i n t h e c o m p a c t l y sup-
p o r t e d case analogous t o t h e code for t h e g l o b a l i n t e r p o l a n t s , i.e., first c o n s t r u c t a
distance m a t r i x , a n d t h e n a p p l y t h e a n o n y m o u s f u n c t i o n r b f t o o b t a i n t h e i n t e r p o -
l a t i o n / e v a l u a t i o n m a t r i x (as o n lines 1 3 - 1 4 a n d 1 5 - 1 6 o f P r o g r a m 2.1). H o w e v e r , i t
t u r n s o u t t h a t i t is easier t o d e a l w i t h t h e c o m p a c t s u p p o r t i f we c o m p u t e t h e "dis-
tance m a t r i x " c o r r e s p o n d i n g t o t h e ( 1 s r ) + t e r m since o t h e r w i s e those entries o f
t h e distance m a t r i x t h a t are zero (since t h e m u t u a l distance between t w o i d e n t i c a l
p o i n t s is zero) w o u l d be "lost" i n t h e sparse r e p r e s e n t a t i o n o f t h e m a t r i x .
T h e M A T L A B code DistanceMatrixCSRBF.m ( P r o g r a m 12.1) c o n t a i n s t w o s i m i -
95
lar blocks that will be used depending on whether we have more centers than data
sites or vice versa. For example, if there are more data sites than centers (cf. lines 7
16), then we build a kd-txee for the data sites and find for each center x - those 3
data sites within the support of the basis function centered at Xj, i.e., we construct
the (sparse) matrix column by column. In the other case (cf. lines 18-27) we start
with a tree for the centers and build the matrix row by row. This is accomplished by
determining for each data site Xi all centers whose associated^ basis function
covers data site Xi.
The functions kdtree and kdrangequery are provided by the kd-tree library
mentioned above. T h e call in line 7 (respectively 18) of Program 12.1 generates the
kd-txee of all the centers (data sites), and with the call to kdrangequery in line 9
(respectively 20) we find all centers (data sites) that lie within a distance support
of the jth center point (data site). T h e actual distances are returned in the vector
d i s t and the indices into the list of all data sites are provided in idx. T h e distances
for these points only are stored in the matrix DM. For maximum efficiency (in order
to avoid dynamic memory allocation) it is important to have a good estimate of
the number of nonzero entries in the matrix for the allocation statement in lines 4
and 5. The version of the code presented here has the best performance for larger
problems since s p a r s e is only invoked once.
P r o g r a m 12.1. DistanceMatrixCSRBF.m
% DM = D i s t a n c e M a t r i x C S R B F ( d s i t e s , c t r s , e p )
% Forms t h e d i s t a n c e m a t r i x of two s e t s of p o i n t s i n R~s
% f o r compactly supported r a d i a l b a s i s f u n c t i o n s , i . e . ,
7. DM(i,j) = I I d a t a s i t e _ i - c e n t e r _ j I I _2.
7o The CSRBF used with t h i s code must be g i v e n i n s h i f t e d form
7. rbf2(u) = r b f ( r ) , u=l-e*r.
% F o r example, t h e Wendland C2
7o rbf = @(e,r) max(l-e*r , 0 ) . ~ 4 . * ( 4 * e * r + l ) ;
7 becomes
7. r b f 2 = ffl(u) u. "4.*(4*u+5) ;
7o Input
% d s i t e s : Nxs matrix r e p r e s e n t i n g a s e t of N d a t a s i t e s
% i n R"s ( i . e . , each row c o n t a i n s one
7o s-dimensional p o i n t )
% ctrs: Mxs matrix r e p r e s e n t i n g a s e t of M c e n t e r s f o r
7o RBFs i n R~s ( a l s o one c e n t e r p e r row)
7 ep: determines s i z e of support of b a s i s f u n c t i o n .
7o Small ep y i e l d s wide f u n c t i o n ,
7o i . e . , s u p p o r t s i z e = 1/ep
7. Output
7, DM: NxM SPARSE m a t r i x t h a t c o n t a i n s t h e E u c l i d e a n
12. Interpolation with Compactly Supported RBFs in M A T L A B 97
/
0 u - d i s t a n c e ( u = l - e * r ) between t h e i - t h d a t a
% s i t e and t h e j - t h c e n t e r i n t h e i , j p o s i t i o n
% Uses: k-D t r e e package by Guy S h e c h t e r from
'/, MATLAB C e n t r a l F i l e Exchange
1 f u n c t i o n DM = D i s t a n c e M a t r i x C S R B F ( d s i t e s , c t r s , e p )
2 N = size(dsites,1); M = size(ctrs,1);
7 B u i l d k-D t r e e f o r d a t a s i t e s
% F o r each c e n t e r ( b a s i s f u n c t i o n ) , f i n d t h e d a t a s i t e s
% i n i t s support along w i t h u - d i s t a n c e
3 support = 1/ep;
4 nzmax = 25*N; rowidx = zeros(1,nzmax); c o l i d x = zeros(1,nzmax);
5 v a l i d x = zeros(1,nzmax); i s t a r t = 1; i e n d = 0;
6 i f M > N / f a s t e r i f more c e n t e r s than d a t a s i t e s
7 [tmp,tmp,Tree] = k d t r e e ( c t r s , [] ) ;
8 f o r i = 1:N
9 [pts,dist,idx] = kdrangequery(Tree,dsites(i,:),support);
10 newentries = l e n g t h ( i d x ) ;
11 iend = iend + n e w e n t r i e s ;
12 rowidx(istart:iend) = repmat(i,1,newentries);
13 c o l i d x ( i s t a r t : i e n d ) = idx';
14 validx(istart:iend) = l-ep*dist';
15 i s t a r t = i s t a r t + newentries;
16 end
17 e l s e
18 [tmp,tmp,Tree] = k d t r e e ( d s i t e s , [ ] ) ;
19 f o r j = 1:M
20 [pts.dist,idx] = kdrangequery(Tree,ctrs(j,:),support);
21 newentries = l e n g t h ( i d x ) ;
22 iend = iend + n e w e n t r i e s ;
23 rowidx(istart:iend) = idx';
24 c o l i d x ( i s t a r t : i e n d ) = repmat(j,1.newentries);
25 validx(istart:iend) = l-ep*dist';
26 i s t a r t = i s t a r t + newentries;
27 end
f
28 end
29 idx = f i n d ( r o w i d x ) ;
30 DM = s p a r s e ( r o w i d x ( i d x ) , c o l i d x ( i d x ) , v a l i d x ( i d x ) , N , M ) ;
7. Free t h e k-D Tree from memory.
31 kdtree ( [ ] , , Tree) ;
T h e reason for c o d i n g D i s t a n c e M a t r i x C S R B F . m i n t w o different w a y s is so t h a t
we w i l l be able t o speed u p t h e p r o g r a m w h e n d e a l i n g w i t h n o n - s q u a r e ( e v a l u a t i o n )
m a t r i c e s (for e x a m p l e i n t h e c o n t e x t o f M L S a p p r o x i m a t i o n (c.f. Chapter 24).
-3*, *
O n e c o u l d also i m p l e m e n t t h e d i s t a n c e m a t r i x r o u t i n e for sparse m a t r i c e s as

follows:
1 f u n c t i o n DM = D i s t a n c e M a t r i x C S R B F ( d s i t e s , c t r s , e p )
2 N = size(dsites,l); M = size(ctrs,1);
% B u i l d k-D t r e e f o r d a t a sites
% F o r each c e n t e r ( b a s i s f u n c t i o n ) , f i n d t h e d a t a sites
% i n i t s support along w i t h u - d i s t a n c e
3 support = 1/ep; nzmax = 25*N; DM = spalloc(N,M,nzmax);
4 [tmp, tmp,Tree] = k d t r e e ( d s i t e s , [ ] ) ;
5 f o r j = 1:M
6 Cpts,dist,idx] = kdrangequery(Tree,ctrs(j,:),support);
7 DM(idx.j) = l - e p * d i s t ;
8 end
% F r e e the k-D Tree from memory.
9 kdtree ( [ ] , [ ] , Tree) ;
T h i s code is c e r t a i n l y easier t o f o l l o w , b u t n o t as efficient as t h e one l i s t e d i n

P r o g r a m 12.1. N o t e t h a t we l i s t e d o n l y one v e r s i o n o f t h e code here. C l e a r l y , t h e
a l t e r n a t i v e v e r s i o n can be a d d e d a n a l o g o u s l y t o t h e p r e v i o u s p r o g r a m . j
T h e i n t e r p o l a t i o n p r o g r a m is v i r t u a l l y i d e n t i c a l t o P r o g r a m 2.1. The only
changes are t o replace lines 13 a n d 15 b y t h e c o r r e s p o n d i n g lines
13 DM_data = D i s t a n c e M a t r i x C S R B F ( d s i t e s , c t r s , e p ) ;
15 DM_eval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , c t r s , e p ) ;
2
a n d t o define t h e R B F i n s h i f t e d f o r m , i.e., i n s t e a d o f r e p r e s e n t i n g , e.g., t h e C
W e n d l a n d f u n c t i o n c / ^ i o n line 1 b y
1 r b f = @(e,r) m a x ( l - e * r , 0 ) . ~ 4 . * ( 4 * e * r + l ) ; ep=0.7;
we n o w w r i t e
1 r b f = @(e,r) r . " 4 . * ( 5 * s p o n e s ( r ) - 4 * r ) ; ep=0.7;
N o t e t h e use o f t h e sparse m a t r i x o f ones spones. H a d we used 5-4*r i n s t e a d , t h e n

a full m a t r i x w o u l d have been g e n e r a t e d ( w i t h m a n y a d d i t i o n a l a n d u n w a n t e d
ones).
I n o r d e r t o speed u p t h e s o l u t i o n o f t h e ( s y m m e t r i c p o s i t i v e d e f i n i t e ) sparse
linear s y s t e m we c o u l d use t h e p r e c o n d i t i o n e d c o n j u g a t e g r a d i e n t a l g o r i t h m (peg
in MATLAB) i n s t e a d o f t h e basic backslash \ (or m a t r i x left d i v i s i o n m l d i v i d e )
o p e r a t i o n , i.e., we c o u l d replace line 17 of P r o g r a m 2.1 b y
17 c = p c g ( I M , r h s ) ; Pf = EM * c;
N o t e , however, t h a t t h e \ o p e r a t o r also e m p l o y s s t a t e - o f - t h e - a r t direct sparse solvers

by first a p p l y i n g a m i n i m u m degree p r e o r d e r i n g .
12-2 Numerical Experiments with CSRBFs
-yVe now present t w o sets o f i n t e r p o l a t i o n e x p e r i m e n t s w i t h c o m p a c t l y supported

RBFs. I n T a b l e 12.2 we use t h e non-stationary approach to interpolation, i.e.,
the s u p p o r t size r e m a i n s fixed for i n c r e a s i n g l y denser sets X o f d a t a sites. I n t h i s
approach we w i l l be able t o observe convergence. However, the matrices become
increasingly denser, a n d therefore t h e n o n - s t a t i o n a r y a p p r o a c h is v e r y inefficient.
I n Table 1 2 . 1 , o n t h e o t h e r h a n d , we use t h e stationary a p p r o a c h , i.e., we scale t h e
support size o f t h e basis f u n c t i o n s p r o p o r t i o n a l l y t o t h e fill distance hx,o. (defined
in (2.3)). N o w t h e " b a n d w i d t h " o f t h e i n t e r p o l a t i o n m a t r i x A is c o n s t a n t . This
theoretically results i n O(N) c o m p u t a t i o n a l c o m p l e x i t y , i.e., a v e r y efficient i n t e r -
p o l a t i o n m e t h o d . T h e s t a t i o n a r y i n t e r p o l a t i o n m e t h o d is also n u m e r i c a l l y stable,
but there w i l l be essentially n o convergence (see T a b l e 12.1).
We use W e n d l a n d ' s c o m p a c t l y s u p p o r t e d f u n c t i o n </?3,i(r) = ( 1 r ) + ( 4 r + 1)
to i n t e r p o l a t e Franke's f u n c t i o n (2.2) o n g r i d s o f e q u a l l y spaced p o i n t s i n t h e u n i t
2
square [0, l ] . I n t h e s t a t i o n a r y case ( T a b l e 12.1) t h e s u p p o r t o f t h e basis f u n c t i o n
starts o u t w i t h a n i n i t i a l scale p a r a m e t e r e = 0.7 w h i c h is subsequently multiplied
by a factor o f t w o whenever t h e fill distance is h a l v e d , i.e., w h e n we r e p e a t t h e
experiment o n t h e n e x t finer g r i d . T h i s corresponds t o k e e p i n g a c o n s t a n t number
of r o u g h l y 25 d a t a sites w i t h i n t h e s u p p o r t o f a n y basis f u n c t i o n . T h e r e f o r e , the
" b a n d w i d t h " o f t h e i n t e r p o l a t i o n m a t r i x A is k e p t c o n s t a n t ( a t 2 5 ) , so t h a t A is v e r y
sparse for finer grids. W e c a n observe nice convergence for t h e first few i t e r a t i o n s ,
3
but once an R M S - e r r o r o f a p p r o x i m a t e l y 5 x 1 0 ~ is reached, t h e r e is n o t m u c h
further i m p r o v e m e n t . T h i s b e h a v i o r is n o t y e t f u l l y u n d e r s t o o d . H o w e v e r , i t is
similar t o w h a t happens i n t h e approximate approximation m e t h o d o f M a z ' y a (see,
e.g., [ M a z ' y a a n d S c h m i d t (2001)] a n d o u r discussion i n C h a p t e r 2 6 ) . The rate
r&te
listed i n t h e t a b l e is t h e e x p o n e n t o f t h e observed R M S - c o n v e r g e n c e r a t e 0(h ).
I t is c o m p u t e d u s i n g t h e f o r m u l a
e
r a t '"fa--/ *> fc = 2,3,..., (12.1)
ln(/ifc_i/ftfc)
where e k is t h e e r r o r for e x p e r i m e n t n u m b e r k, a n d h k is t h e fill distance o f t h e

kth c o m p u t a t i o n a l mesh. N o t e , t h a t for u n i f o r m l y spaced p o i n t s t h e r a t i o o f f i l l
distances o f t w o consecutive meshes w i l l always be t w o , w h i l e for r a n d o m p o i n t s
(such as H a l t o n p o i n t s ) we e s t i m a t e t h e fill distance v i a ( 2 . 4 ) . The % nonzero
c o l u m n indicates t h e s p a r s i t y o f t h e i n t e r p o l a t i o n m a t r i c e s , a n d t h e t i m e is m e a s u r e d
in seconds. E r r o r s are c o m p u t e d o n a n e v a l u a t i o n g r i d o f 40 x 40 e q u a l l y spaced
2
points i n [0, l ] .
I n t h e n o n - s t a t i o n a r y case ( T a b l e 12.2) we use basis f u n c t i o n s w i t h o u t a d j u s t i n g
their s u p p o r t size, i.e., e = 0.7 is k e p t fixed for a l l e x p e r i m e n t s . W e have convergence
a l t h o u g h i t is n o t o b v i o u s w h a t t h e r a t e m i g h t be. However, t h e m a t r i c e s b e c o m e
increasingly dense a n d c o m p u t a t i o n requires l o t s o f s y s t e m m e m o r y . T h e r e f o r e , we
left o u t t h e s o l u t i o n for t h e N = 16641 a n d N = 66049 cases i n T a b l e 12.2. T h e t i m e
Table 12.1 Stationary interpolation at N equally

2
spaced points in [0, l ] (constant 25 points in support)
with Wendland's function <p(r) = (1 r ) ( 4 r + 1). +
N RMS-error rate % nonzero time
9 1.562729e-001 100 0.23

25 2.690350e-002 2.5382 57.8 0.31
81 1.027881e-002 1.3881 23.2 0.33
289 6.589552e-003 0.6414 7.47 0.41
1089 3.891263e-003 0.7599 2.13 0.63
4225 3.726913e-003 0.0623 0.57 1.23
16641 2.638296e-003 0.4984 0.15 3.75
66049 2.467867e-003 0.0963 0.04 15.48
Table 12.2 Non-stationary interpolation

at N equally spaced points in [0,1] 2
(e = 0.7 fixed) with Wendland's! function

4
<p(r) = (1 - r ) (4r + 1)
N RMS-error rate time
9 1.562729e-001 0.03
25 2.807706e-002 2.4766 0.04
81 4.853006e-003 2.5324 0.12
289 2.006041e-004 4.5965 0.45
1089 1.288000e-005 3.9611 2.75
4225 1.382497e-006 3.2198 47.92
c o m p a r i s o n b e t w e e n t h e entries i n T a b l e 12.1 a n d T a b l e 12.2 is n o t a s t r a i g h t f o r w a r d

one since we used t h e (dense) code P r o g r a m 2.1 t o do t h e e x p e r i m e n t s for T a b l e 12.2
since t h e r e is no sparseness t o be e x p l o i t e d a n d t h e kd-tvees actually introduce
a d d i t i o n a l overhead.
T h e i n t e r p l a y b e t w e e n c o m p u t a t i o n a l efficiency a n d non-convergence i n t h e sta-
t i o n a r y case a n d convergence a n d c o m p u t a t i o n a l inefficiency i n t h e n o n - s t a t i o n a r y
case is a g a i n a trade-off principle similar t o the i n t e r p l a y between accuracy and
i l l - c o n d i t i o n i n g for g l o b a l l y s u p p o r t e d R B F s (c.f. C h a p t e r 2 ) . These t r a d e - o f f p r i n -
ciples were e x p l a i n e d t h e o r e t i c a l l y as w e l l as i l l u s t r a t e d w i t h n u m e r i c a l e x p e r i m e n t s
i n [Schaback ( 1 9 9 7 b ) ] , a n d we w i l l consider t h e m i n C h a p t e r 16.
For c o m p a r i s o n purposes we r e p e a t t h e e x p e r i m e n t s w i t h t h e o s c i l l a t o r y basic
function
2 3
>(r) = o (r)
2 = ( 1 - r)% (3 + 1 8 r + 3 r - 192r ) ,
4 s
w h i c h is also C s m o o t h a n d s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n M for s < 3
(see T a b l e 11.5). T h e results are l i s t e d i n T a b l e 12.3 for t h e s t a t i o n a r y case a n d i n
T a b l e 12.4 for t h e n o n - s t a t i o n a r y case. N o t e t h a t t h e f u n c t i o n is i m p l e m e n t e d as
rbf = @(e,r) -r."6.*(168*spones(r)-552*r+573*r.~2-192*r."3);

Table 12.3 Stationary interpolation at N equally spaced

2
points in [0, l ] (constant 25 points in support) with the os-
2 3
cillatory function <p(r) = ( l - r ) (3 + 18r + 3 r - 1 9 2 r ) .
+
N RMS-error rate % nonzero time
9 1.655969e-001 100 0.28

25 3.941226e-002 2.0710 57.8 0.34
81 2.978973e-002 0.4038 23.2 0.36
289 2.914215e-002 0.0317 7.47 0.42
1089 3.063424e-002 -0.0720 2.13 0.64
4225 3.094308e-002 -0.0145 0.57 1.31
16641 3.089882e-002 0.0021 0.15 4.13
66049 3.086639e-002 0.0015 0.04 16.81
Table 12.4 Non-stationary interpolation

2
at N equally spaced points in [0,1]
(e = 0.7 fixed) with the oscillatory function
<p(r) = 2
( 3 + 1 8 r + 3 r - 192r ). 3
N RMS-error rate time
9 1.655969e-001 0.03
25 3.097850e-002 2.4183 0.06
81 4.612941e-003 2.7475 0.20
289 1.305297e-004 5.1432 0.72
1089 4.780575e-006 4.7711 4.06
4225 2.687479e-007 4.1529 55.09
i n t h e sparse s e t t i n g a n d as
rbf = @(e,r) m a x ( l - e * r , 0 ) . " 6 . * ( 3 + 1 8 * e * r + 3 * ( e * r ) . " 2 - 1 9 2 * ( e * r ) . " 3 ) ;
for t h e dense code.

W h i l e t h e p e r f o r m a n c e o f t h e o s c i l l a t o r y f u n c t i o n s for t h e s t a t i o n a r y e x p e r i -
m e n t is even m o r e d i s a p p o i n t i n g t h a n t h a t o f W e n d l a n d ' s f u n c t i o n s , t h e s i t u a t i o n
is reversed i n t h e n o n - s t a t i o n a r y case. I n fact, t h e e r r o r s o b t a i n e d w i t h t h e o s c i l l a -
t o r y basis f u n c t i o n s are a l m o s t as g o o d as those achieved w i t h " o p t i m a l l y " scaled
Gaussians (c.f. T a b l e 2.2).
I n o r d e r t o overcome t h e p r o b l e m s d u e t o t h e t r a d e - o f f p r i n c i p l e t h a t are ap-
parent i n b o t h the stationary and non-stationary approach t o i n t e r p o l a t i o n w i t h
c o m p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s we w i l l l a t e r consider u s i n g a m u l t i l e v e l sta-
t i o n a r y scheme (see C h a p t e r 3 2 ) .
Chapter 13
Reproducing Kernel Hilbert Spaces and

Native Spaces for Strictly Positive
Definite Functions
I n t h e n e x t few c h a p t e r s w e w i l l present some o f t h e t h e o r e t i c a l w o r k o n e r r o r

b o u n d s for a p p r o x i m a t i o n a n d i n t e r p o l a t i o n w i t h r a d i a l basis f u n c t i o n s . Since t h e
discussion for s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s w i l l a l r e a d y be t e c h n i c a l e n o u g h ,
we focus o n t h i s case, a n d o n l y m e n t i o n a few r e s u l t s for t h e c o n d i t i o n a l l y p o s i t i v e
definite case. T h e f o l l o w i n g discussion f o l l o w s m o s t l y t h e p r e s e n t a t i o n i n [ W e n d l a n d
(2005a)] w h e r e t h e i n t e r e s t e d reader c a n f i n d m a n y m o r e d e t a i l s .
13.1 Reproducing Kernel Hilbert Spaces
O u r first set o f e r r o r b o u n d s w i l l c o m e r a t h e r n a t u r a l l y once w e associate w i t h each

( s t r i c t l y p o s i t i v e d e f i n i t e ) r a d i a l basic f u n c t i o n a c e r t a i n space o f f u n c t i o n s c a l l e d i t s
native space. W e w i l l t h e n be able t o e s t a b l i s h a c o n n e c t i o n t o r e p r o d u c i n g k e r n e l
H i l b e r t spaces, w h i c h i n t u r n w i l l g i v e us t h e desired e r r o r b o u n d s as w e l l as c e r t a i n
o p t i m a l i t y results for r a d i a l basis f u n c t i o n i n t e r p o l a t i o n (see C h a p t e r 18).
R e p r o d u c i n g kernels are a classical c o n c e p t i n analysis i n t r o d u c e d b y N a c h m a n
A r o n s z a j n (see [ A r o n s z a j n ( 1 9 5 0 ) ] ) . W e b e g i n w i t h
S
D e f i n i t i o n 1 3 . 1 . L e t H be a r e a l H i l b e r t space o f f u n c t i o n s / : Q(Q R ) > R w i t h
i n n e r p r o d u c t (-, - ) ^ - A f u n c t i o n K : Q x ft R is called reproducing kernel for Ji
if
(1) K(-,x) e H for a l l x Q,

(2) f(x) = (f,K(-,x)) n for a l l / e W a n d all x fl.
T h e n a m e reproducing kernel is i n s p i r e d b y t h e r e p r o d u c i n g p r o p e r t y (2) i n

D e f i n i t i o n 1 3 . 1 . I t is k n o w n t h a t t h e r e p r o d u c i n g k e r n e l o f a H i l b e r t space is u n i q u e ,
a n d t h a t existence o f a r e p r o d u c i n g k e r n e l is e q u i v a l e n t t o t h e fact t h a t t h e p o i n t
evaluation functionals 5 X are b o u n d e d l i n e a r f u n c t i o n a l s o n Cl, i.e., t h e r e exists a
positive constant M = M x such t h a t
\8 f\
x = \f(x)\<--M\\f\\ n
103
for a l l / G 7i a n d a l l x G Cl. T h i s l a t t e r fact is d u e t o t h e Riesz representation

theorem.
O t h e r p r o p e r t i e s o f r e p r o d u c i n g kernels are g i v e n b y
Theorem 1 3 . 1 . Suppose H is a Hilbert space of functions f : Cl * K. with repro-

ducing kernel K. Then we have
(1) K(x, y) = (K(-,y), K(; x)) n forx,yeCl.

(2) K(x, y) = K(y, x) for x,yeCl.
(3) Convergence in Hilbert space norm implies pointwise convergence, i.e., if we
have ||/ fnWn 0 for n > oo then
>
\f(x) f {x)\
n 0 for all x G Cl.
Proof. B y P r o p e r t y (1) o f D e f i n i t i o n 13.1 K(-,y) G 7i f o r e v e r y y G Cl. T h e n t h e

r e p r o d u c i n g p r o p e r t y ( 2 ) o f t h e d e f i n i t i o n gives us
K(x,y) = (K(;y),K(;x)) n
for a l l x, y G Cl. T h i s establishes ( 1 ) . P r o p e r t y (2) follows f r o m ( 1 ) b y t h e s y m m e t r y

o f t h e H i l b e r t space inner p r o d u c t . F o r ( 3 ) w e use t h e r e p r o d u c i n g p r o p e r t y o f K
along w i t h the Cauchy-Schwarz inequality:
\f(x) - f {x)\
n = \(f - f ,K(-,x)) \
n n < ||/ - f \\ \\K(;x)\\ .
n n n D
N o w i t is i n t e r e s t i n g for us t h a t t h e r e p r o d u c i n g k e r n e l K is k n o w n t o b e p o s i t i v e
definite. Here w e use a s l i g h t g e n e r a l i z a t i o n o f t h e n o t i o n o f a p o s i t i v e d e f i n i t e func-
t i o n t o a p o s i t i v e d e f i n i t e k e r n e l . Essentially, w e replace &(xj X k ) i n D e f i n i t i o n 3.2
b y K(xj, X k ) . A t t h i s p o i n t w e r e m i n d t h e reader t h a t t h e space o f b o u n d e d l i n e a r
f u n c t i o n a l s o n 7i is k n o w n as i t s dual, a n d d e n o t e d b y 7i*.
Theorem 1 3 . 2 . Suppose Ji is a reproducing kernel Hilbert function space with

reproducing kernel K : Cl x Cl E . Then K is positive definite. Moreover, K is
strictly positive definite if and only if the point evaluation functionals S x are linearly
independent in 7i*.
Proof. Since t h e k e r n e l is r e a l - v a l u e d w e c a n r e s t r i c t ourselves t o a quadratic

N
f o r m w i t h real coefficients. F o r d i s t i n c t p o i n t s X \ , . . . , x^ a n d n o n z e r o c G M. w e
have
N N N N
^^CjCkKixj^Xk) = ^Y2cjC (K{-,Xj),K{-,Xk))n

k
j=l k=l j=l k=l

N n
c K x
= ( j (-> j)>^CkK(;X ))<H k
j=l k=l
N
2
= ||^c -K(-,x )|| ,>0.
J J
j=i
T h u s K is p o s i t i v e d e f i n i t e . T o e s t a b l i s h t h e second c l a i m w e assume K is n o t
s t r i c t l y p o s i t i v e d e f i n i t e a n d show t h a t t h e p o i n t e v a l u a t i o n f u n c t i o n a l s m u s t b e
13. Reproducing Kernel Hilbert Spaces for Strictly Positive Definite Functions 105
linearly dependent. I f K is n o t s t r i c t l y p o s i t i v e d e f i n i t e t h e n t h e r e e x i s t d i s t i n c t
p o i n t s Xi,..., XN a n d n o n z e r o coefficients c - such t h a t 3
N N
c c
Y2Y2 j kK{xj,x ) k = 0.
3=1 k=l
T h e first p a r t o f t h e p r o o f t h e r e f o r e i m p l i e s
3= 1
N o w we t a k e t h e H i l b e r t space i n n e r p r o d u c t w i t h a n a r b i t r a r y f u n c t i o n / 6 H a n d
use t h e r e p r o d u c i n g p r o p e r t y o f K t o o b t a i n
3= 1
N
c K x
= ^2 3(fi (-, 3))n
3= 1
N
3= 1
N
3=1
T h i s , however, i m p l i e s t h e l i n e a r dependence o f t h e p o i n t e v a l u a t i o n f u n c t i o n a l s
S Xj ( / ) f( j)i
x
j 1, ) AT, since t h e coefficients Cj were assumed t o be n o t a l l
zero. A n analogous a r g u m e n t c a n be used t o e s t a b l i s h t h e converse.
T h i s t h e o r e m p r o v i d e s one d i r e c t i o n o f t h e c o n n e c t i o n b e t w e e n s t r i c t l y p o s i t i v e
definite f u n c t i o n s a n d r e p r o d u c i n g kernels. H o w e v e r , we are also i n t e r e s t e d i n t h e
o t h e r d i r e c t i o n . Since t h e R B F s w e have b u i l t o u r i n t e r p o l a t i o n m e t h o d s f r o m are
strictly positive definite functions, we w a n t t o k n o w how t o construct a r e p r o d u c i n g
k e r n e l H i l b e r t space associated w i t h those s t r i c t l y p o s i t i v e d e f i n i t e basic f u n c t i o n s .
13.2 N a t i v e S p a c e s for S t r i c t l y P o s i t i v e D e f i n i t e Functions
I n t h i s section we w i l l show t h a t e v e r y s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l basic f u n c t i o n

can indeed be associated w i t h a r e p r o d u c i n g k e r n e l H i l b e r t space i t s native space.
F i r s t , we n o t e t h a t D e f i n i t i o n 13.1 t e l l s us t h a t Ti. c o n t a i n s a l l f u n c t i o n s o f t h e
form
N
K X
f = ^2c3 (-i 3)
3=1
p r o v i d e d Xj Cl. A s a consequence o f T h e o r e m 13.1 we have t h a t

N N
11/11* = (f,f)n = ( Y , ^ K { . , X j ) ^ c K { . , x ) )
k k u
3= 1 k=l
N N
S c
= Z2^C3 k{K(-,x ),K(-,x ))n j k
3= 1 k=l
N N
C C K X X
= ^2Yl i k ( 3' k)-
3=1 k=l
Therefore, we define t h e (possibly i n f i n i t e - d i m e n s i o n a l ) space
H (Cl)
K = s p a n { K ( - , y) : y Cl} (13.1)
w i t h a n associated b i l i n e a r f o r m (, )#- g i v e n b y
=
N K N K N K N K
C^Z 3 i^ j)^Yl ^' ^

C K x
dkK yk K
Y2^2 j k ( 3^Vk), C d K X
3= 1 k=l j=l k=l

w h e r e NK = oo is also a l l o w e d .
T h e o r e m 1 3 . 3 . If K : Cl x Cl > E is a symmetric strictly positive definite kernel,

then the bilinear form {-,-)K defines an inner product on HK(CI). Furthermore,
HK(CI) is a pre-Hilbert space with reproducing kernel K .
Proof. (-, -)K is o b v i o u s l y b i l i n e a r a n d s y m m e t r i c . W e j u s t need t o show t h a t

(/> f ) K > 0 for nonzero / G HK(CI). A n y such / c a n be w r i t t e n i n t h e f o r m
N K
f = ^2cjK(;Xj), XjeCl.
3= 1
Then
N K N K
(f, f)K = Y2Y2c c K(x ,x ) j k j k >0

3= 1 k=l
since K is s t r i c t l y p o s i t i v e d e f i n i t e . T h e r e p r o d u c i n g p r o p e r t y follows f r o m
N K
{f, K(; X)) K = CjK(x, Xj) = f(x).

3= 1
Since we j u s t showed t h a t HK{CI) is a p r e - H i l b e r t space, i.e., need n o t be c o m -

plete, we n o w define t h e native space JVK(CI) o f K t o be t h e c o m p l e t i o n o f HK(CI)
G
w i t h respect t o t h e i f - n o r m || \ \ K so t h a t \\f\\K = H/HA/W^) ^ O R A 1 1
f H {Cl).
K
T h e t e c h n i c a l details concerned w i t h t h i s c o n s t r u c t i o n are discussed i n [ W e n d l a n d

(2005a)].
I n t h e special case w h e n we are d e a l i n g w i t h s t r i c t l y p o s i t i v e d e f i n i t e ( t r a n s l a t i o n
i n v a r i a n t ) f u n c t i o n s &(x y) = K(x, y) a n d w h e n Cl = R s
we get a c h a r a c t e r i z a t i o n
of n a t i v e spaces i n t e r m s o f F o u r i e r t r a n s f o r m s .
S s
T h e o r e m 1 3 . 4 . Suppose $ G C ( R ) n L i ( R ) is a real-valued strictly positive def-
inite function. Define
S S 3
g = {fe L (R ) n C(R ):
2 -= e -MR )}
7
ana egwip /ns space with the bilinear form
i ,/ 9 , i / / M I R do;.
Then Q is a real Hilbert space with inner product (, -)g and reproducing kernel
s S
<&( ) . Hence, Q is the native space of & on R , i.e., Q = A / $ ( R ) and both
S
inner products coincide. In particular, every f G A / $ ( R ) can be recovered from its
s S
Fourier transform f G Za(R ) fl L ( R ) . 2
A n o t h e r c h a r a c t e r i z a t i o n o f t h e n a t i v e space is g i v e n i n t e r m s o f t h e eigenfunc-
t i o n s o f a linear o p e r a t o r associated w i t h t h e r e p r o d u c i n g k e r n e l . This operator,
T $ : L 2 ( P ) > L (fl),2 is g i v e n b y
T*(v)(x)= &(x,y)v(y)dy, v G L (fl),2 ^ G fi.

in
For t h e eigenvalues Afc, k = 1, 2 , . . ., a n d eigenfunctions (p o f t h i s o p e r a t o r M e r c e r ' s k
t h e o r e m [Riesz a n d Sz.-Nagy (1955)] states
T h e o r e m 1 3 . 5 ( M e r c e r ) . Let <&(, ) be a continuous positive definite kernel that

satisfies
/ $(x,y)v(x)v(y)dxdy > 0, for all v G L (fl), 2 x,y E fl. (13.2)
Then <fr can be represented by

oo
$(x,y) = ^X (t> {x)4> {y),
k k k (13.3)
fc=i
where X k are the (non-negative) eigenvalues and 4> are the (L2-orthonormal) k eigen-
functions of T<&. Moreover, this representation is absolutely and uniformly conver-
gent.
W e can i n t e r p r e t c o n d i t i o n (13.2) as a t y p e o f integral positive definiteness. As

usual, t h e eigenvalues a n d eigenfunctions satisfy T<s>(f> = \ <p k k k or
/ $(x,y)(p (y)dy
k = \ <f) (x),
k k A; = 1 , 2 , . . . .
Jn
I n general, M e r c e r ' s t h e o r e m allows us t o c o n s t r u c t a r e p r o d u c i n g k e r n e l H i l b e r t
space 7i b y representing t h e f u n c t i o n s i n 7i as i n f i n i t e linear c o m b i n a t i o n s o f t h e
eigenfunctions, i.e.,
{
oo
/: / = ^ C f c 0 *
I t is clear t h a t t h e k e r n e l <3> i t s e l f is i n 7i since i t has t h e e i g e n f u n c t i o n expansion

( 1 3 . 3 ) . T h e i n n e r p r o d u c t for 7i is g i v e n b y
oo oo oo ^
(f,g)n = C ^ C j ^ j ^ Y ^ d k ^ n = ^ '
3=1
k
k=l k=l
w h e r e we used t h e 7 i - o r t h o g o n a l i t y
(<t>j,4>k)H =
y/Xj^/Xk
of t h e eigenfunctions.
W e n o t e t h a t $ is indeed t h e r e p r o d u c i n g k e r n e l o f Ti since t h e eigenfunction
e x p a n s i o n (13.3) o f <E> a n d t h e o r t h o g o n a l i t y o f t h e eigenfunctions imply
oo oo
(/,$(, x)) n = (Y2cj<pj,^2Xk(f)k<Pk(x))n
j=i k=i
_ CkXk4>k{x)
oo
- ^c <f> ix)
k k = fix).
k=i
F i n a l l y , one has (c./. [ W e n d l a n d (2005a)]) t h a t t h e n a t i v e space A/$(f2) is g i v e n

by
A / - * ( f i ) = j / L (n) 2 : J-|(/,0 f c ) i 2 ( o ) |2 < o o |
a n d t h e n a t i v e space i n n e r p r o d u c t c a n be w r i t t e n as
oo ^
(f,g)rt* = J^T-(/i0fc)L (n)(^,0fc>L (n), a 2 f,g e jV&iQ,).
A k
k=i
Since A/$(f2) is a subspace o f L (fl) 2 t h i s c o r r e s p o n d s t o t h e i d e n t i f i c a t i o n Ck =

(/) 4>k)L (Q) 3 f ^he generalized F o u r i e r coefficients i n t h e discussion above.
13.3 E x a m p l e s o f N a t i v e S p a c e s for P o p u l a r R a d i a l B a s i c
Functions
T h e o r e m 13.4 shows t h a t n a t i v e spaces o f t r a n s l a t i o n i n v a r i a n t f u n c t i o n s c a n be

v i e w e d as a g e n e r a l i z a t i o n o f s t a n d a r d Sobolev spaces. I n d e e d , for m > s/2 the
Sobolev space W c a n be defined as (see, e.g., [ A d a m s (1975)])
S S S 2 m/2 S
W^{R ) = { / G L (R ) 2 n C(R ) : / ( - ) ( 1 + || \\ ) 2 G L (R )}.
2 (13.4)
O n e also f r e q u e n t l y sees t h e d e f i n i t i o n
m a s
W 2 ( f l ) = { / G L ( f i ) n C(Q)
2 : Df G L {Q)
2 for a l l \a\ < m, a GN }, (13.5)
S
w h i c h applies whenever Q, C R is a b o u n d e d d o m a i n . This interpretation will
make clear t h e c o n n e c t i o n b e t w e e n t h e n a t i v e s spaces o f Sobolev splines a n d t h o s e
3
of p o l y h a r m o n i c splines t o be discussed b e l o w . The n o r m of W ^ R ) is u s u a l l y
given by
I|/IIW7(R-) = { H j D a
/lli (R ) 2
S
||<m
A c c o r d i n g t o ( 1 3 . 4 ) , any s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n 3> whose F o u r i e r t r a n s -
f o r m decays o n l y a l g e b r a i c a l l y has a Sobolev space as i t s n a t i v e space. I n p a r t i c u l a r ,
the M a t e r n functions
K -t(\\x\\)\\x\\f>-*
0 8
= ' ^ 2'
of Section 4.4 w i t h F o u r i e r t r a n s f o r m
* () = (i + H | ) -
/9
2 / 9
8
can i m m e d i a t e l y be seen t o have n a t i v e space JV* (R ) P = Wg(R) with B > s/2
( w h i c h is w h y some people refer t o t h e M a t e r n f u n c t i o n s as Sobolev splines).
W e n d l a n d ' s c o m p a c t l y s u p p o r t e d f u n c t i o n s <& fc = <>s,fc(|| l b ) o f C h a p t e r 11 c a n
S]
s 2 + k + 1 2 s
be s h o w n t o have n a t i v e spaces A / $ a k (R ) = W ^ 2 ^ (R ) (where the r e s t r i c t i o n
s > 3 is r e q u i r e d for t h e special case k = 0 ) .
N a t i v e spaces for s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s c a n also be
c o n s t r u c t e d . H o w e v e r , since t h i s is m o r e t e c h n i c a l , we l i m i t e d t h e discussion a b o v e
t o s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s , a n d refer t h e i n t e r e s t e d reader t o t h e b o o k
[ W e n d l a n d (2005a)] o r t h e p a p e r s [Schaback (1999a); Schaback (2000a)]. W i t h t h e
extension of the theory t o s t r i c t l y c o n d i t i o n a l l y positive definite functions the native
spaces o f t h e r a d i a l powers a n d t h i n p l a t e (or surface) splines o f Sections 8.2 a n d
8.3 c a n be s h o w n t o be t h e so-called Beppo-Levi spaces o f o r d e r k
S S a S s
BL F C ( R ) = { / C{R ) : Df G L (R )
2 for a l l | a | = k, a e N } ,
a
where D denotes a generalized derivative o f o r d e r at (defined i n t h e same s p i r i t as
t h e generalized F o u r i e r t r a n s f o r m , see A p p e n d i x B ) . I n fact, t h e i n t e r s e c t i o n o f a l l
S 3
B e p p o - L e v i spaces B L f c ( R ) o f o r d e r k < m y i e l d s t h e Sobolev space W ^ R ) . In
S
t h e l i t e r a t u r e t h e B e p p o - L e v i spaces B L f c ( R ) are sometimes referred t o as homo-
s
geneous Sobolev spaces of order k. A l t e r n a t i v e l y , t h e B e p p o - L e v i spaces o n R are
defined as
S S S
BL F C ( R ) = { / G C{R ) : /(-)|| | | ? G L (R )},
2
a n d t h e f o r m u l a s g i v e n i n C h a p t e r 8 for t h e F o u r i e r t r a n s f o r m s o f r a d i a l powers a n d
t h i n p l a t e splines show i m m e d i a t e l y t h a t t h e i r n a t i v e spaces are B e p p o - L e v i spaces.
T h e s e m i - n o r m o n BLfc is g i v e n b y
mBL* = { t ^ ^ i i ^ / i i i 2 ( R . > } , (13.6)

a n d i t s k e r n e l is t h e p o l y n o m i a l space n | _ . x F o r m o r e details see [Wendland

(2005a)]. B e p p o - L e v i spaces were a l r e a d y s t u d i e d i n t h e e a r l y papers [ D u c h o n
(1976); D u c h o n (1977); D u c h o n (1978); D u c h o n ( 1 9 8 0 ) ] .
T h e n a t i v e spaces for Gaussians a n d (inverse) m u l t i q u a d r i c s are r a t h e r s m a l l .
For e x a m p l e , a c c o r d i n g t o T h e o r e m 13.4, for Gaussians t h e F o u r i e r t r a n s f o r m o f
/ J\f(fl) m u s t decay faster t h a n t h e F o u r i e r t r a n s f o r m o f t h e G a u s s i a n ( w h i c h is
itself a G a u s s i a n ) . I t is k n o w n t h a t , even t h o u g h t h e n a t i v e space o f Gaussians is
s m a l l , i t does c o n t a i n t h e i m p o r t a n t class o f so-called band-limited functions, i.e.,
functions whose F o u r i e r t r a n s f o r m is c o m p a c t l y s u p p o r t e d . These f u n c t i o n s p l a y
an i m p o r t a n t role i n sampling theory w h e r e S h a n n o n ' s famous s a m p l i n g t h e o r e m
[Shannon (1949)] states t h a t a n y b a n d - l i m i t e d f u n c t i o n c a n be c o m p l e t e l y recovered
f r o m i t s discrete samples p r o v i d e d t h e f u n c t i o n is s a m p l e d at a s a m p l i n g r a t e at
least t w i c e i t s b a n d w i d t h . T h e c o n t e n t o f t h i s t h e o r e m was a l r e a d y k n o w n m u c h
earlier (see [ W h i t t a k e r ( 1 9 1 5 ) ] ) .
T h e o r e m 1 3 . 6 ( S h a n n o n S a m p l i n g ) . Suppose f C ( J R ) n L i ( I R ) such that

5 s
its
s
Fourier transform vanishes outside the cube Q = [ | , | ] . Then f can be uniquely
reconstructed from its values on Z s
; i.e.,
s
Here t h e sine f u n c t i o n is defined for a n y x = (xi,... ,x )
s M as sine a; =
m o r e d e t a i l s o n S h a n n o n ' s s a m p l i n g t h e o r e m see, e.g., Chap-
ter 29 i n t h e b o o k [Cheney a n d L i g h t (1999)] or t h e p a p e r [Unser ( 2 0 0 0 ) ] .
Chapter 14
The Power Function and Native Space

Error Estimates
14.1 Fill Distance and Approximation Orders
Our goal i n t h i s s e c t i o n is t o p r o v i d e e r r o r e s t i m a t e s for s c a t t e r e d d a t a i n t e r p o l a t i o n

w i t h strictly (conditionally) positive definite functions. A s i n the previous chapter
we w i l l p r o v i d e m o s t o f t h e d e t a i l s for t h e s t r i c t l y p o s i t i v e d e f i n i t e case, a n d o n l y
m e n t i o n t h e e x t e n s i o n t o t h e c o n d i t i o n a l l y p o s i t i v e d e f i n i t e case i n t h e e n d . I n t h e i r
final f o r m we w i l l w a n t o u r e s t i m a t e s t o d e p e n d o n some k i n d o f measure o f t h e
d a t a d i s t r i b u t i o n . T h e measure t h a t is u s u a l l y used i n a p p r o x i m a t i o n t h e o r y is t h e
so-called fill distance
h = h ,n x = sup m i n \\x Xj\\ 2
already i n t r o d u c e d i n (2.3) i n C h a p t e r 2. T h e fill distance i n d i c a t e s h o w w e l l t h e

d a t a fill o u t t h e d o m a i n Q, a n d i t t h e r e f o r e denotes t h e r a d i u s o f t h e largest e m p t y
b a l l t h a t can be p l a c e d a m o n g t h e d a t a l o c a t i o n s . W e w i l l be i n t e r e s t e d i n w h e t h e r
the e r r o r
tends t o zero as h > 0, a n d i f so, h o w fast. H e r e {V^}h presents a sequence

of i n t e r p o l a t i o n (or, m o r e generally, p r o j e c t i o n ) o p e r a t o r s t h a t v a r y w i t h t h e fill
n s
distance h. For e x a m p l e , c o u l d d e n o t e i n t e r p o l a t i o n t o d a t a g i v e n at ( 2 + l ) ,
S n
n = 1 , 2 , . . . , e q u a l l y spaced p o i n t s i n t h e u n i t cube i n JR ( w i t h h = 2~ ) as w e
used i n some o f o u r earlier e x a m p l e s . O f course, t h e d e f i n i t i o n o f t h e fill d i s t a n c e
also covers s c a t t e r e d d a t a such as sets o f H a l t o n p o i n t s . I n fact, since H a l t o n p o i n t s
n
are q u a s i - u n i f o r m l y d i s t r i b u t e d (see A p p e n d i x A ) we c a n assume h ^ 2 ~ for a set
n s S
of ( 2 + l) H a l t o n p o i n t s i n JR . T h i s e x p l a i n s t h e specific sizes o f t h e p o i n t sets
we used i n earlier examples.
Since we w a n t t o e m p l o y t h e m a c h i n e r y o f r e p r o d u c i n g k e r n e l H i l b e r t spaces
presented i n t h e p r e v i o u s c h a p t e r w e w i l l c o n c e n t r a t e o n e r r o r e s t i m a t e s for f u n c t i o n s
/ G A / $ . I n t h e n e x t c h a p t e r w e w i l l also m e n t i o n some m o r e general estimates.
T h e t e r m t h a t is o f t e n used t o m e a s u r e t h e speed o f convergence t o zero is approx-
imation order. W e say t h a t t h e a p p r o x i m a t i o n o p e r a t o r has L -approximation
p
ill
order k i f
h) k
\\f-V$ \\ p = 0(h ) for/i-O.
h) k w
M o r e o v e r , i f we c a n also show t h a t | | / - V \\ f p / o(h ), then V has exact
L -approximation
p order k. W e w i l l c o n c e n t r a t e m o s t l y o n t h e case p = oo {i.e.,
p o i n t w i s e estimates), b u t a p p r o x i m a t i o n o r d e r i n o t h e r n o r m s c a n also be s t u d i e d .
I n order t o keep t h e f o l l o w i n g discussion as t r a n s p a r e n t as possible we w i l l r e s t r i c t
ourselves t o s t r i c t l y p o s i t i v e definite f u n c t i o n s . W i t h ( c o n s i d e r a b l y ) m o r e t e c h n i c a l
d e t a i l s t h e f o l l o w i n g can also be f o r m u l a t e d for s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
f u n c t i o n s (see [ W e n d l a n d (2005a)] for d e t a i l s ) .
14.2 L a g r a n g e F o r m of t h e I n t e r p o l a n t a n d Cardinal
Basis Functions
T h e key idea for t h e f o l l o w i n g discussion is t o express t h e i n t e r p o l a n t i n Lagrange

form, i.e., u s i n g so-called cardinal basis functions. F o r r a d i a l basis f u n c t i o n a p p r o x -
i m a t i o n t h i s idea is due t o [ W u a n d Schaback ( 1 9 9 3 ) ] . I n t h e p r e v i o u s c h a p t e r s we
established t h a t , for a n y s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n <E>, t h e l i n e a r s y s t e m
Ac = y
T
with Aij = (xi - Xj), i,j = 1,...,N, c = [ci,..., c],
N and y =
T
[f(xi),...,f(xN)] has a u n i q u e s o l u t i o n . I n t h e f o l l o w i n g we w i l l consider the
m o r e general s i t u a t i o n w h e r e $ is a s t r i c t l y p o s i t i v e d e f i n i t e k e r n e l , i.e., t h e entries
of A are g i v e n b y A^ = &(xi,Xj). T h e uniqueness r e s u l t holds i n t h i s case also.
I n order t o o b t a i n t h e c a r d i n a l basis f u n c t i o n s u* , j = 1,... 3 ,N, w i t h the prop-
e r t y u*(xi) = Sij, i.e.,
Uj{Xi) =
we consider t h e linear s y s t e m
Au*(x) = b(x), (14.1)

T
where t h e m a t r i x A is as above ( a n d therefore i n v e r t i b l e ) , u* = [u\,..., u* } ,
N and
6 = [ ^ ( ; X 1 ) , . . . , ^ ( ; X N ) ] T
. Thus,
S
T h e o r e m 1 4 . 1 . Suppose <fr is a strictly positive definite kernel on I R . Then, for any
distinct points x\,..., XN, there exist functions u* e span{<E>(-, Xj),j = 1,..., N}
such that Uj(xi) = 5ij.
T h e r e f o r e , we can w r i t e t h e i n t e r p o l a n t Vf t o / a t x\,..., XN m t h e c a r d i n a l
form
14- The Power Function and Native Space Error Estimates 113
I t is o f interest t o n o t e t h a t t h e c a r d i n a l f u n c t i o n s d o n o t d e p e n d o n t h e d a t a
values o f t h e i n t e r p o l a t i o n p r o b l e m . O n c e t h e d a t a sites are fixed a n d t h e basic
f u n c t i o n is chosen w i t h a n a p p r o p r i a t e s h a p e p a r a m e t e r (whose o p t i m a l v a l u e w i l l
d e p e n d o n t h e d a t a ) , t h e n t h e c a r d i n a l f u n c t i o n s are d e t e r m i n e d b y t h e l i n e a r s y s t e m
(14.1). W e have p l o t t e d v a r i o u s c a r d i n a l f u n c t i o n s based o n t h e G a u s s i a n basic
f u n c t i o n w i t h shape p a r a m e t e r e = 5 i n F i g u r e s 1 4 . 1 - 1 4 . 3 . T h e dependence o n t h e
d a t a l o c a t i o n s is c l e a r l y a p p a r e n t w h e n c o m p a r i n g t h e different d a t a d i s t r i b u t i o n s
( u n i f o r m l y spaced i n F i g u r e 1 4 . 1 , t e n s o r - p r o d u c t C h e b y s h e v i n F i g u r e 14.2, and
H a l t o n p o i n t s i n F i g u r e 14.3). T h e d a t a sets c a n be seen i n F i g u r e 14.5 b e l o w .
Fig. 14.1 Cardinal functions for Gaussian interpolation (with e = 5) on 81 uniformly spaced
2
points in [0, l ] . Centered at an edge point (left) and at an interior point (right).
Fig. 14.2 Cardinal functions for Gaussian interpolation (with e = 5) on 81 tensor-product Cheby-
2
shev points in [0, l ] . Centered at an edge point (left) and at an interior point (right).
Basic f u n c t i o n s t h a t g r o w w i t h i n c r e a s i n g d i s t a n c e f r o m t h e center p o i n t (such as

m u l t i q u a d r i c s ) are s o m e t i m e s c r i t i c i z e d for b e i n g " c o u n t e r - i n t u i t i v e " for s c a t t e r e d
d a t a a p p r o x i m a t i o n . H o w e v e r , as F i g u r e 14.4 shows, t h e c a r d i n a l f u n c t i o n s are j u s t
as l o c a l i z e d as those for t h e G a u s s i a n basic f u n c t i o n s , a n d t h u s t h e f u n c t i o n space
Fig. 14.3 Cardinal functions for Gaussian interpolation (with e = 5) on 81 Halton points in
2
[0, l ] . Centered at an edge point (left) and at an interior point (right).
spanned by multiquadrics is a "good" local space.
Fig. 14.4 Cardinal functions for multiquadric interpolation (with e = 5) on 81 Halton points in
2
[0, l ] . Centered at an edge point (left) and at an interior point (right).
The M A T L A B program RBFCardinalFunction.m used to produce the plots in

Figures 14.1-14.3 is provided in Program 14.1. Note that we use the pseudo-inverse
(via the MATLABcommand pinv) to stably compute the inverse of the interpolation
matrix (see line 13 of Program 14.1). A specific cardinal function is then chosen in
line 15.
P r o g r a m 14.1. RBFCardinalFunction.m
7o R B F C a r d i n a l F u n c t i o n
/ Computes and p l o t s c a r d i n a l f u n c t i o n f o r 2D RBF i n t e r p o l a t i o n
/ C a l l s on: D i s t a n c e M a t r i x
0
1 r b f = @(e,r) e x p ( - ( e * r ) . " 2 ) ; ep = 5 ;
2 N = 81; g r i d t y p e = 'u';
3 neval = 8 0 ; M = neval~2;
/ Load data p o i n t s
0
4 name = s p r i n t f ( 'Data2D_%d/ s' ,N,gridtype) ; load(name)

0
5 ctrs = dsites; 7 c e n t e r s c o i n c i d e w i t h data s i t e s

6 g r i d = l i n s p a c e (0 ,1, n e v a l ) ; [xe,ye] = mesh.gr i d ( g r i d ) ;
7 epoints = [xe(:) y e ( : ) ] ;
7o Compute d i s t a n c e m a t r i x between e v a l u a t i o n p o i n t s and c e n t e r s
% Compute d i s t a n c e m a t r i x between t h e data s i t e s and c e n t e r s
% Compute c a r d i n a l f u n c t i o n s a t e v a l u a t i o n p o i n t s
12 invIM = p i n v ( I M ) ;
% centered a t d a t a s i t e ( 5 0 )
13 f o r j=l:M
14 cardvec = ( i n v I M * E M ( j , : ) ' ) ' ;
15 cardfun(j) = cardvec(50);
16 end
17 figure
18 RBFplot = s u r f ( x e , y e , r e s h a p e ( c a r d f u n , n e v a l , n e v a l ) ) ;
19 set(RBFplot,'FaceColor','interp','EdgeColor','none')
20 colormap autumn; v i e w ( [ 1 4 5 4 5 ] ) ; camlight; l i g h t i n g gouraud
14.3 The Power Function
A n o t h e r i m p o r t a n t i n g r e d i e n t needed for o u r e r r o r estimates is t h e so-called power

s
function. T o t h i s end, we consider a d o m a i n fl C M . T h e n for any s t r i c t l y p o s i t i v e
definite k e r n e l <& G C(fl x fl), a n y set o f d i s t i n c t p o i n t s X = {x\,..., XN} ^ ^l, a n d
any vector u G JR^, we define t h e q u a d r a t i c form
N N N
Q(u) = $(x,x) - 2^>2uj$(x,Xj) + y^ y^ UiUj$(x ,x ).

j j i j
j=l i=l j = l
Then
s
D e f i n i t i o n 1 4 . 1 . Suppose fiCl a n d $ G C(fl x fl) is s t r i c t l y p o s i t i v e d e f i n i t e
S
on JR . For a n y d i s t i n c t p o i n t s X = {x,..., XN} C fl t h e power function is defined
by
w h e r e u* is t h e v e c t o r o f c a r d i n a l f u n c t i o n s f r o m T h e o r e m 1 4 . 1 .
U s i n g t h e d e f i n i t i o n o f t h e n a t i v e space n o r m f r o m t h e p r e v i o u s c h a p t e r we c a n
r e w r i t e t h e q u a d r a t i c f o r m Q(u) as
^2 Y2 Yl
N N N
Q{u) = <fr(cc, x) 2 Uj$(x, Xj) + u

i u
3 ^ { x
i i X
j )
j=l i=l j=l
N
2 x $ x
= ($(, x), $ ( - , W n } - )> ("' i))wsi)
N N
+ i j{(-i
u u x
i), H-> x
j))rt*(n)
i=i j=i
N N
3=1 3=1
2
N
(14.2)
3= 1
The name power function was chosen b y Schaback based o n i t s c o n n e c t i o n t o t h e

power function of a statistical decision f u n c t i o n (originally i n t r o d u c e d i n [ N e y m a n
a n d P e a r s o n ( 1 9 3 6 ) ] ) . I n t h e p a p e r [ W u a n d S c h a b a c k (1993)] t h e p o w e r f u n c t i o n
was referred t o as kriging function. T h i s t e r m i n o l o g y comes f r o m geostatistics (see,
e.g., [Myers (1992)]).
U s i n g t h e l i n e a r s y s t e m n o t a t i o n e m p l o y e d earlier, i.e., A\j = &(xi,Xj), i,j =
T
1 , . . . , N, u = [ i i i , . . . , w y v ] , a n d b = [<&(, xi),..., $(, XJV)] , T
we n o t e t h a t we c a n
also r e w r i t e t h e q u a d r a t i c f o r m Q(u) as
T T
Q(u) = (x, x) - 2u b(x) 4- u Au. (14.3)
T h i s suggests t w o a l t e r n a t i v e r e p r e s e n t a t i o n s o f t h e p o w e r f u n c t i o n . Using the

m a t r i x - v e c t o r n o t a t i o n for Q ( u ) , t h e p o w e r f u n c t i o n is g i v e n as
T T
P* (x)
jX = y/Q(u*(x)) = \J${x,x) - 2(u*(x)) b(x) + (u*(x)) Au*(x).
H o w e v e r , b y t h e d e f i n i t i o n o f t h e c a r d i n a l f u n c t i o n s Au*(x) = b(x), and therefore

w e have t h e t w o n e w v a r i a n t s
x x x T
P*,x{x) = \/$( , )-(u*( )) b(x)
T
= \J$(x,x) - (u*(x)) Au*(x).
These f o r m u l a s c a n be used for t h e n u m e r i c a l e v a l u a t i o n o f t h e p o w e r f u n c t i o n

a t x. T o t h i s e n d one has t o f i r s t find t h e v a l u e o f t h e c a r d i n a l f u n c t i o n s u*(x) by
s o l v i n g t h e s y s t e m Au*(x) = b(x). T h i s results i n
P*,x(x) = y/${x,x) - {b(x)) A- b(x). T l

(14.4)
Since A is a p o s i t i v e definite m a t r i x whenever $ is a s t r i c t l y p o s i t i v e d e f i n i t e k e r n e l

we see t h a t t h e p o w e r f u n c t i o n satisfies t h e b o u n d s
0<P*,*(aO < y / $ ( x , x ) .
P l o t s o f t h e p o w e r f u n c t i o n for t h e G a u s s i a n w i t h e 6 o n t h r e e different p o i n t
sets w i t h N = 81 i n t h e u n i t square are p r o v i d e d i n F i g u r e 14.5. T h e sets o f d a t a
p o i n t s are displayed o n t h e left, w h i l e t h e p l o t s o f t h e power f u n c t i o n are d i s p l a y e d
in the right column. Dependence o f t h e p o w e r f u n c t i o n o n t h e d a t a l o c a t i o n s is
clearly v i s i b l e . I n fact, t h i s c o n n e c t i o n was used i n a recent p a p e r [De M a r c h i et al.
(2005)] t o i t e r a t i v e l y o b t a i n a n o p t i m a l set o f d a t a l o c a t i o n s t h a t are independent
of t h e d a t a values.
A t t h i s p o i n t t h e power f u n c t i o n is m o s t l y a t h e o r e t i c a l t o o l t h a t helps us b e t t e r
u n d e r s t a n d e r r o r estimates since we c a n decouple t h e effects due t o t h e d a t a f u n c t i o n
/ f r o m those due t o t h e basic f u n c t i o n <& a n d t h e d a t a l o c a t i o n s X (see t h e f o l l o w i n g
T h e o r e m 14.2).
T h e power f u n c t i o n is defined i n a n analogous w a y for s t r i c t l y c o n d i t i o n a l l y
p o s i t i v e definite functions.
14.4 G e n e r i c E r r o r E s t i m a t e s for F u n c t i o n s i n Af<p(fl)
N o w we c a n give a first generic e r r o r e s t i m a t e .
s s
T h e o r e m 14.2. Let C I , $ e C ( f 2 x f2) be strictly positive definite on R,
and suppose that the points X = {xi,... ,xjy} are distinct. Denote the interpolant
to f G A/$(f2) on X by Vf. Then for every x G O
\f(x)-V (x)\<P*, (x)\\f\U .

f x m
Proof. Since / is assumed t o lie i n t h e n a t i v e space o f <& t h e r e p r o d u c i n g p r o p e r t y

of $ y i e l d s
f(x) = (/,$(", 2 0 ) ^ ( 0 ) .
W e express t h e i n t e r p o l a n t i n i t s c a r d i n a l f o r m a n d a p p l y t h e r e p r o d u c i n g p r o p e r t y
o f 3>. T h i s gives us
N
Vf(x) = J2f(*j)uj(x)
3= 1
N
3= 1
N
5
z-m-
0 0
1* o o
o o
0.8
0.6
o o
0.4
o o o o
0.2
o o o o o
o o o o o
0*
0.2 0.4 0.6 0.8 0 0
X
0 0
Fig. 14.5 Data sites and power function for Gaussian interpolant with e = 6 based on N = 81
uniformly distributed points (top), tensor-product Chebyshev points (middle), and Halton points
(bottom).
N o w a l l t h a t r e m a i n s t o be done is t o c o m b i n e t h e t w o f o r m u l a s j u s t d e r i v e d a n d
apply the Cauchy-Schwarz inequality. T h u s ,
N
\f(x)-V (x)\
f = (f,&(-,x)-Y2 j( )*(-i j))M-*(n)
u x x
3= 1
<
i=i
= H/II.V(n)-P*,*(aO,
where we have a p p l i e d (14.2) a n d t h e d e f i n i t i o n o f t h e power f u n c t i o n .

One o f t h e m a i n benefits o f T h e o r e m 14.2 is t h a t we are n o w able t o e s t i m a t e
the i n t e r p o l a t i o n e r r o r b y c o n s i d e r i n g t w o i n d e p e n d e n t phenomena:
the smoothness o f t h e d a t a ( m e a s u r e d i n t e r m s o f t h e n a t i v e space n o r m o f /

w h i c h is independent o f t h e d a t a l o c a t i o n s , b u t does d e p e n d o n <&),
a n d t h e c o n t r i b u t i o n due t o t h e use o f t h e specific k e r n e l (i.e., basic f u n c t i o n )
$ and the d i s t r i b u t i o n o f the d a t a (measured i n terms of the power function
independent o f t h e a c t u a l d a t a values).
T h i s is analogous t o t h e s t a n d a r d e r r o r e s t i m a t e for p o l y n o m i a l i n t e r p o l a t i o n c i t e d
i n most n u m e r i c a l analysis t e x t s . N o t e , however, t h a t , for a n y g i v e n basic f u n c t i o n
$ , a change o f t h e shape p a r a m e t e r e w i l l have a n effect o n b o t h t e r m s i n t h e e r r o r
b o u n d i n T h e o r e m 14.2 since t h e n a t i v e space n o r m o f / varies w i t h e.
14.5 Error Estimates i n Terms of the Fill Distance
T h e next step is t o refine t h i s e r r o r b o u n d b y expressing t h e influence o f t h e d a t a

locations i n t e r m s o f t h e fill distance. A n d t h e n , o f course, t h e b o u n d needs t o be
specialized t o v a r i o u s choices o f basic f u n c t i o n s <E>.
T h e most c o m m o n s t r a t e g y for o b t a i n i n g e r r o r b o u n d s i n n u m e r i c a l analysis is
t o take advantage o f t h e p o l y n o m i a l p r e c i s i o n o f a m e t h o d (at least l o c a l l y ) , a n d
t h e n t o a p p l y a T a y l o r e x p a n s i o n . W i t h t h i s i n m i n d we observe
S
T h e o r e m 1 4 . 3 . Let fl C JR , and suppose 3> C(fl x fl) is strictly positive definite
on R . s
Let X = {x\,..., XN} be a set of distinct points in fl, and define the
quadratic form Q(u) as in (14-2). The minimum of Q(u) is given for the vector
u = u*(x) from Theorem 14-1, i-e.,
N
Q(u*(x)) < Q(u) for all u e R .
Proof. W e showed above (see (14.3)) t h a t
T T
Q{u) = $(x, x) - 2u b(x) + u Au.
T h e m i n i m u m o f t h i s q u a d r a t i c f o r m is g i v e n b y t h e s o l u t i o n o f t h e linear s y s t e m
Au = b(x).
T h i s , however, yields t h e c a r d i n a l f u n c t i o n s u = u*(x).

I n t h e p r o o f b e l o w we w i l l use a special coefficient v e c t o r u w h i c h p r o v i d e s t h e

p o l y n o m i a l p r e c i s i o n desired for t h e p r o o f o f t h e r e f i n e d e r r o r e s t i m a t e . I t s existence
is g u a r a n t e e d b y t h e f o l l o w i n g t h e o r e m o n local polynomial reproduction proved i n
[ W e n d l a n d (2005a)]. T h i s t h e o r e m r e q u i r e s t h e n o t i o n o f a d o m a i n t h a t satisfies a n
i n t e r i o r cone c o n d i t i o n .
S
D e f i n i t i o n 1 4 . 2 . A r e g i o n fl C I R satisfies a n interior cone condition i f t h e r e exists
a n angle 6 G ( 0 , 7 r / 2 ) a n d a r a d i u s r > 0 s u c h t h a t for every x fl t h e r e exists a
u n i t v e c t o r (x) such t h a t t h e cone
a T
C={x + Xy: y G E , | | y | | = 1 , y (x)
2 >cosd, AG[0,r]}
is c o n t a i n e d i n fl.
A consequence o f t h e i n t e r i o r cone c o n d i t i o n is t h e fact t h a t a d o m a i n that

satisfies t h i s c o n d i t i o n c o n t a i n s b a l l s o f a c o n t r o l l a b l e r a d i u s . I n p a r t i c u l a r , t h i s w i l l
be i m p o r t a n t w h e n b o u n d i n g t h e r e m a i n d e r o f t h e T a y l o r e x p a n s i o n s b e l o w . For
m o r e d e t a i l s see [ W e n d l a n d (2005a)].
E x i s t e n c e o f a n a p p r o x i m a t i o n scheme w i t h l o c a l p o l y n o m i a l p r e c i s i o n is g u a r -
anteed by
S
Theorem 1 4 . 4 . Suppose fl C I R is bounded and satisfies an interior cone condi-
tion, and let be a non-negative integer. Then there exist positive constants ho, c\,
and C2 such that for all X = {x,..., Ejv} C fl with h ,nx < ho and every x G fl
there exist numbers ui(x),..., ujv(x) with
N
(1) ^Ûj(x)p(xj) = p(x) for all polynomials p GII|,
i=i
N
(2) X>;(a:)| <ci,
3= 1
(3) Uj{x) = 0 if \\x - xj\\ 2 > c h Q.
2 Xj
P r o p e r t y (1) y i e l d s t h e p o l y n o m i a l p r e c i s i o n , a n d p r o p e r t y (3) shows t h a t the

scheme is l o c a l . T h e b o u n d i n p r o p e r t y ( 2 ) is essential for c o n t r o l l i n g t h e g r o w t h
o f e r r o r e s t i m a t e s a n d t h e q u a n t i t y o n t h e l e f t - h a n d side o f (2) is k n o w n as the
Lebesgue constant at x.
I n t h e f o l l o w i n g t h e o r e m a n d i t s p r o o f w e w i l l m a k e r e p e a t e d use o f m u l t i - i n d e x
n o t a t i o n and m u l t i v a r i a t e T a y l o r expansions. F o r j3 = ( / 5 i , . . . , B)
s G NQ w i t h
\(3\ = YH=i Pi w e
define t h e d i f f e r e n t i a l o p e r a t o r D 13
as
9 l m
(dxi)h (dx )0*'

s
a n d t h e n o t a t i o n D^îw, ) used b e l o w i n d i c a t e s t h a t t h e o p e r a t o r is a p p l i e d t o
$ ( i y , ) v i e w e d as a f u n c t i o n o f t h e second v a r i a b l e .
T h e m u l t i v a r i a t e T a y l o r e x p a n s i o n o f t h e f u n c t i o n $ ( t o , ) centered a t w is g i v e n
by
$(w,z)= J2 3 i \ z - w f + R(w,z)
\/3\<2k P
'
w i t h remainder
R(w,z)= 2^ /3!
\0\=2k P
'
where ^ , 2 lies somewhere o n t h e l i n e segment c o n n e c t i n g w a n d z.

T h e generic error e s t i m a t e o f T h e o r e m 14.2 c a n n o w be f o r m u l a t e d i n t e r m s o f
t h e f i l l distance.
S
T h e o r e m 14.5. Suppose fl C JR is bounded and satisfies an interior cone condi-
2k
tion. Suppose <& G C (fl x fl) is symmetric and strictly positive definite. Denote
the interpolant to f G A/$(f2) on the set X byVf. Then there exist positive constants
ho and C (independent of x, f and such that
\f(x) - V ( )\
f X < Ch y/C^x)\\f\U^ ,
XtCi a)
provided h ,n
x < ho. Here
C${x) ~ max max l-Off <E(it;, z ) |

\(3\=2k iv,zenr\B(x,C2h ,n) x
with B(x,C2hx,vi) denoting the ball of radius ch n

2 Xt centered at x.
Proof. B y T h e o r e m 14.2 we k n o w
\f(x)-Vf(x)\<P*, (*)\\f\U.M- x
Therefore, we n o w derive t h e b o u n d
P*M*) < c h k
x ^ c * { x )
for t h e power f u n c t i o n i n t e r m s o f t h e f i l l distance.

W e k n o w t h a t t h e p o w e r f u n c t i o n is defined b y
2
[P^ (x)}
x =Q(u*(x)).
M o r e o v e r , we k n o w f r o m T h e o r e m 14.3 t h a t t h e q u a d r a t i c f o r m Q(u) is m i n i m i z e d
by u u*(x). Therefore, a n y o t h e r coefficient v e c t o r u w i l l y i e l d a n u p p e r b o u n d
on t h e power f u n c t i o n . W e t a k e u = u{x) f r o m T h e o r e m 14.4 so t h a t we are ensured
t o have p o l y n o m i a l p r e c i s i o n o f degree i > 2k 1.
For t h i s specific choice o f coefficients we have
[P^ {x)\
jX
2
< Q(u) = $(x, x) - 2 ^2 Uj$(x : xj) + Y2Y2 u u
i i( iix x
i)i
3 i 3
w h e r e t h e sums are over those indices j w i t h uj ^ 0. N o w we a p p l y t h e T a y l o r

e x p a n s i o n c e n t e r e d a t x t o <&(cc, ) a n d c e n t e r e d a t Xi t o &(xi, ), and evaluate b o t h
f u n c t i o n s a t Xj. T h i s yields
Q(u) = ${x,x)-2Y J m ( g
i - x
f R
+ (> x x
i)
\/3\<2k
,/3
D%$(xi,Xi)
W
+ EE * 3
E
\0\<2k
{xj Xi)^ ~\~ R{x>i, Xj)
N e x t , we i d e n t i f y p(z) = [z x)& so t h a t p(x) 0 unless /3 = 0. T h e r e f o r e the

p o l y n o m i a l p r e c i s i o n p r o p e r t y o f t h e coefficient v e c t o r u s i m p l i f i e s t h i s expression
to
Q(u) (x, x) 2(x, x) 2 23 UjR{x, Xj)

3
+* E P
*(* ~ *^ + E E ( i 4 . 5 )
i |/3|<2fc ' i j
Now we c a n a p p l y t h e T a y l o r e x p a n s i o n a g a i n a n d m a k e t h e o b s e r v a t i o n that
D^(xi,Xj)
E
\/3\<2k
0!
(x - Xi) 13
= <E>(xi, x) - R(xi, x). (14.6)
I f we use (14.6) a n d r e a r r a n g e t h e t e r m s i n (14.5) we get
Q(u) = Q(x, x) 23 u
2R(x,
j Xj) ^3 UiR(x{,Xj)
+ 23 U
i [( ii x x
) - R( i, x x
)\ (14.7)
O n e f i n a l T a y l o r e x p a n s i o n we need is ( u s i n g t h e s y m m e t r y o f <E)
$(xi,x) = &(x,Xi)= 2^ ^ ^ ~ ^ - ( x i - x ) p
+ R(x,Xi). (14.8)
\/3\<2k
I f we i n s e r t (14.8) i n t o (14.7) a n d once m o r e t a k e a d v a n t a g e o f t h e p o l y n o m i a l

p r e c i s i o n p r o p e r t y of t h e coefficient v e c t o r u we are left w i t h
Q( )u
= _
U
J
R(x, Xj) + R(xj,x) 23 UiR(x Xj)i:
Now T h e o r e m 14.4 a l l o w s us t o b o u n d ]3 \uj\ < c\. M o r e o v e r , \\x Xj H2 < C2hx,

and II H2 < 2c2Â",r2- T h e r e f o r e , a l l t h r e e r e m a i n d e r t e r m s c a n be bounded
by a n expression o f t h e f o r m C / I ^ Q C $ ( C C ) . H e r e we m a d e use o f t h e i n t e r i o r cone
p r o p e r t y o f Q e n a b l i n g us t o define t h e t e r m C^(x). C o m b i n i n g these b o u n d s a n d
t a k i n g t h e square r o o t y i e l d s t h e s t a t e d b o u n d for t h e p o w e r f u n c t i o n .
2 k
T h e o r e m 14.5 says t h a t i n t e r p o l a t i o n w i t h a C s m o o t h k e r n e l <E> has approx-
i m a t i o n order k. T h u s , for i n f i n i t e l y s m o o t h s t r i c t l y p o s i t i v e definite functions
such as t h e Gaussians, L a g u e r r e - G a u s s i a n s , P o i s s o n r a d i a l f u n c t i o n s , a n d t h e gen-
eralized inverse m u l t i q u a d r i c s we see t h a t t h e a p p r o x i m a t i o n o r d e r k is a r b i t r a r i l y
h i g h . For s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s w i t h l i m i t e d smoothness such as t h e
M a t e r n f u n c t i o n s , t h e W h i t t a k e r r a d i a l f u n c t i o n s , as w e l l as a l l o f t h e compactly
s u p p o r t e d functions, t h e a p p r o x i m a t i o n o r d e r is l i m i t e d b y t h e smoothness o f t h e
basic f u n c t i o n .
T h e e s t i m a t e i n T h e o r e m 14.5 is s t i l l generic. I t does n o t f u l l y a c c o u n t for t h e
p a r t i c u l a r basic f u n c t i o n <& b e i n g used for t h e i n t e r p o l a t i o n since t h e factor C$(x)
s t i l l depends o n <E. M o r e o v e r , we p o i n t o u t t h a t t h e t e r m C$(x) may include a
h i d d e n dependence o n hx,n- For m o s t basic f u n c t i o n s i t w i l l be possible t o use
C<s>(x) t o "squeeze o u t " a d d i t i o n a l powers o f h. T h i s is t h e reason for s p l i t t i n g t h e
c o n s t a n t i n f r o n t o f t h e fo-power i n t o a generic C a n d a C$(x).
T h e s t a t e m e n t o f T h e o r e m 14.5 c a n be generalized for s t r i c t l y c o n d i t i o n a l l y pos-
i t i v e definite functions a n d also t o cover t h e error for a p p r o x i m a t i n g t h e d e r i v a t i v e s
of / b y derivatives o f Vf. W e state this general theorem w i t h o u t comment (c.f.
[ W e n d l a n d (2005a)] for d e t a i l s ) .
S
Theorem 14.6. Suppose Q C ] R is open and bounded and satisfies an interior
2k
cone condition. Suppose <& C (Q x Q.) is symmetric and strictly conditionally
positive definite of order m on W. Denote the interpolant to f j\f<&(Q) on the
(m 1 )-unisolvent set X by Vf. Fix ex NQ with \ot\ < k. Then there exist positive
constants ho and C (independent of x, f and such that
a a
\D f(x) - D V (x)\ f < C h ^ V C ^ l f U ^ ,
provided h ,n
x < ho- Here
C<f,(x) = max max |>f D7&(w, z)\.

|/3| + |-y|=2fc
N o t e t h a t for c o n d i t i o n a l l y p o s i t i v e definite f u n c t i o n s we have o n l y a n a t i v e

space s e m i - n o r m instead o f a n o r m .
1
Chapter 15
Refined and Improved Error Bounds
15.1 N a t i v e S p a c e E r r o r B o u n d s for S p e c i f i c B a s i s F u n c t i o n s
For t h e first p a r t o f t h i s c h a p t e r we discuss t h e n o n - s t a t i o n a r y s e t t i n g . T h e ad-

d i t i o n a l refinement of t h e e r r o r e s t i m a t e o f T h e o r e m 14.6 for specific functions
is r a t h e r t e c h n i c a l (for d e t a i l s see, e.g., the b o o k [Wendland (2005a)]). A large
b o d y o f l i t e r a t u r e exists o n t h i s t o p i c such as, e.g., [ B u h m a n n a n d D y n (1991);
L i g h t (1996); L i g h t a n d W a y n e ( 1 9 9 5 ) ; L i g h t a n d W a y n e (1998); M a d y c h (1992);
M a d y c h a n d N e l s o n ( 1 9 9 2 ) ; N a r c o w i c h a n d W a r d (2004); N a r c o w i c h et al. ( 2 0 0 3 ) ;
N a r c o w i c h et al. ( 2 0 0 5 ) ; Schaback ( 1 9 9 5 b ) ; Schaback (1996); W e n d l a n d (1998);
W e n d l a n d (1997); W u a n d Schaback (1993); Y o o n ( 2 0 0 3 ) ] ) . W e n o w l i s t some o f
t h e results t h a t can be o b t a i n e d .
15.1.1 Infinitely Smooth Basis Functions
A s m e n t i o n e d before, a n a p p l i c a t i o n o f T h e o r e m 14.6 t o i n f i n i t e l y s m o o t h func-

t i o n s such as Gaussians o r generalized (inverse) m u l t i q u a d r i c s i m m e d i a t e l y y i e l d s
a r b i t r a r i l y h i g h algebraic convergence rates, i.e., for every N a n d | a | < we
have
a a
\D f(x) - D T (x)\ f < CV/-ll|/|^ ( n ) . (15.1)
whenever / Af$>(fl). A c o n s i d e r a b l e a m o u n t o f w o r k has gone i n t o i n v e s t i g a t i n g
t h e dependence o f t h e c o n s t a n t Ci o n (see, e.g., [ W e n d l a n d ( 2 0 0 1 b ) ] ) .
U s i n g different p r o o f t e c h n i q u e s (see, e.g., [ M a d y c h a n d N e l s o n (1988)] o r [ W e n d -
l a n d (2005a)] for d e t a i l s ) i t is possible t o d e r i v e m o r e precise e r r o r b o u n d s for Gaus-
sians a n d (inverse) m u l t i q u a d r i c s . T h e r e s u l t i n g t h e o r e m f r o m [ W e n d l a n d (2005a)]
is
T h e o r e m 1 5 . 1 . Let fl be a cube in M . . Suppose S

that <E> = tp(|| ||) is a strictly con-
ditionally positive definite radial function such that ip = <^(\A) satisfies \ip^(r)\ <
\M L
for all integers > Q and all r > 0, where M is a fixed positive constant.
Then there exists a constant c such that for any f j\f$>(fl)
\\f--P \\
f Loo(Q) <e*^|/Ufc ( n ) > (15.2)
125
for all data sites X with sufficiently small fill distance hx,a-
e
Moreover, if ip satisfies even \ipW(r)\ < M, then
e l n l
Wf-PfU^n) <e ^n' ||/|k. n) ( (15-3)
provided hx,o, is sufficiently small.
_ e 2 x 2 e T
Example 15.1. For Gaussians &(x) = e H H , e > 0 fixed, we have ip{r) = e~ * ',
e 2 i 2 r 2
so t h a t ipW(r) = (-l) e e- for t > = 0. T h u s , M = e , a n d t h e e r r o r b o u n d
0
(15.3) applies. T h i s k i n d o f e x p o n e n t i a l a p p r o x i m a t i o n o r d e r is u s u a l l y r e f e r r e d t o
as spectral (or even s u p e r - s p e c t r a l ) a p p r o x i m a t i o n o r d e r . W e emphasize t h a t t h i s
nice p r o p e r t y holds o n l y i n t h e n o n - s t a t i o n a r y s e t t i n g a n d for d a t a f u n c t i o n s / t h a t
are i n t h e n a t i v e space of t h e Gaussians such as b a n d - l i m i t e d f u n c t i o n s .
2
Example 15.2. For generalized (inverse) m u l t i q u a d r i c s $(x) = ( 1 + |Ja5|| )^, (3 < 0,
e 6
or 0 < 3 N , we have ip(r) = ( 1 + r ) ^ . I n t h i s case one c a n show t h a t \ip (r)\ < \M
w h e n e v e r ^ > \3~\. H e r e M = 1 + 1/3+11. T h e r e f o r e , t h e e r r o r e s t i m a t e (15.2) applies,
i.e., i n t h e n o n - s t a t i o n a r y s e t t i n g generalized (inverse) m u l t i q u a d r i c s have s p e c t r a l
a p p r o x i m a t i o n order.
/2 2 2 2
E x a m p l e 1 5 . 3 . For L a g u e r r e - G a u s s i a n s $(cc) = L (\\ex\\ )e- ^ , n e > 0 fixed,
2 2 e2r e
w e have ip(r) = Ln (e r)e~ a n d t h e d e r i v a t i v e s ip^ w i l l be b o u n d e d b y ip^ \0) =
2i
Pn{) , where p n is a p o l y n o m i a l o f degree n . T h u s , t h e a p p r o x i m a t i o n p o w e r o f
L a g u e r r e - G a u s s i a n s falls b e t w e e n (15.3) a n d (15.2) a n d L a g u e r r e - G a u s s i a n s have at
least s p e c t r a l a p p r o x i m a t i o n p o w e r .
15.1.2 Basis Functions with Finite Smoothness
For f u n c t i o n s w i t h f i n i t e s m o o t h n e s s (such as t h e M a t e r n f u n c t i o n s , r a d i a l powers,

t h i n p l a t e splines, a n d W e n d l a n d ' s c o m p a c t l y s u p p o r t e d f u n c t i o n s ) i t is possible t o
b o u n d t h e c o n s t a n t C$(x) b y some a d d i t i o n a l p o w e r s o f h, a n d t h e r e b y t o i m p r o v e
k
t h e o r d e r p r e d i c t e d b y T h e o r e m 14.6. I n p a r t i c u l a r , for C f u n c t i o n s t h e factor
Cq>(x) can be expressed as
C*(x) = max \\D^\\ L o o { B ( 0 j 2 c h x n ) )
i n d e p e n d e n t o f x (see [ W e n d l a n d ( 2 0 0 5 a ) ] ) . T h e r e f o r e , t h i s results i n t h e f o l l o w i n g
e r r o r e s t i m a t e s (see, e.g., [ W e n d l a n d ( 2 0 0 5 a ) ] , o r t h e m u c h earlier [ W u a n d Schaback
(1993)] w h e r e o t h e r p r o o f t e c h n i q u e s were used).
K0 > w e e t
E x a m p l e 1 5 . 4 . For t h e M a t e r n f u n c t i o n s (CE) = ~^-^vlt) ' @ '
a
\D~f{x)-D V,{x)\ < Ch -*-\f\^ . x m (15.4)
p r o v i d e d \a\ < 3 f ^ ! ) hx,u 5

is sufficiently s m a l l , a n d / G J\f<s>(Q).
15. Refined and Improved Error Bounds 127
Example 1 5 . 5 . For t h e c o m p a c t l y s u p p o r t e d W e n d l a n d functions <fr fc(cc)

Si =
35
^s.fcdl !!) f h i s first r e f i n e m e n t leads t o
a | t t |
\D f{x) - DV (x)\ f < C^*- ||/||^ ( n ) . (15.5)
p r o v i d e d | a | < k, hx,n is s u f f i c i e n t l y s m a l l , a n d / A/$(S1).

2
E x a m p l e 1 5 . 6 . For t h e r a d i a l p o w e r s $(sc) = ( - 1 ) ^ / 1 ||a:||0, 0 < 3 2 N , w e get
a
\D f(x) - D<*V {x)\ } < C h ^ l f U ^ . (15.6)
x
p r o v i d e d \a\ < ^ ~ , hx,o,
2 is s u f f i c i e n t l y s m a l l , a n d / G A / $ ( f 2 ) .
f e + 1 2 / i :
E x a m p l e 1 5 . 7 . For t h i n p l a t e splines <&(x) = ( - l ) ||cc|| l o g ||cc||, w e get
a a k
\D f{x) - D V (x)\ f < Ch -^\fW, .x {Cl) (15.7)
p r o v i d e d \ot\ < k 1, hx,n is s u f f i c i e n t l y s m a l l , a n d / G j\f<s>(fl).
15.2 I m p r o v e m e n t s for N a t i v e S p a c e E r r o r Bounds
R a d i a l powers a n d t h i n p l a t e splines c a n be i n t e r p r e t e d as a g e n e r a l i z a t i o n o f u n i -
v a r i a t e n a t u r a l splines. T h e r e f o r e , w e k n o w t h a t t h e a p p r o x i m a t i o n o r d e r e s t i m a t e s
o b t a i n e d v i a t h e n a t i v e space a p p r o a c h are n o t o p t i m a l . For e x a m p l e , for i n t e r -
p o l a t i o n w i t h u n i v a r i a t e piecewise l i n e a r splines ( i . e . , $(cc) = ||cc|| i n x G R ) w e
k n o w t h e a p p r o x i m a t i o n o r d e r t o b e O(h), whereas t h e e s t i m a t e (15.6) y i e l d s o n l y
2 2
a p p r o x i m a t i o n o r d e r 0{h}/ ). S i m i l a r l y , for t h i n p l a t e splines &(x) = ||cc|| l o g ||ic||
2
one w o u l d expect o r d e r 0(h ) i n t h e case o f p u r e f u n c t i o n a p p r o x i m a t i o n . H o w e v e r ,
t h e e s t i m a t e (15.7) yields o n l y 0(h). These t w o e x a m p l e s suggest t h a t i t s h o u l d be
possible t o " d o u b l e " t h e a p p r o x i m a t i o n o r d e r s o b t a i n e d t h u s far.
O n e c a n i m p r o v e t h e e s t i m a t e s for f u n c t i o n s w i t h finite smoothness ( i . e . , M a t e r n
f u n c t i o n s , W e n d l a n d f u n c t i o n s , r a d i a l powers, a n d t h i n p l a t e splines) b y e i t h e r (or
b o t h ) o f t h e f o l l o w i n g t w o ideas:
b y r e q u i r i n g t h e d a t a f u n c t i o n / t o be even s m o o t h e r t h a n w h a t t h e n a t i v e
space prescribes, i.e., b y b u i l d i n g c e r t a i n b o u n d a r y c o n d i t i o n s i n t o t h e n a t i v e
space;
b y u s i n g weaker n o r m s t o m e a s u r e t h e e r r o r .
T h e idea t o localize t h e d a t a b y a d d i n g b o u n d a r y c o n d i t i o n s was i n t r o d u c e d i n

t h e p a p e r [ L i g h t a n d W a y n e ( 1 9 9 8 ) ] . T h i s " t r i c k " a l l o w s us t o "square" t h e a p p r o x -
i m a t i o n order, a n d t h u s r e a c h t h e e x p e c t e d a p p r o x i m a t i o n orders. T h e second i d e a
can a l r e a d y be f o u n d i n t h e e a r l y p a p e r [ D u c h o n ( 1 9 7 8 ) ] .
A f t e r a p p l y i n g b o t h o f these t e c h n i q u e s t h e f i n a l a p p r o x i m a t i o n o r d e r e s t i m a t e
for i n t e r p o l a t i o n w i t h t h e c o m p a c t l y s u p p o r t e d f u n c t i o n s <& fc is (see [ W e n d l a n d S)
(1997)])
2k 1+s
11/ - P / I U n ) < Ch + \\f\\ , r s
a ( w k+ + {R3)1 (15.8)
2 f c + 1 + S S s
w h e r e / is assumed t o lie i n t h e subspace W 2 ( R ) o f Af<j,(R ) = W^"^ . For
r =
e x a m p l e , for t h e p o p u l a r basic f u n c t i o n ^3,1 ( ) 0- ~~ r)+(4r + 1) w e have
6
\\f~V \\ ^ f L <Ch \\f\\ e . w iRs)
N o t e t h a t t h e n u m e r i c a l e x p e r i m e n t s i n T a b l e 12.2 p r o d u c e d R M S - c o n v e r g e n c e r a t e s
o n l y as h i g h as 4.5.
For r a d i a l powers a n d t h i n p l a t e splines o n e o b t a i n s L 2 - e r r o r e s t i m a t e s o f o r -
f3+s 2 k + s
der 0(h ) and 0 ( h ), respectively. T h e s e e s t i m a t e s a r e o p t i m a l , i.e., e x a c t
a p p r o x i m a t i o n orders, as s h o w n i n [ B e j a n c u ( 1 9 9 9 ) ] .
M o r e w o r k o n i m p r o v e d e r r o r b o u n d s c a n b e f o u n d i n , e.g., [ J o h n s o n (2004a)]
or [Schaback ( 1 9 9 9 b ) ] .
15.3 E r r o r B o u n d s for F u n c t i o n s O u t s i d e t h e N a t i v e Space
T h e e r r o r b o u n d s m e n t i o n e d so f a r were a l l v a l i d u n d e r t h e a s s u m p t i o n t h a t t h e
f u n c t i o n / p r o v i d i n g t h e d a t a c a m e f r o m ( a subspace o f ) t h e n a t i v e space o f t h e
RBF employed i n the interpolation. W e n o w m e n t i o n a f e w recent r e s u l t s that
provide error bounds for i n t e r p o l a t i o n o f functions / not i n t h e n a t i v e space o f
t h e basic f u n c t i o n . I n p a r t i c u l a r , t h e case w h e n / lies i n some S o b o l e v space is o f
great i n t e r e s t . A r a t h e r general t h e o r e m w a s r e c e n t l y g i v e n i n [ N a r c o w i c h et al.
(2005)]. I n t h i s t h e o r e m N a r c o w i c h , W a r d a n d W e n d l a n d p r o v i d e S o b o l e v b o u n d s
for f u n c t i o n s w i t h m a n y zeros. H o w e v e r , since t h e i n t e r p o l a t i o n e r r o r f u n c t i o n is
j u s t such a f u n c t i o n , these b o u n d s have a d i r e c t a p p l i c a t i o n t o o u r s i t u a t i o n . W e
p o i n t o u t t h a t t h i s t h e o r e m a g a i n applies t o t h e n o n - s t a t i o n a r y s e t t i n g .
T h e o r e m 15.2. Let k be a positive integer, 0 < a < l , l \ct\ + s/p or, for p = 1, k > \a\ -f- s.
Let X C fl be a discrete set with fill distance h hx,n where fl is a compact set
k+a
with Lipschitz boundary which satisfies an interior cone condition. If u e W (fl)
satisfies u\x = 0, then
k+a a s 1 1
1 , < rh ~\ \- ( /P- /l)+1 | 7/ t ,
where c is a constant independent of u and h, and {x) + is the cutoff function.
k + a
Suppose we have a n i n t e r p o l a t i o n process V : W ' (fl) > V t h a t m a p s S o b o l e v
k + a
f u n c t i o n s t o a f i n i t e - d i m e n s i o n a l subspace V o f W (fl) w i t h t h e a d d i t i o n a l p r o p -
f e + C T
e r t y \pf\ k+a^ w < |/lvK ( f i ) ' t h e n T h e o r e m 15.2 i m m e d i a t e l y y i e l d s t h e e r r o r
estimate
f c + C T s
T h e a d d i t i o n a l p r o p e r t y [Pf\ k+a^ w < |/lw ( n ) * c e r t a i n l y satisfied p r o v i d e d
t h e n a t i v e space o f t h e basic f u n c t i o n is a Sobolev space. T h u s , T h e o r e m 15.2
p r o v i d e s a n a l t e r n a t i v e t o t h e p o w e r f u n c t i o n a p p r o a c h discussed i n t h e p r e v i o u s
chapter i f we base V o n l i n e a r c o m b i n a t i o n s o f shifts o f t h e basic f u n c t i o n <E>. T h i s

new a p p r o a c h has t h e a d v a n t a g e t h a t t h e t e r m C<j>(x) w h i c h m a y d e p e n d o n b o t h
<& a n d X no longer needs t o be d e a l t w i t h .
I n p a r t i c u l a r , t h e a u t h o r s o f [ N a r c o w i c h et al. (2005)] s h o w t h a t i f t h e F o u r i e r
t r a n s f o r m o f <E> satisfies
ci(l + ||u>||l)- T
< $(w) < c (l + 2 ||a;||l)- , T
- oo, u e R,
s
(15.9)
t h e n t h e above e r r o r e s t i m a t e h o l d s w i t h r = k + a a n d p = 2 p r o v i d e d t h e f i l l
distance is sufficiently s m a l l . E x a m p l e s o f basic f u n c t i o n s w i t h a n a p p r o p r i a t e l y
decaying F o u r i e r t r a n s f o r m are p r o v i d e d b y t h e families o f W e n d l a n d or M a t e r n
functions. I n a d d i t i o n , N a r c o w i c h , W a r d a n d W e n d l a n d s h o w t h a t analogous e r r o r
b o u n d s h o l d for r a d i a l powers a n d t h i n p l a t e splines (whose n a t i v e spaces are B e p p o -
L e v i spaces).
For f u n c t i o n s / outside t h e n a t i v e space o f a basic f u n c t i o n <E> whose F o u r i e r
t r a n s f o r m satisfies (15.9) N a r c o w i c h , W a r d a n d W e n d l a n d p r o v e
T h e o r e m 15.3. Let k and n be integers with 0 < n < k < r and k > s/2, and let
k
f 6 C (fl). Also suppose that X { x i , . . . ,x^} C fl satisfies diam(A') < 1 with
sufficiently small fill distance. Then for any 1 < q < oo we have
1/ - Pflwîn) < cPx fc

C (fi)'
where px = is the mesh ratio for X and qx is the separation distance

\xmm^j \\xi - Xj\\ . 2
We r e m i n d t h e reader t h a t t h e f i l l distance c o r r e s p o n d s t o t h e r a d i u s o f t h e
largest possible e m p t y b a l l t h a t c a n be p l a c e d b e t w e e n t h e p o i n t s i n X. T h e sep-
a r a t i o n distance (c.f. C h a p t e r 16), o n t h e o t h e r h a n d , c a n be i n t e r p r e t e d as t h e
r a d i u s o f t h e largest b a l l t h a t c a n be p l a c e d a r o u n d every p o i n t i n X such t h a t n o
t w o balls o v e r l a p . T h u s , t h e m e s h r a t i o is a measure o f t h e n o n - u n i f o r m i t y o f t h e
d i s t r i b u t i o n o f the points i n X.
S i m i l a r results were o b t a i n e d earlier i n [ B r o w n l e e a n d L i g h t (2004)] (for r a d i a l
powers a n d t h i n p l a t e splines o n l y ) , a n d i n [ Y o o n (2003)] (for shifted surface splines,
see b e l o w ) .
Example 1 5 . 8 . I f we consider p o l y h a r m o n i c splines, t h e n t h e decay c o n d i t i o n

(15.9) for t h e F o u r i e r t r a n s f o r m is satisfied w i t h r = 23 for t h i n p l a t e splines a n d
w i t h T 3 for r a d i a l powers. I f we t a k e k r , n = 0, a n d q = oo i n T h e o r e m 15.3
t h e n we a r r i v e at t h e b o u n d
s 2
\f-r \ f L o o <ch^- / \\f\\ c 0 { m
for r a d i a l powers $ ( x ) = | | x | | ^ . These b o u n d s i m m e d i a t e l y c o r r e s p o n d t o t h e "op-

t i m a l " n a t i v e space b o u n d s o b t a i n e d earlier o n l y after t h e i m p r o v e m e n t s discussed
i n t h e p r e v i o u s subsection. For d a t a f u n c t i o n s / w i t h less smoothness t h e a p p r o x i -

m a t i o n order is reduced a c c o r d i n g l y .
L o w e r b o u n d s o n t h e a p p r o x i m a t i o n o r d e r for a p p r o x i m a t i o n b y p o l y h a r m o n i c
splines were r e c e n t l y p r o v i d e d i n [ M a i o r o v (2005)]. M a i o r o v studies for a n y 1 <
p, q < oo a n d 3/s > (1/p l/q)+ the error E of //^-approximation of functions
b y p o l y h a r m o n i c splines. M o r e precisely,
a s
E(W^{0, l] ),n {p ,3),L {^,lY))>cN-^ ,
N p q
w h e r e 71^(^/3,0) denotes t h e l i n e a r space f o r m e d b y a l l possible linear c o m b i n a t i o n s

of N p o l y h a r m o n i c (or t h i n - p l a t e t y p e ) splines
a n d m u l t i v a r i a t e p o l y n o m i a l s o f degree at m o s t 3 1. N o t e t h a t these b o u n d s are

i n t e r m s o f t h e n u m b e r N o f d a t a sites i n s t e a d o f t h e u s u a l fill distance h.
For t h e special cases p q = oo a n d p = 2, 1 < g < 2 t h e above lower b o u n d is
s h o w n t o be a s y m p t o t i c a l l y exact.
15.4 E r r o r B o u n d s for S t a t i o n a r y Approximation
T h e s t a t i o n a r y s e t t i n g is a n a t u r a l a p p r o a c h for use w i t h l o c a l basis f u n c t i o n s . T h e

m a i n m o t i v a t i o n comes f r o m t h e c o m p u t a t i o n a l p o i n t o f v i e w . W e are i n t e r e s t e d i n
m a i n t a i n i n g sparse i n t e r p o l a t i o n m a t r i c e s as t h e d e n s i t y o f t h e d a t a increases. T h i s
can be achieved b y scaling t h e basis f u n c t i o n s p r o p o r t i o n a l t o t h e d a t a density.
I n p r i n c i p l e we c a n t a k e a n y o f o u r basic f u n c t i o n s a n d a p p l y a s c a l i n g o f t h e
variable, i.e., we replace x b y ex, e > 0. A s m e n t i o n e d several t i m e s earlier, t h i s
scaling results i n "peaked" or " n a r r o w " basis f u n c t i o n s for large values o f e, a n d
"flat" basis functions for e * 0. W e w i l l n o w discuss w h a t h a p p e n s i f we choose e
inversely p r o p o r t i o n a l t o t h e fill distance, i.e.,
for some fixed base scale eo a n d s t u d y t h e a p p r o x i m a t i o n e r r o r based o n t h e R B F

interpolant
N
Vf(x) = c ^ e h ( | |X - Xj\\),
3= 1
where
Ve
h = <p(h-)-
E x a m p l e 1 5 . 9 . A r a t h e r d i s a p p o i n t i n g fact is t h a t Gaussians do not provide any

positive approximation order, i.e., t h e a p p r o x i m a t i o n process is saturated. T h i s was
s t u d i e d b y [ B u h m a n n (1989a)] o n i n f i n i t e l a t t i c e s . H o w e v e r , for q u a s i - i n t e r p o l a t i o n
the approximate approximation a p p r o a c h o f M a z ' y a shows t h a t i t is possible t o
choose eo i n such a w a y t h a t t h e level at w h i c h t h e s a t u r a t i o n occurs can be c o n -
t r o l l e d (see, e.g., [ M a z ' y a a n d S c h m i d t ( 1 9 9 6 ) ] ) . T h e r e f o r e , Gaussians m a y v e r y
w e l l be used for s t a t i o n a r y i n t e r p o l a t i o n p r o v i d e d a n a p p r o p r i a t e i n i t i a l shape pa-
rameter is chosen. W e w i l l i l l u s t r a t e t h i s b e h a v i o r i n t h e n e x t c h a p t e r . T h e same
k i n d o f a r g u m e n t also applies t o t h e L a g u e r r e - G a u s s i a n s o f S e c t i o n 4.2.
Example 1 5 . 1 0 . Basis f u n c t i o n s w i t h c o m p a c t s u p p o r t such as t h e W e n d l a n d

functions also do n o t p r o v i d e a n y p o s i t i v e a p p r o x i m a t i o n o r d e r i n t h e station-
ary case. T h i s can be seen b y l o o k i n g at t h e p o w e r f u n c t i o n for t h e scaled ba-

sic f u n c t i o n & h = 3>( h-) w h i c h is o f t h e f o r m P$ ,x(x) Sh = P<$,x e {EHX) where
X h = {eh,x\,... ,ehXN} denotes t h e scaled d a t a set. M o r e o v e r , t h e fill distances o f
the sets X h a n d X satisfy hx e ,JI = hh x SL- Therefore, the power function ( w h i c h
can be b o u n d e d i n t e r m s o f t h e fill distance, c.f. t h e p r o o f o f T h e o r e m 14.5) satisfies
P<b, ,x{<x)
h < C (e h s^J
h x
for some a > 0. T h i s , however, does n o t go t o zero i f eh is chosen as i n ( 1 5 . 1 0 ) .

If, o n t h e o t h e r h a n d , we w o r k i n t h e a p p r o x i m a t e a p p r o x i m a t i o n r e g i m e , t h e n
we can o b t a i n g o o d convergence i n m a n y cases (see t h e n e x t c h a p t e r for some
numerical experiments).
E x a m p l e 1 5 . 1 1 . S t a t i o n a r y i n t e r p o l a t i o n w i t h (inverse) m u l t i q u a d r i c s , r a d i a l p o w -
ers a n d t h i n p l a t e splines presents no difficulties. I n fact, [Schaback (1995c)] shows
t h a t t h e n a t i v e space e r r o r b o u n d for t h i n p l a t e splines a n d r a d i a l powers is i n v a r i a n t
under a s t a t i o n a r y scaling. T h e r e f o r e , t h e n o n - s t a t i o n a r y b o u n d o f T h e o r e m 15.3
applies i n t h e s t a t i o n a r y case also. T h e a d v a n t a g e o f scaling t h i n p l a t e splines or
r a d i a l powers comes f r o m t h e a d d e d s t a b i l i t y one can g a i n b y p r e v e n t i n g t h e sepa-
r a t i o n distance f r o m b e c o m i n g t o o s m a l l (see C h a p t e r 16 a n d t h e w o r k o f Iske o n
local p o l y h a r m o n i c spline a p p r o x i m a t i o n , e.g., [Iske ( 2 0 0 4 ) ] ) .
Y o o n p r o v i d e s e r r o r e s t i m a t e s for s t a t i o n a r y a p p r o x i m a t i o n o f r o u g h f u n c t i o n s
(i.e., f u n c t i o n s t h a t are n o t i n t h e n a t i v e space o f t h e basic f u n c t i o n ) b y so-called
shifted surface splines. S h i f t e d surface splines are o f t h e f o r m
I r / 3 S / 2 1 2 7 2
*M = J (- ) " (1 + I N I ) " - , * odd,
1 } 2 s 2 2 1/2
\ ( - l y - ^ + ^ l + \\x\ f~ l log(l + ||a:|| ) > s even,
where s/2 < 6 G N . These f u n c t i o n s i n c l u d e a l l o f t h e (inverse) m u l t i q u a d r i c s ,
r a d i a l powers a n d t h i n p l a t e splines.
Y o o n has t h e f o l l o w i n g t h e o r e m (see [ Y o o n (2003)] for t h e L p case, a n d [ Y o o n
(2001)] for Loo b o u n d s o n l y ) .
T h e o r e m 15.4. Let $ be a shifted surface spline with shape parameter e inversely

proportional to the fill distance hx,ci- Then there exists a positive constant C (in-
dependent of X) such that for every f in the Sobolev space W - f ( f ) ) D W ^ , ( f i ) we
have
11/ - V \ \
f L p i a ) < Ch^\f\ , w { R 3 ) , 1 < p < oo,
with
7 p = mm{3,3-s/2 + s/p}.
Furthermore, if f G W?(fl) D w i t / i m a x { 0 , s / 2 - s/p} < a < 3, then
7 P _ / 3 + Q
I I / - ^ / I I M ) = ^ ) -
Y o o n ' s estimates address t h e q u e s t i o n o f h o w w e l l t h e i n f i n i t e l y s m o o t h (inverse)

m u l t i q u a d r i c s a p p r o x i m a t e f u n c t i o n s t h a t are less s m o o t h t h a n those i n t h e i r n a t i v e
space. F o r example, T h e o r e m 15.4 states t h a t / ^ - a p p r o x i m a t i o n t o f u n c t i o n s i n
s 2 2
W | ( f 2 ) , Q C E , b y m u l t i q u a d r i c s $ (x) e = y/l + ||ea;|| is o f t h e o r d e r 0{h ).
However, we emphasize once m o r e t h a t t h i s refers t o s t a t i o n a r y a p p r o x i m a t i o n o f
r o u g h f u n c t i o n s , i.e., is scaled inversely p r o p o r t i o n a l t o t h e f i l l distance a n d /
need n o t lie i n t h e n a t i v e space o f <E>, whereas t h e s p e c t r a l order g i v e n i n (15.2)
corresponds t o a p p r o x i m a t i o n o f f u n c t i o n s i n t h e n a t i v e space i n t h e n o n - s t a t i o n a r y
case w i t h fixed e.
For t h i n p l a t e splines a n d r a d i a l powers t h e a p p r o x i m a t i o n orders i n T h e o -
rem 15.4 are equivalent t o those o f T h e o r e m 15.3 a n d t h e results o f B r o w n l e e a n d
L i g h t m e n t i o n e d above. T h i s is t o be e x p e c t e d due t o t h e i n v a r i a n c e o f these basic
f u n c t i o n s w i t h respect t o scaling.
T h e second p a r t o f Y o o n ' s r e s u l t is a step t o w a r d exact approximation orders as
is t h e w o r k o f [ M a i o r o v (2005)] a n d [ B e j a n c u (1999)] m e n t i o n e d above.
15.5 C o n v e r g e n c e w i t h R e s p e c t to the S h a p e P a r a m e t e r
N o n e o f t h e e r r o r b o u n d s discussed t h u s far have t a k e n i n t o account t h e pos-

s i b i l i t y o f v a r y i n g t h e shape p a r a m e t e r e for a fixed d a t a set X. However, i n
t h e l i t e r a t u r e t h e i n f i n i t e l y s m o o t h basic f u n c t i o n s s u c h as t h e Gaussians a n d
(inverse) m u l t i q u a d r i c s are u s u a l l y f o r m u l a t e d i n c l u d i n g t h e shape p a r a m e t e r e
(or another parameter equivalent t o i t ) and one m a y wonder how a change
i n t h i s shape p a r a m e t e r affects t h e convergence properties of the R B F inter-
polant. I n fact, quite a bit of work has been spent on the quest for the
" o p t i m a l " shape p a r a m e t e r (see, e.g., [ C a r l s o n a n d F o l e y (1991); F o l e y (1994);
H a g a n a n d K a n s a (1994); K a n s a a n d C a r l s o n (1992); R i p p a (1999); T a r w a t e r (1985);
W e r t z et al. ( 2 0 0 6 ) ] ) .
Convergence o f t h e i n f i n i t e l y s m o o t h Gaussians a n d (inverse) m u l t i q u a d r i c s w i t h
respect t o t h e shape p a r a m e t e r was s t u d i e d e a r l y o n i n [ M a d y c h (1991)]. M a d y c h
showed t h a t for these basic f u n c t i o n s t h e r e exists a p o s i t i v e c o n s t a n t A < 1 such
that
l ehx
\f(x) - V/(x)\ < C\ ^ >^ (15.11)
p r o v i d e d / is i n t h e n a t i v e space o f <E>. T h i s e s t i m a t e shows t h a t t a k i n g e i t h e r t h e

shape p a r a m e t e r e or t h e f i l l d i s t a n c e h ,a x t o zero r e s u l t s i n e x p o n e n t i a l conver-
gence.
15.6 P o l y n o m i a l I n t e r p o l a t i o n as t h e L i m i t of R B F I n t e r p o l a t i o n
Recently, a n u m b e r o f a u t h o r s (see, e.g., [ D r i s c o l l a n d F o r n b e r g ( 2 0 0 2 ) ; F o r n b e r g a n d

F l y e r (2005); F o r n b e r g a n d W r i g h t ( 2 0 0 4 ) ; L a r s s o n a n d F o r n b e r g ( 2 0 0 5 ) ; Schaback
(2005); Schaback ( 2 0 0 6 b ) ] ) have s t u d i e d t h e l i m i t i n g case as e * 0 o f scaled r a d i a l
basis f u n c t i o n i n t e r p o l a t i o n w i t h i n f i n i t e l y s m o o t h basic f u n c t i o n s such as Gaussians
a n d generalized (inverse) m u l t i q u a d r i c s . I t t u r n s o u t t h a t t h e r e is a n i n t e r e s t i n g
connection t o p o l y n o m i a l interpolation.
I n [ D r i s c o l l a n d F o r n b e r g (2002)] u n i v a r i a t e (s 1) i n t e r p o l a t i o n w i t h e-scaled
i n f i n i t e l y s m o o t h r a d i a l basic f u n c t i o n s is s t u d i e d . D r i s c o l l a n d F o r n b e r g s h o w t h a t
the R B F interpolant
N
f( )
V x
= Y, M\\ ( - 3)\\)>
C X X x e
W,b]cM,
t o f u n c t i o n values a t N d i s t i n c t d a t a sites t e n d s t o t h e L a g r a n g e i n t e r p o l a t i n g
p o l y n o m i a l o f / as e 0.
T h e m u l t i v a r i a t e case is m o r e c o m p l i c a t e d . However, the l i m i t i n g R B F inter-
p o l a n t ( p r o v i d e d i t exists) is g i v e n b y a low-degree m u l t i v a r i a t e p o l y n o m i a l (see
[Larsson a n d F o r n b e r g ( 2 0 0 5 ) ; Schaback ( 2 0 0 5 ) ; Schaback ( 2 0 0 6 b ) ] ) . F o r e x a m p l e ,
if t h e d a t a sites are l o c a t e d s u c h t h a t t h e y g u a r a n t e e a u n i q u e p o l y n o m i a l i n t e r -
p o l a n t , t h e n t h e l i m i t i n g R B F i n t e r p o l a n t is g i v e n b y t h i s p o l y n o m i a l . I f p o l y n o -
m i a l i n t e r p o l a t i o n is n o t u n i q u e , t h e n t h e R B F l i m i t depends o n t h e choice o f basic
function. H o w e v e r , these s t a t e m e n t s r e q u i r e t h e R B F s t o satisfy a n (unproven)
c o n d i t i o n o n c e r t a i n coefficient m a t r i c e s A j. Pt I n [ L a r s s o n a n d F o r n b e r g (2005)]
t h e a u t h o r s also p r o v i d e a n e x p l a n a t i o n for t h e e r r o r b e h a v i o r for s m a l l values o f
t h e shape p a r a m e t e r , a n d for t h e existence o f a n o p t i m a l ( p o s i t i v e ) value o f e g i v -
ing rise t o a g l o b a l m i n i m u m o f t h e e r r o r f u n c t i o n . F o r t h e special case o f scaled
Gaussians Schaback [Schaback (2005)] shows t h a t t h e R B F i n t e r p o l a n t converges
t o t h e d e B o o r a n d R o n least p o l y n o m i a l i n t e r p o l a n t (see [ d e B o o r a n d R o n ( 1 9 9 0 ) ;
d e B o o r a n d R o n (1992a)] a n d also [ d e B o o r ( 2 0 0 6 ) ] ) as e - > 0.
I n [ F o r n b e r g a n d W r i g h t (2004)] t h e a u t h o r s describe a so-called Contour-Pade
a l g o r i t h m t h a t makes i t possible (for d a t a sets o f r e l a t i v e l y m o d e s t size) t o c o m p u t e
t h e R B F i n t e r p o l a n t for a l l values o f t h e shape p a r a m e t e r e i n c l u d i n g t h e l i m i t i n g
case e -> 0. W e present some n u m e r i c a l r e s u l t o b t a i n e d w i t h G r a d y W r i g h t ' s
M A T L A B toolbox in Chapter 17.
We w i l l later exploit the connection between R B F and p o l y n o m i a l interpolants
t o design n u m e r i c a l solvers for p a r t i a l d i f f e r e n t i a l e q u a t i o n s .
Chapter 16
Stability and Trade-Off Principles
16.1 S t a b i l i t y a n d C o n d i t i o n i n g of R a d i a l B a s i s Function
Interpolants
A s t a n d a r d c r i t e r i o n for m e a s u r i n g t h e n u m e r i c a l s t a b i l i t y o f a n approximation
m e t h o d is i t s c o n d i t i o n n u m b e r . I n p a r t i c u l a r , for r a d i a l basis f u n c t i o n i n t e r p o l a t i o n
we need t o l o o k at t h e c o n d i t i o n n u m b e r o f t h e i n t e r p o l a t i o n m a t r i x A w i t h entries
Aij = $(xi Xj). For a n y m a t r i x A i t s -condition
2 number is g i v e n b y
1
cond(A) = | | A | | | | A - | |
2 2 = ^ ,
^min
where c r m a x a n d c r i n are t h e largest a n d smallest s i n g u l a r values o f A.
m I f we
c o n c e n t r a t e o n p o s i t i v e d e f i n i t e m a t r i c e s A, t h e n t h e c o n d i t i o n n u m b e r o f A can
also t a k e be c o m p u t e d as t h e r a t i o
Amax
Amin
of t h e largest a n d smallest eigenvalues.
W h a t do we k n o w a b o u t these eigenvalues? F i r s t , G e r s h g o r i n ' s t h e o r e m (see,
e . g . , [Meyer (2000)]) says t h a t
JV
| Amax An | 5: ^ ^ | Aij |
J= I
for some i { 1 , . . . , N}. Therefore,
Amax < N max \Aij \ = N max \$(xi - Xj)\,

x,j=l,...,N Xi,XjX
w h i c h , since $ is s t r i c t l y p o s i t i v e definite, becomes
A m a x < JV$(0)
b y t h e p r o p e r t i e s o f p o s i t i v e d e f i n i t e f u n c t i o n s ( P r o p e r t y (4) i n T h e o r e m 3.1). N o w ,
s
as l o n g as t h e d a t a are n o t t o o w i l d l y d i s t r i b u t e d , N w i l l g r o w as h x Q which
is acceptable. Therefore, t h e m a i n w o r k i n e s t a b l i s h i n g a b o u n d for t h e c o n d i t i o n
n u m b e r o f A lies i n finding lower b o u n d s for A i m n (or c o r r e s p o n d i n g l y u p p e r b o u n d s
135
_ 1
for t h e n o r m o f t h e inverse | | A | | 2 ) . T h i s is t h e s u b j e c t o f several papers b y B a l l ,
N a r c o w i c h , S i v a k u m a r a n d W a r d [ B a l l et al. (1992); N a r c o w i c h et al. (1994);
N a r c o w i c h a n d W a r d (1991a); N a r c o w i c h a n d W a r d ( 1 9 9 1 b ) ; N a r c o w i c h a n d W a r d
(1992)] w h o m a k e use o f a r e s u l t f r o m [ B a l l (1992)] o n eigenvalues o f distance
m a t r i c e s . B a l l ' s result follows f r o m t h e Rayleigh quotient (or t h e C o u r a n t - F i s c h e r
T h e o r e m 9.5), w h i c h gives t h e smallest eigenvalue o f a s y m m e t r i c p o s i t i v e d e f i n i t e
m a t r i x as
T
c Ac
Amin = mm .
w 1
cK \0 C C
T h i s can be used t o p r o v e t h e f o l l o w i n g b o u n d for t h e n o r m o f t h e inverse o f A

w h i c h covers also t h e case o f c o n d i t i o n a l p o s i t i v e ( n e g a t i v e ) definiteness o f o r d e r
one.
s s
L e m m a 1 6 . 1 . Let X\,..., x^ be distinct points in R and let <3> : R > R be either
strictly positive definite, or strictly conditionally negative definite of order one with
<&(0) < 0. Also, let A be the interpolation matrix with entries A^ = $>(xi Xj). If
the inequality
N N
2
CiCj-Ay > 9\\c\\ 2
i=l j=l
c =
is satisfied whenever the components of c satisfy X^jLi j > then
l
\\A- \\ <6-\ 2
N o t e t h a t for p o s i t i v e d e f i n i t e m a t r i c e s t h e R a y l e i g h q u o t i e n t i m p l i e s 6 = A i n m
w h i c h shows w h y lower b o u n d s o n t h e smallest eigenvalue c o r r e s p o n d t o u p p e r

b o u n d s o n t h e n o r m o f t h e inverse o f A. I n o r d e r t o o b t a i n t h e b o u n d for con-
d i t i o n a l l y negative definite m a t r i c e s t h e C o u r a n t - F i s c h e r t h e o r e m 9.5 needs t o be
employed.
T h e b o u n d i n L e m m a 16.1 is t o o generic t o give us a n y i n f o r m a t i o n for specific
basic f u n c t i o n s <3>. T h i s e x t e n s i o n was a c c o m p l i s h e d i n some o f t h e o t h e r p a p e r s
m e n t i o n e d above. N a r c o w i c h a n d W a r d e s t a b l i s h b o u n d s o n t h e n o r m o f t h e inverse
of A i n t e r m s o f t h e separation distance o f t h e d a t a sites
qx = \ m i n |jac< - Xj\\ . 2
We c a n p i c t u r e qx as t h e r a d i u s o f t h e largest b a l l t h a t c a n be p l a c e d a r o u n d e v e r y
p o i n t i n X such t h a t no t w o balls o v e r l a p (see F i g u r e 16.1). T h e s e p a r a t i o n d i s t a n c e
is sometimes also referred t o as t h e packing radius. I n our MATLAB code we c a n
c o m p u t e t h e s e p a r a t i o n distance v i a
qX = m i n ( m i n ( D M _ d a t a + e y e ( s i z e ( D M _ d a t a ) ) ) ) / 2
0 0.2 0.4 0.6 0.8 1

X
Fig. 16.1 The separation distance for N = 25 Halton points (qx ~ 0.0597).
w h e r e DM_data is t h e m a t r i x o f p a i r w i s e distances a m o n g t h e d a t a sites X. The

i d e n t i t y m a t r i x is a d d e d o n l y t o a v o i d c o u n t i n g t h e d i s t a n c e o f a p o i n t Xj t o i t s e l f
as a p o t e n t i a l m i n i m u m .
T h e d e r i v a t i o n o f these b o u n d s is r a t h e r t e c h n i c a l , a n d for details we refer t o
e i t h e r t h e o r i g i n a l papers b y N a r c o w i c h , W a r d a n d co-workers l i s t e d above, the
m o r e recent p a p e r [Schaback ( 2 0 0 2 ) ] , or t h e b o o k [ W e n d l a n d (2005a)]. B y p r o v i d i n g
_ 1
m a t c h i n g lower b o u n d s for | | A | | 2 (i.e., upper b o u n d s for A i ) m n Schaback s h o w e d
t h a t t h e u p p e r b o u n d s o n t h e n o r m o f t h e inverse o b t a i n e d b y N a r c o w i c h , W a r d
a n d o t h e r s are near o p t i m a l (see [Schaback ( 1 9 9 4 b ) ] ) .
W e n o w l i s t several ( l o w e r ) b o u n d s for A i m n as d e r i v e d i n [ W e n d l a n d ( 2 0 0 5 a ) ] .
I n t h e examples b e l o w t h e e x p l i c i t (space-dependent) c o n s t a n t s
M , = 1 2 ( L i i j < 6.38s and C. = ^ ( ^ j
are used. T h e u p p e r b o u n d for M s c a n be o b t a i n e d u s i n g S t i r l i n g ' s f o r m u l a (see

[ W e n d l a n d (2005a)]).
Since t h e b o u n d s i n t h e l i t e r a t u r e for Gaussians a n d m u l t i q u a d r i c s also i n c l u d e
t h e influence o f t h e shape p a r a m e t e r e we present t h e basic f u n c t i o n s i n t h e i r scaled
v e r s i o n here.
e 2 2
Example 16.1. For Gaussians &(x) = e~ ^ one o b t a i n s
s 4 0 7 1 s 2
A m i n > C ( v ^ ) - e -
s - / ^ V -
W e see t h a t , for a fixed shape p a r a m e t e r e, t h e lower b o u n d for A i m n goes e x p o -
n e n t i a l l y t o zero as t h e s e p a r a t i o n d i s t a n c e qx decreases. Since we observed above
t h a t t h e c o n d i t i o n n u m b e r o f t h e i n t e r p o l a t i o n m a t r i x A depends o n t h e r a t i o o f
its largest a n d smallest eigenvalues a n d t h e g r o w t h o f A m a x is o f o r d e r N we see
t h a t t h e c o n d i t i o n n u m b e r g r o w s e x p o n e n t i a l l y w i t h decreasing s e p a r a t i o n d i s t a n c e .
T h i s shows t h a t , i f one adds m o r e i n t e r p o l a t i o n p o i n t s i n o r d e r t o i m p r o v e t h e ac-
c u r a c y o f t h e i n t e r p o l a n t ( w i t h i n t h e same d o m a i n fl), t h e n t h e p r o b l e m becomes
increasingly ill-conditioned. O f course one w o u l d a l w a y s expect t h i s t o h a p p e n ,

b u t here t h e i l l - c o n d i t i o n i n g grows p r i m a r i l y d u e t o t h e decrease i n t h e s e p a r a t i o n
d i s t a n c e qx, a n d n o t t o t h e increase i n t h e n u m b e r N o f d a t a p o i n t s . W e w i l l come
b a c k t o t h i s o b s e r v a t i o n w h e n w e discuss a possible change o f basis i n S e c t i o n 34.4.
O n t h e o t h e r h a n d , i f one keeps t h e n u m b e r o f p o i n t s ( o r a t least t h e s e p a r a t i o n
distance) fixed a n d i n s t e a d decreases t h e v a l u e o f e, t h e n t h e c o n d i t i o n n u m b e r o f
A suffers i n a l m o s t t h e same e x p o n e n t i a l m a n n e r . O f course, a n increase i n e c a n
be used t o i m p r o v e t h e c o n d i t i o n n u m b e r o f A (however, as we saw earlier, a t t h e
expense o f a c c u r a c y o f t h e f i t ) .
Example 16.2. F o r scaled generalized (inverse) m u l t i q u a d r i c s <$>(x) =

2
(l + I M I ) ^ , 8 e R \ N 0 one o b t a i n s
i + i 2 M e
A m i n > C(a, A e ) 4 " e- '/<* )
w i t h another explicitly k n o w n constant C(s,0,e).

T h e same c o m m e n t s as i n t h e p r e v i o u s e x a m p l e a p p l y .
/ 3 + 1 2 / 3
E x a m p l e 1 6 . 3 . F o r t h i n p l a t e splines $(cc) = ( - l ) ||cc|| l o g \\x\\, 0 G N , one
obtains
s 2
A m i n > C s C / 3 (2M )- - ^^s
w i t h another explicitly k n o w n constant c p .

I n t h i s case t h e lower b o u n d also goes t o zero w i t h decreasing s e p a r a t i o n distance.
However t h e decay is o n l y o f p o l y n o m i a l o r d e r .
2 p
E x a m p l e 1 6 . 4 . F o r t h e r a d i a l powers $ ( J C ) = ( - l ) ^ / ! \\x\\ , 0 < Q 2 N , one
obtains
s / 3
A m i n > C c s / 3 (2M )- - s 4
w i t h a n o t h e r e x p l i c i t l y k n o w n c o n s t a n t c p (different f r o m c p i n E x a m p l e 3 ) . A g a i n ,
t h e decay is o f p o l y n o m i a l order.
E x a m p l e 1 6 . 5 . F o r t h e c o m p a c t l y s u p p o r t e d f u n c t i o n s o f W e n d l a n d $> k(x) Sj
(p k(||x||)
St one o b t a i n s
k+1
A m i n > C(s,k)q%
w i t h a c o n s t a n t C(s, k) d e p e n d i n g o n s a n d k. T h e lower b o u n d goes t o zero w i t h

t h e s e p a r a t i o n distance a t a p o l y n o m i a l r a t e .
16.2 Trade-Off Principle I : Accuracy vs. Stability
T h e observations m a d e i n E x a m p l e s 16.1 a n d 16.2 above set u p t h e first trade-

off principle. T h i s p r i n c i p l e states t h a t i f w e use t h e s t a n d a r d a p p r o a c h t o t h e
R B F i n t e r p o l a t i o n p r o b l e m (i.e., s o l u t i o n o f t h e l i n e a r s y s t e m (6.3)) t h e n t h e r e is
a conflict between t h e o r e t i c a l l y achievable a c c u r a c y a n d n u m e r i c a l s t a b i l i t y . For

example, t h e e r r o r b o u n d s for n o n - s t a t i o n a r y i n t e r p o l a t i o n u s i n g i n f i n i t e l y s m o o t h
basis f u n c t i o n s show t h a t t h e e r r o r decreases ( e x p o n e n t i a l l y ) as t h e fill distance
decreases. For w e l l - d i s t r i b u t e d d a t a a decrease i n t h e fill distance also i m p l i e s a
decrease o f t h e s e p a r a t i o n distance. B u t n o w t h e c o n d i t i o n e s t i m a t e s o f t h e p r e v i o u s
subsection i m p l y t h a t t h e c o n d i t i o n n u m b e r o f A g r o w s e x p o n e n t i a l l y . T h i s leads
to numerical instabilities w h i c h make i t v i r t u a l l y impossible to o b t a i n the h i g h l y
accurate results p r o m i s e d b y t h e t h e o r e t i c a l e r r o r b o u n d s .
S i m i l a r l y , i f we use t h e shape p a r a m e t e r t o ( e x p o n e n t i a l l y ) increase a c c u r a c y
as g u a r a n t e e d b y M a d y c h ' s e r r o r b o u n d (15.11), t h e n t h e c o n d i t i o n n u m b e r a g a i n
grows e x p o n e n t i a l l y . T h i s is t o be e x p e c t e d since for s m a l l values o f e t h e basic
functions m o r e a n d m o r e resemble a c o n s t a n t f u n c t i o n , a n d therefore t h e rows (as
w e l l as c o l u m n s ) o f t h e m a t r i x A become m o r e a n d m o r e a l i k e , so t h a t t h e m a t r i x
becomes a l m o s t s i n g u l a r even for w e l l s e p a r a t e d d a t a sites.
I n t h e l i t e r a t u r e t h i s p h e n o m e n o n has been referred t o as trade-off or [uncer-
tainty) principle (see, e.g., t h e papers [Schaback ( 1 9 9 5 b ) ; Schaback ( 1 9 9 5 c ) ] ) .
Schaback l o o k e d at t h e p o w e r f u n c t i o n P<&,x a n d showed t h a t i t c a n always be
b o u n d e d f r o m above b y a f u n c t i o n F$> d e p e n d i n g o n t h e fill distance. O n t h e o t h e r
h a n d , he showed t h a t t h e R a y l e i g h q u o t i e n t c a n always be b o u n d e d f r o m b e l o w b y a
f u n c t i o n G$ d e p e n d i n g o n t h e s e p a r a t i o n distance. F u r t h e r m o r e , Schaback showed
that
G*(q )x < F*(h ,n),

x
a n d therefore, for w e l l - d i s t r i b u t e d d a t a ( w i t h q x ~ hx,n), a small error b o u n d (i.e.,

s m a l l F$(hx,n.)) w i l l necessarily r e s u l t i n a s m a l l lower b o u n d (i.e., s m a l l G<f>(qx))
for t h e R a y l e i g h q u o t i e n t , a n d therefore for t h e smallest eigenvalue. T h i s however
i m p l i e s a large c o n d i t i o n n u m b e r .
We have seen evidence o f t h e first t r a d e - o f f p r i n c i p l e i n v a r i o u s n u m e r i c a l ex-
periments. T h i s t r a d e - o f f has l e d a n u m b e r o f people t o search for a n " o p t i m a l "
value o f t h e shape p a r a m e t e r , i.e., a value t h a t yields m a x i m a l accuracy, w h i l e s t i l l
maintaining numerical stability.
I n p a r t i c u l a r , m u l t i q u a d r i c s have a t t r a c t e d t h e best p a r t o f t h i s a t t e n t i o n . F o r
2
e x a m p l e , i n his o r i g i n a l w o r k o n (inverse) m u l t i q u a d r i c i n t e r p o l a t i o n i n M Hardy
[ H a r d y (1971)] suggested u s i n g e = 1 / ( 0 . 8 1 5 d ) , w h e r e d = j^J2iL\di, a n d di is
t h e distance f r o m Xi t o i t s nearest n e i g h b o r . L a t e r , i n [Franke (1982a)], one c a n
8
find t h e r e c o m m e n d e d value e = ' ^ , w h e r e D is t h e d i a m e t e r o f t h e smallest
circle c o n t a i n i n g a l l d a t a p o i n t s . A n o t h e r s t r a t e g y for f i n d i n g a g o o d value for e is
based o n t h e o b s e r v a t i o n t h a t such a value seems t o be s i m i l a r for m u l t i q u a d r i c s a n d
inverse m u l t i q u a d r i c s (see [Foley ( 1 9 9 4 ) ] ) . O t h e r studies were r e p o r t e d i n [ C a r l s o n
a n d Foley (1992); C a r l s o n a n d N a t a r a j a n (1994)]. W e w i l l consider a m o r e recent
a l g o r i t h m p r o p o s e d i n [ R i p p a (1999)] i n t h e n e x t chapter.
16.3 Trade-Off Principle I I : A c c u r a c y a n d Stability vs. Problem

Size
M o r e recently, F o r n b e r g a n d co-workers h a v e i n v e s t i g a t e d t h e dependence o f t h e

s t a b i l i t y o n t h e values o f t h e shape p a r a m e t e r e i n a series o f p a p e r s (e.g., [Driscoll
a n d F o r n b e r g (2002); F o r n b e r g a n d W r i g h t ( 2 0 0 4 ) ; L a r s s o n a n d F o r n b e r g (2005);
P l a t t e a n d D r i s c o l l ( 2 0 0 5 ) ] ) . T h e y suggest a w a y t o s t a b l y c o m p u t e v e r y a c c u r a t e
generalized (inverse) m u l t i q u a d r i c a n d G a u s s i a n i n t e r p o l a n t s w i t h e x t r e m e values
o f e 0 b y u s i n g a c o m p l e x C o n t o u r - P a d e i n t e g r a t i o n a l g o r i t h m . T h u s , t h i s ap-
p r o a c h allows us t o overcome t h e first t r a d e - o f f p r i n c i p l e m e n t i o n e d i n t h e p r e v i o u s
section. H o w e v e r , t h e r e is a n o t h e r k i n d o f t r a d e - o f f associated w i t h t h e C o n t o u r -
P a d e a p p r o a c h . N a m e l y i t is l i m i t e d t o o n l y r a t h e r s m a l l d a t a sets ( r o u g h l y A" = 20
for s = 1 a n d N = 80 for s = 2 ) .
I n spite o f these l i m i t a t i o n s t h e C o n t o u r - P a d e a l g o r i t h m has been used t o g a i n
a n u m b e r o f t h e o r e t i c a l i n s i g h t s such as t h e c o n n e c t i o n b e t w e e n R B F i n t e r p o l a t i o n
a n d p o l y n o m i a l i n t e r p o l a t i o n m e n t i o n e d i n S e c t i o n 15.6. W e present some n u m e r i c a l
e x p e r i m e n t s based o n t h e C o n t o u r - P a d e a p p r o a c h i n t h e n e x t c h a p t e r .
16.4 Trade-Off Principle I I I : A c c u r a c y vs. Efficiency
T h e r e is also a t r a d e - o f f p r i n c i p l e for c o m p a c t l y s u p p o r t e d f u n c t i o n s . T h i s was ex-

p l a i n e d t h e o r e t i c a l l y as w e l l as i l l u s t r a t e d w i t h n u m e r i c a l e x p e r i m e n t s i n [Schaback
( 1 9 9 7 b ) ] . T h e consequences are as follows. I n t h e case o f s t a t i o n a r y i n t e r p o l a t i o n ,
i.e., i f we scale t h e s u p p o r t size o f t h e basis f u n c t i o n s p r o p o r t i o n a l t o t h e f i l l dis-
t a n c e hx,n, t h e " b a n d w i d t h " o f t h e i n t e r p o l a t i o n m a t r i x A is k e p t c o n s t a n t . T h i s
means we c a n a p p l y n u m e r i c a l a l g o r i t h m s (e.g., t h e c o n j u g a t e g r a d i e n t m e t h o d ) for
t h e s o l u t i o n o f t h e i n t e r p o l a t i o n s y s t e m t h a t c a n be p e r f o r m e d w i t h 0(N) compu-
t a t i o n a l c o m p l e x i t y . T h e m e t h o d is n u m e r i c a l l y stable, b u t t h e r e w i l l be essentially
n o convergence (see o u r earlier n u m e r i c a l e x p e r i m e n t s i n T a b l e 12.1). I n t h e n o n -
s t a t i o n a r y case, i.e., w i t h f i x e d s u p p o r t size, t h e b a n d w i d t h o f A increases as hx,n
decreases. T h i s results i n convergence (i.e., t h e e r r o r decreases) as we s h o w e d w i t h
o u r e x p e r i m e n t s i n T a b l e 12.2 a n d t h e e r r o r b o u n d s i n S e c t i o n 15.1.2. However,
t h e i n t e r p o l a t i o n m a t r i c e s w i l l b e c o m e m o r e a n d m o r e densely p o p u l a t e d as w e l l as
i l l - c o n d i t i o n e d . T h e r e f o r e , t h i s a p p r o a c h is n o t v e r y efficient.
Chapter 17
Numerical Evidence for Approximation

Order Results
17.1 I n t e r p o l a t i o n for e 0
W e b e g i n b y c o n s i d e r i n g t h e choice o f t h e shape p a r a m e t e r for a fixed d a t a set.

T h i s is p r o b a b l y t h e s i t u a t i o n t h a t w i l l arise m o s t f r e q u e n t l y i n p r a c t i c a l s i t u a t i o n s .
I n o t h e r w o r d s , we assume we are g i v e n a set o f d a t a (xj, fj), j 1 , . . . , N, with
s
d a t a sites Xj R ( w i t h s 1 or s 2 for t h e p u r p o s e o f o u r e x p e r i m e n t s ) , and
f u n c t i o n values fj f{xj) e R . O u r g o a l is t o use a n R B F i n t e r p o l a n t
N
v x c x x
f( ) = Yl w(W - i\\)
3= 1
t o m a t c h these d a t a e x a c t l y , i.e., t o satisfy Vf(xi) = f(xi), i = 1 , . . . , N. The two
m o s t i m p o r t a n t questions n o w seem t o be:
W h i c h basic f u n c t i o n <p s h o u l d we use?

H o w s h o u l d we scale t h e basis f u n c t i o n s (fj </?(|| Xj\\)?
T h e e r r o r b o u n d s we r e v i e w e d i n p r e v i o u s c h a p t e r s give us some i n s i g h t i n t o
t h e first issue. I f we k n o w t h a t t h e d a t a come f r o m a v e r y s m o o t h f u n c t i o n , t h e n
a p p l i c a t i o n o f one o f t h e s m o o t h e r basic f u n c t i o n s is called for. O t h e r w i s e , t h e r e
is n o t m u c h t o be g a i n e d f r o m d o i n g so. I n fact, these f u n c t i o n s m a y a d d t o o
m u c h smoothness t o t h e i n t e r p o l a n t . A first a t t e m p t at p r o v i d i n g guidelines for t h e
selection o f a p p r o p r i a t e basic f u n c t i o n s (or kernels) c a n be f o u n d i n [Schaback a n d
W e n d l a n d (2006)]. W e w i l l n o t p u r s u e t h i s issue a n y f u r t h e r .
I n s t e a d we w a n t t o focus o u r a t t e n t i o n o n t h e second q u e s t i o n , i.e., t h e choice o f
t h e shape p a r a m e t e r e. A n u m b e r o f strategies c a n be used t o g u i d e us i n m a k i n g a
decision. W e w i l l assume t h r o u g h o u t t h a t a (fixed) basic f u n c t i o n has been chosen,
a n d t h a t we w i l l use o n l y one value t o scale a l l basis f u n c t i o n s u n i f o r m l y . Clearly,
one can also follow o t h e r strategies such as u s i n g a shape p a r a m e t e r t h a t varies
w i t h j , or even basic f u n c t i o n s t h a t v a r y w i t h j . W h i l e some w o r k has b e e n d o n e
i n these d i r e c t i o n s (see, e.g., [ B o z z i n i et al. ( 2 0 0 2 ) ; K a n s a a n d C a r l s o n (1992);
Schaback a n d W e n d l a n d ( 2 0 0 0 b ) ; F o r n b e r g a n d Z u e v ( 2 0 0 6 ) ] ) , n o t m u c h c o n c r e t e
c a n be said i n these cases.
141
W e n o w discuss four strategies for c h o o s i n g a " g o o d " value o f s.
17.1.1 Choosing a Good Shape Parameter via Trial and Error
T h e s i m p l e s t s t r a t e g y is t o p e r f o r m a series o f i n t e r p o l a t i o n e x p e r i m e n t s w i t h v a r y -
i n g shape p a r a m e t e r , a n d t h e n t o p i c k t h e "best" one. T h i s s t r a t e g y c a n be used i f
we k n o w t h e f u n c t i o n / t h a t g e n e r a t e d t h e d a t a , a n d t h e r e f o r e c a n c a l c u l a t e some
s o r t o f e r r o r for t h e i n t e r p o l a n t . O f course, i f we a l r e a d y k n o w / , t h e n t h e exercise
o f f i n d i n g a n i n t e r p o l a n t Vf m a y be m o s t l y p o i n t l e s s . H o w e v e r , t h i s is t h e s t r a t e g y
we used for t h e "academic" examples i n C h a p t e r 2.
I f we do n o t have a n y k n o w l e d g e o f / , t h e n i t becomes v e r y d i f f i c u l t t o decide
w h a t "best" means. O n e ( n o n - o p t i m a l ) c r i t e r i o n w e used i n C h a p t e r 2 was based
o n t h e t r a d e - o f f p r i n c i p l e , i.e., t h e fact t h a t for s m a l l e t h e e r r o r i m p r o v e s w h i l e
t h e c o n d i t i o n n u m b e r g r o w s . W e t h e n defined "best" t o be t h e smallest e for w h i c h
M A T L A B d i d n o t issue a n e a r - s i n g u l a r w a r n i n g .
I n m a n y cases selection o f a n o p t i m a l shape p a r a m e t e r v i a trial and error will
e n d u p b e i n g a r a t h e r s u b j e c t i v e process. H o w e v e r , t h i s m a y p r e s e n t l y be the
approach taken b y most practitioners.
For c o m p a r i s o n w i t h t h e o t h e r s e l e c t i o n m e t h o d s f e a t u r e d b e l o w we present t h r e e
o n e - d i m e n s i o n a l test cases for w h i c h w e k n o w t h e d a t a f u n c t i o n / . W e use
Fi(x) = sine (a;),
2 2 2 2
JP ( )2 X = | ^ -(9x-2) /4
e + e -(9x+l) /49^ + ^ ~(Qx-7) /4
e _ _Lg-(9x-4) ^
F (x)
3 = ( l - \x - \ \ j (\ + 5\x - \ \ - 27\x - i | 2 ) .
T h e first o f these f u n c t i o n s is t h e classical b a n d l i m i t e d f u n c t i o n ( a n d t h u s i n t h e

n a t i v e space o f Gaussians). T h e second f u n c t i o n is a o n e - d i m e n s i o n a l v a r i a n t o f
2
Franke's f u n c t i o n , a n d t h e t h i r d f u n c t i o n is one o f G n e i t i n g ' s C oscillatory com-
p a c t l y s u p p o r t e d R B F s s h i f t e d t o t h e p o i n t ( 1 / 2 , 1 / 2 ) (see T a b l e 11.4).
For these f u n c t i o n s w e l i s t m a x i m u m e r r o r s a n d o p t i m a l shape p a r a m e t e r s e i n
T a b l e 1 7 . 1 . M a x i m u m e r r o r s for a l a r g e r a n g e o f e values a n d t h e different values
o f N used i n T a b l e 17.1 are d i s p l a y e d i n F i g u r e 1 7 . 1 . T h e o p t i m a l e values l i s t e d
i n T a b l e 17.1 corresponds t o t h e lowest p o i n t for each o f t h e curves i n t h e figure.
Clearly, t h e o p t i m a l value o f t h e shape p a r a m e t e r is s t r o n g l y d e p e n d e n t on the
function t h a t generated the test data.
17.1.2 The Power Function as Indicator for a Good Shape

Parameter
A n o t h e r s t r a t e g y is suggested b y t h e e r r o r a n a l y s i s o f C h a p t e r 14. W e s h o w e d t h e r e
i n T h e o r e m 14.2 t h a t
\f(x)-Vf{x)\<P* (x)\\f\\ ,
tX Wn)
17. Numerical Evidence for Approximation Order Results 143
Table 17.1 Optimal e values based on Gaussian interpolation to various test func-
tions in I D for various choices of N uniform points.
Fi F 2 F 3
N max-error optimal max-error optimal e max-error optimal e
3 2.1403e-003 1.12 4.9722e-001 2.20 3.7087e-002 5.68

5 2.3260e-005 0.68 6.9380e-002 5.44 2.5253e-002 5.20
9 4.8764e-009 0.64 1.7555e-002 5.20 2.5360e-003 8.84
17 1.8922e-010 1.00 2.1928e-004 5.80 1.4380e-003 9.52
33 1.5250e-010 2.04 1.6536e-007 6.08 3.4189e-004 13.24
65 4.1307e-010 2.04 3.6260e-009 7.48 8.6431e-005 21.84
Fig. 17.1 Optimal e curves based on Gaussian interpolation in I D for various choices of N uniform
2
points. Data sampled from sine function Fx (left) and C oscillatory function F3 (right).
where P,x denotes t h e p o w e r f u n c t i o n . T h i s e s t i m a t e decouples t h e i n t e r p o l a t i o n

e r r o r i n t o a c o m p o n e n t i n d e p e n d e n t o f t h e d a t a f u n c t i o n / a n d one d e p e n d i n g o n
/. Once we have decided o n a basic f u n c t i o n <> a n d a d a t a set X we c a n use t h e
power f u n c t i o n based o n scaled versions o f <& t o o p t i m i z e t h e e r r o r c o m p o n e n t t h a t
is i n d e p e n d e n t o f / . ' W h i l e t h i s a p p r o a c h has t h e a d v a n t a g e over t h e p r e v i o u s l y
m e n t i o n e d t r i a l a n d e r r o r a p p r o a c h t h a t i t is o b j e c t i v e a n d does n o t d e p e n d o n
a n y k n o w l e d g e o f t h e d a t a f u n c t i o n , u n f o r t u n a t e l y , t h i s a p p r o a c h w i l l n o t be a n
o p t i m a l one since t h e second c o m p o n e n t o f t h e e r r o r b o u n d also d e p e n d s o n t h e
basic f u n c t i o n v i a t h e n a t i v e space n o r m ( w h i c h changes w h e n is scaled).
W e said earlier (see ( 1 4 . 4 ) ) t h a t t h e p o w e r f u n c t i o n c a n be c o m p u t e d v i a
1
P*,x{x) = ^ ( x , ^ ) - ^ ) ) ^ - ^ ) ,
T
where A is t h e i n t e r p o l a t i o n m a t r i x a n d 6 = [ $ ( - , x i ) , . . . , $ ( - , C j v ) ] . T h i s f o r m u l a
is i m p l e m e n t e d o n lines 1 5 - 1 8 i n t h e M A T L A B p r o g r a m P o w e r f u n c t i o n 2 D .m. W e
c o m p u t e t h e inverse o f A u s i n g t h e f u n c t i o n p i n v w h i c h is based o n t h e s i n g u l a r
value d e c o m p o s i t i o n o f A a n d t h e r e f o r e g u a r a n t e e s m a x i m u m s t a b i l i t y . A l s o , due t o
r o u n d o f f some o f t h e a r g u m e n t s o f t h e s q r t f u n c t i o n o n l i n e 18 come o u t n e g a t i v e .
This explains the use of the r e a l command. The vectors b(x) are just rows of the
evaluation matrix if x is taken from the grid of evaluation points we used earlier
for error computations and plotting purposes. Except for the loop over the shape
parameter e (lines 12-20) the rest of the program is similar to earlier code.
P r o g r a m 1 7 . 1 . Powerfunction2D.m
% Powerfunction2D
S c r i p t t h a t f i n d s "optimal" shape parameter by computing t h e power
'/, f u n c t i o n f o r the 2D RBF i n t e r p o l a t i o n approach w i t h v a r y i n g e p s i l o n
/, C a l l s on: DistanceMatrix
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; 7. Define t h e Gaussian RBF
% Parameters f o r shape parameter loop below
2 mine = 0; maxe = 20;
3 ne = 500; ep = linspace(mine,maxe,ne);
'/, Number and type of data p o i n t s
4 N = 81; g r i d t y p e = 'u';
7, R e s o l u t i o n of g r i d f o r power f u n c t i o n norm computation
5 n e v a l = 2 0 ; M = neval"2;
% Load data p o i n t s
, ,
6 name = s p r i n t f ( Data2D_7od7oS ,N,gridtype) ; load(name)
7 ctrs = dsites; % c e n t e r s c o i n c i d e w i t h data s i t e s
8 g r i d = l i n s p a c e ( 0 , 1 , n e v a l ) ; [xe,ye] = m e s h g r i d ( g r i d ) ;
9 epoints = [ x e ( : ) y e ( : ) ] ;
7. Compute d i s t a n c e m a t r i x between t h e data s i t e s and c e n t e r s
12 for i=l:length(ep)
7, Compute i n t e r p o l a t i o n m a t r i x
13 IM = r b f ( e p ( i ) , D M _ d a t a ) ;
7o Compute e v a l u a t i o n matrix
14 EM = r b f ( e p ( i ) , D M _ e v a l ) ;
7o Compute power f u n c t i o n a t e v a l u a t i o n p o i n t s
15 invIM = i n v ( I M ) ; phiO = r b f ( e p ( i ) , 0 ) ;
16 f o r j=l:M
17 powfun(j) = r e a l ( s q r t ( p h i O - ( i n v I M * E M ( j , : ) ' ) ' * E M ( j , : ) ' ) ) ;
18 end
7o Compute max. norm of power f u n c t i o n on e v a l u a t i o n g r i d
19 maxPF(i) = max(powfun);
20 end
21 f p r i n t f (' S m a l l e s t maximum norm: 7oe\n' , min(maxPF))
22 f p r i n t f ('at e p s i l o n = 7.f \n' , ep(maxPF==min(maxPF) ) )
&4
23 f p r i n t f ( ' w i t h cond (A) = 7 e \ n ' , ...

condest(rbf(ep(find(maxPF==min(maxPF))),DM_data)))
7, P l o t p o w e r f u n c t i o n n o r m
24 figure; semilogy(ep,maxPF,'b');
I n F i g u r e 17.2 w e show p l o t s o f t h e m a x i m u m n o r m s o f t h e p o w e r function

vs. e for a o n e - d i m e n s i o n a l experiment (left) a n d a 2 D e x p e r i m e n t (right). Each
p l o t shows several curves c o r r e s p o n d i n g t o different choices o f N (set o n l i n e 4 o f
P o w e r f u n t i o n 2 D . m ) . T h e o p t i m a l e values a l o n g w i t h t h e c o r r e s p o n d i n g c o n d i t i o n
numbers of the interpolation m a t r i x (computed using the c o n d e s t command) are
l i s t e d i n T a b l e 17.2. T h e g r a p h s o f t h e m a x i m u m n o r m o f t h e p o w e r f u n c t i o n c a n
a l l be i n c l u d e d i n a single p l o t b y a d d i n g a n o t h e r l o o p t o t h e p r o g r a m w h i c h varies
N. T h e p r o g r a m for t h e o n e - d i m e n s i o n a l case is a l m o s t i d e n t i c a l t o t h e one p r i n t e d
a n d therefore o m i t t e d .
0 5 10 15 20 0 5 10 15 20
e e
Fig. 17.2 Optimal e curves based on power functions for Gaussians in I D (left) and 2D (right)
for various choices of N uniform points.
Clearly, even for t h e s m a l l d a t a sets considered here, t h e n u m e r i c a l i n s t a b i l i t y ,

i.e., large c o n d i t i o n n u m b e r o f t h e i n t e r p o l a t i o n m a t r i x A, plays a s i g n i f i c a n t r o l e .
Table 17.2 Optimal e values based on power functions for Gaus-

sians in I D and 2D for various choices of N uniform points.
ID 2D
N optimal e cond(A) N optimal e cond(A)
3 0.04 1.8749e+007 9 0.16 5.3534e+009

5 0.44 5.7658e+007 25 0.84 1.0211e+011
9 1.72 6.5682e+008 81 0.04 2.0734e+019
17 4.48 6.1306e+009 289 0.56 1.2194e+020
33 9.60 5.4579e+010
65 19.52 1.2440e+011
17.1.3 Choosing a Good Shape Parameter via Cross Validation
A t h i r d s t r a t e g y for f i n d i n g a n " o p t i m a l " shape p a r a m e t e r is t o use a cross valida-

tion a p p r o a c h . I n [ R i p p a (1999)] a n a l g o r i t h m is d e s c r i b e d t h a t corresponds t o a
v a r i a n t o f cross v a l i d a t i o n k n o w n as "leave-one-out" cross validation. This method
is r a t h e r p o p u l a r i n t h e statistics l i t e r a t u r e w h e r e i t is also k n o w n as P R E S S (Pre-
d i c t i v e R E s i d u a l S u m o f Squares) p r o v i d e d t h e 2 - n o r m is used. I n t h i s a l g o r i t h m
a n " o p t i m a l " value o f e is selected b y m i n i m i z i n g t h e (least squares) e r r o r for a f i t
t o t h e d a t a based o n an i n t e r p o l a n t for w h i c h one o f t h e centers was "left o u t " . A
m a j o r advantage over t h e p r e v i o u s m e t h o d is t h a t n o w t h e dependence o f t h e e r r o r
on t h e d a t a f u n c t i o n is also t a k e n i n t o a c c o u n t . T h e r e f o r e , t h e p r e d i c t e d " o p t i m a l "
shape p a r a m e t e r is closer t o t h e one we f o u n d v i a t h e t r i a l a n d e r r o r a p p r o a c h (for
w h i c h we h a d t o assume k n o w l e d g e o f t h e exact s o l u t i o n ) .
A s i m i l a r s t r a t e g y was p r o p o s e d earlier i n [ G o l b e r g et al. (1996)] for t h e s o l u t i o n
of e l l i p t i c p a r t i a l differential e q u a t i o n s v i a t h e d u a l r e c i p r o c i t y m e t h o d based o n
multiquadric interpolation.
Specifically, i f Vf is t h e r a d i a l basis function interpolant to the data
{fi, , fk-i, fk+i, , / A T } , i-e-,
p}k*) = i>?V(ii*-*jii),
cf
such t h a t
Vfixi) = f it i = 1 , . . . , k - 1, k + 1 , . . . , N,
a n d i f Ek is t h e e r r o r
E k = fk-Vf{x )
k
at t h e one p o i n t Xk n o t used t o d e t e r m i n e t h e i n t e r p o l a n t , t h e n t h e q u a l i t y o f t h e
T
fit is d e t e r m i n e d b y t h e n o r m o f t h e v e c t o r o f errors E \E\,..., Ejy} obtained
by r e m o v i n g i n t u r n one o f t h e d a t a p o i n t s a n d c o m p a r i n g t h e r e s u l t i n g f i t w i t h
t h e ( k n o w n ) value at t h e r e m o v e d p o i n t . I n [ R i p p a (1999)] t h e a u t h o r presented
examples based o n use o f t h e l\ and i 2 n o r m s . W e w i l l m o s t l y use t h e m a x i m u m
n o r m (see line 14 i n t h e code b e l o w ) .
B y a d d i n g a l o o p over e we c a n c o m p a r e t h e e r r o r n o r m s for different values o f
t h e shape p a r a m e t e r , a n d choose t h a t value o f e t h a t y i e l d s t h e m i n i m a l e r r o r n o r m
as t h e o p t i m a l one.
W h i l e a naive i m p l e m e n t a t i o n o f t h e leave-one-out a l g o r i t h m is r a t h e r expensive
4
(on t h e order o f N o p e r a t i o n s ) , R i p p a showed t h a t t h e c o m p u t a t i o n o f t h e e r r o r
c o m p o n e n t s can be s i m p l i f i e d t o a single f o r m u l a
Ek = (17-1)
A
kk
w h e r e Ck is t h e A;th coefficient i n t h e i n t e r p o l a n t Vf based o n t h e full d a t a set, a n d
is t h e kth d i a g o n a l element o f t h e inverse o f t h e c o r r e s p o n d i n g i n t e r p o l a t i o n
m a t r i x . Since b o t h Cfc a n d A need t o be c o m p u t e d o n l y once for each value o f s

3
t h i s results i n 0(N ) c o m p u t a t i o n a l c o m p l e x i t y . N o t e t h a t a l l entries i n t h e e r r o r
vector E c a n be c o m p u t e d i n a single s t a t e m e n t i n M A T L A B i f we v e c t o r i z e t h e
c o m p o n e n t f o r m u l a (17.1) (see l i n e 13 i n P r o g r a m 17.2). T h e sine f u n c t i o n used
on line 5 is n o t a s t a n d a r d M A T L A B f u n c t i o n ( i t is p a r t o f t h e S i g n a l Processing
T o o l b o x ) . T h e r e f o r e we p r o v i d e i t i n P r o g r a m C.2 i n A p p e n d i x C.
P r o g r a m 1 7 . 2 . L00CV2D.m
% L00CV2D
% S c r i p t t h a t performs leave-one-out c r o s s - v a l i d a t i o n
/ (Rippa's method) t o f i n d a good e p s i l o n f o r 2D RBF i n t e r p o l a t i o n
0
1 r b f = @(e,r) e x p ( - ( e * r ) . " 2 ) ; % Gaussian RBF
% Parameters f o r shape parameter loop below
2 mine = 0; maxe = 20; ne = 500;
3 ep = linspace(mine,maxe,ne);
% Number and type of d a t a p o i n t s
4 N = 81; gridtype = 'u';
7, Define t e s t f u n c t i o n
5 t e s t f u n c t i o n = @(x,y) s i n e ( x ) . * s i n c ( y ) ;
,
6 name = s p r i n t f ( Data2D_/ d%s',N,gridtype) ; load(name)
0
7 ctrs = dsites; '/ c e n t e r s c o i n c i d e w i t h data s i t e s

7, Create r i g h t - h a n d s i d e v e c t o r , i . e . ,
7o evaluate t h e t e s t f u n c t i o n a t t h e d a t a p o i n t s .
8 rhs = testfunction(dsites(:,1),dsites(:,2));
'/, Compute d i s t a n c e m a t r i x between t h e data s i t e s and c e n t e r s
10 f o r i = l : l e n g t h ( e p )
7o Compute i n t e r p o l a t i o n m a t r i x
11 IM = r b f ( e p ( i ) , D M _ d a t a ) ;
% Compute e r r o r f u n c t i o n ( i . e . , " c o s t " of e p s i l o n )
12 invIM = p i n v ( I M ) ;
13 EF = ( i n v I M * r h s ) . / d i a g ( i n v I M ) ;
7 Compute maximum norm of E F
14 maxEF(i) = n o r m ( E F ( : ) , i n f ) ;
15 end
16 f p r i n t f ('Smallest maximum norm: 7 e\n' , min(maxEF)) 0
17 f p r i n t f ( a t e p s i l o n = 7f \n',ep(maxEF==min(maxEF)) )
J
/ P l o t cost f u n c t i o n norm
0
18 f i g u r e ; semilogy(ep,maxEF,'b');
I n F i g u r e 17.3 w e show p l o t s o f t h e p r e d i c t e d m a x i m u m e r r o r s vs. e for a one-

d i m e n s i o n a l e x p e r i m e n t (left) a n d a 2 D e x p e r i m e n t ( r i g h t ) based o n d a t a s a m p l e d
f r o m t h e sine f u n c t i o n F\. E a c h p l o t shows several curves c o r r e s p o n d i n g t o different
choices o f N (set o n l i n e 4 o f L00CV2D.m). T h e o p t i m a l e values are l i s t e d i n
T a b l e 17.3.
0 5 10 15 20 0 5 10 15 20
e e
Fig. 17.3 Optimal e curves based on leave-one-out cross validation for interpolation to the sine
function with Gaussians in I D (left) and 2D (right) for various choices of N uniform points.
Table 17.3 Optimal e values based

on leave-one-out cross validation for
interpolation to the sine function
with Gaussians in I D and 2D for
various choices of N uniform points.
ID 2D
N optimal e N optimal
3 0.96 9 0.96
5 1.00 25 1.00
9 0.80 81 1.48
17 0.92 289 1.60
33 1.92
65 1.76
T h e graphs i n F i g u r e 17.4 s h o w side-by-side t h e o p t i m a l s curves for t h e t r i a l a n d

e r r o r a p p r o a c h a n d for t h e leave-one-out cross v a l i d a t i o n a p p r o a c h i n t h e case o f I D
Gaussian i n t e r p o l a t i o n t o d a t a sampled f r o m the test function F - 2 The similarity
o f t h e curves is c l e a r l y a p p a r e n t . T h u s , t h e leave-one-out cross v a l i d a t i o n a p p r o a c h
c a n be r e c o m m e n d e d as a g o o d m e t h o d for s e l e c t i n g a n " o p t i m a l " shape p a r a m e t e r
e since for t h i s m e t h o d n o k n o w l e d g e o f t h e e x a c t e r r o r is needed. A n o t h e r p a i r o f
c o m p a r i s o n p l o t s is g i v e n b y t h e G a u s s i a n i n t e r p o l a n t s t o t h e sine f u n c t i o n F\ i n
F i g u r e 17.1 (left) a n d F i g u r e 17.3 ( l e f t ) .
S i m i l a r conclusions h o l d for o t h e r basic f u n c t i o n s , o t h e r test f u n c t i o n s , o t h e r
d a t a d i s t r i b u t i o n s , a n d o t h e r space d i m e n s i o n s . F o r e x a m p l e , F i g u r e 17.5 shows t h e
10 10"
W -5 Of "
10
10 5
10" 10"
10 15 20 10 15 20
e
Fig. 17.4 Optimal e curves based on interpolation to the test function with Gaussians for
various choices of N uniform points. Trial and error approach (left), leave-one-out cross validation
(right).
o p t i m a l e curves for i n t e r p o l a t i o n t o t h e I D F r a n k e f u n c t i o n F 2 with Wendland's

2
C f u n c t i o n (ps i o n u n i f o r m l y spaced p o i n t s , a n d o n C h e b y s h e v p o i n t s . N o t e t h a t
;
for t h i s c o n f i g u r a t i o n a l l c o m p u t a t i o n s are stable, a n d t h e o p t i m a l scale p a r a m e t e r

is q u i t e s m a l l , i.e., t h e s u p p o r t r a d i u s o f t h e c o m p a c t l y s u p p o r t e d basic f u n c t i o n is
chosen t o be v e r y large. I n o t h e r w o r d s , t h e best results for c o m p a c t l y s u p p o r t e d
f u n c t i o n s are o b t a i n e d w i t h dense m a t r i c e s .
10
10
10 y /
f 4
/ 4*
10" _ // * N=3
* N=5
N=9
10 t N=17
N=33
N=65
10
10 15 20

Fig. 17.5 Optimal e curves based on leave-one-out cross validation for interpolation to I D Franke's
function with Wendland's function 953,1 for various choices of N uniform points (left) and Cheby-
shev points (right).
I f we are n o t i n t e r e s t e d i n t h e e-curves d i s p l a y e d above, b u t o n l y w a n t t o find

a g o o d value o f t h e shape p a r a m e t e r as q u i c k l y as possible, t h e n we c a n use the
M A T L A B f u n c t i o n f m i n b n d t o find t h e m i n i m u m o f t h e cost f u n c t i o n for e. First,
we i m p l e m e n t t h e cost f u n c t i o n i n t h e s u b r o u t i n e C o s t E p s i l o n . m d i s p l a y e d i n P r o -
g r a m 17.3. T h e c o m m a n d s are t h e same as those o n lines 1 1 - 1 4 i n P r o g r a m 17.2.
P r o g r a m 17.3. CostEpsilon.m
7. ceps = C o s t E p s i l o n ( e p , r , r b f , r h s )
% Implements cost f u n c t i o n f o r o p t i m i z a t i o n of shape parameter
7, v i a Rippa's L00CV algorithm
% Example of usage i n L00CV2Dmin.m
1 f u n c t i o n ceps = C o s t E p s i l o n ( e p , r , r b f , r h s )
2 A = rbf(ep,r);
3 invA = p i n v ( A ) ;
4 EF = ( i n v A * r h s ) . / d i a g ( i n v A ) ;
5 ceps = n o r m ( E F ( : ) , i n f ) ;
In order to demonstrate the use of the C o s t E p s i l o n function we use a modifi-

cation of Program 17.2 which we list as Program 17.4.
P r o g r a m 17.4. L00CV2Dmin.m
'/. L00CV2Dmin
% S c r i p t t h a t performs leave-one-out c r o s s - v a l i d a t i o n
7 5
( R i p p a s method) to f i n d a good e p s i l o n f o r 2D RBF i n t e r p o l a t i o n
7.with the help of MATLAB' s fminbnd
7,Requires: CostEpsilon
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; 7o Gaussian RBF
7o Parameters f o r shape parameter o p t i m i z a t i o n below
2 mine = 0; maxe = 20;
7o Number and type of data p o i n t s
3 N = 81; g r i d t y p e = 'u';
7o Define t e s t f u n c t i o n
7o Load data p o i n t s
5 name = s p r i n t f ( 'Data2D_7od7 s ' ,N,gridtype) ; load(name)
0
6 ctrs = dsites; % centers coincide with data s i t e s

7 Create right-hand s i d e v e c t o r , i . e . ,
7o e v a l u a t e the t e s t f u n c t i o n a t the d a t a p o i n t s .
7 rhs = t e s t f u n c t i o n ( d s i t e s ( : , 1 ) , d s i t e s ( : , 2 ) ) ;
7o Compute d i s t a n c e matrix between the d a t a s i t e s and c e n t e r s
9a [ e p , f v a l ] = fminbnd(@(ep) CostEpsilon(ep,DM_data,rbf,rhs),...
9b mine,maxe);
10 f p r i n t f ('Smallest maximum norm: 7oe\n', f v a l )
11 f p r i n t f ('at e p s i l o n = 7of\n' , ep)
17.1.4 The Contour-Pade Algorithm
The Contour-Pade a l g o r i t h m was t h e s u b j e c t o f G r a d y W r i g h t ' s P h . D . thesis

[ W r i g h t (2003)] a n d was r e p o r t e d i n [ F o r n b e r g a n d W r i g h t ( 2 0 0 4 ) ] . T h e a i m o f
t h e C o n t o u r - P a d e a l g o r i t h m is t o come u p w i t h a m e t h o d t h a t allows t h e c o m p u t a -
t i o n a n d e v a l u a t i o n o f R B F i n t e r p o l a n t s for i n f i n i t e l y s m o o t h basic f u n c t i o n s w h e n
t h e shape p a r a m e t e r e tends t o zero ( i n c l u d i n g t h e l i m i t i n g case).
T h e s t a r t i n g p o i n t is t o consider e v a l u a t i o n o f t h e R B F i n t e r p o l a n t
N
'Pf{x,e) = ^2c (p (\\x-x \\)
j e j
j=i
for a fixed e v a l u a t i o n p o i n t x as a n a n a l y t i c f u n c t i o n o f e.
T h e key idea is t o represent Vf(x, e) b y a Laurent series i n e, a n d a p p r o x i m a t e
t h e "negative p a r t " o f t h e series b y a P a d e a p p r o x i m a n t , i.e.,
oo
d ek
V (x,e)
f ^r(e) + J2 k >
k=0
where r(e) is t h e r a t i o n a l P a d e a p p r o x i m a n t .
We t h e n r e w r i t e t h e i n t e r p o l a n t i n c a r d i n a l f o r m , i.e., as
Vf{x,e) = ^C V {\\X
3 - Xj\\)
j=i
T
= b (x, e)c
T
= b {x,e)A-\e)f
= (u*(x,e)ff
T
where b(x,e) - 3 = ip (\\x
- xj\\), A(e)ij = <p (\\xi
- x - \ \ ) , c = [a,...,
3 c],
N f =
T
[fi,-- -,fN] , and
_1
u*(x,e) = A ()6(a;,)
denotes t h e vector o f values o f t h e c a r d i n a l f u n c t i o n s at x (c.f. C h a p t e r 14).

I t is n o w t h e g o a l t o s t a b l y c o m p u t e t h e v e c t o r u* (e) for a l l values o f e > 0 w i t h -
1
o u t e x p l i c i t l y f o r m i n g t h e inverse A(e)~ and w i t h o u t c o m p u t i n g the m a t r i x vector
1
p r o d u c t A(e)~ b(e). Here t h e v e c t o r s u*(e) a n d b(s) are o b t a i n e d b y e v a l u a t i n g
t h e vector f u n c t i o n s u*(-,e) a n d b(-,e) on an appropriate evaluation grid.
T h e s o l u t i o n p r o p o s e d b y W r i g h t a n d F o r n b e r g is t o use C a u c h y ' s i n t e g r a l t h e -
o r e m t o i n t e g r a t e a r o u n d a c i r c l e i n t h e c o m p l e x e-plane. T h e residuals (i.e., co-
efficients i n t h e L a u r e n t e x p a n s i o n ) are o b t a i n e d u s i n g t h e (inverse) fast F o u r i e r
transform. T h e t e r m s w i t h n e g a t i v e powers o f e are t h e n a p p r o x i m a t e d u s i n g a
r a t i o n a l Pade a p p r o x i m a n t . T h e i n t e g r a t i o n c o n t o u r ( u s u a l l y a circle) has t o l i e
between t h e r e g i o n o f i n s t a b i l i t y near e = 0 a n d possible b r a n c h p o i n t s i n g u l a r i t i e s
t h a t lie somewhere i n t h e c o m p l e x p l a n e d e p e n d i n g o n t h e choice o f ip. D e t a i l s o f
t h e m e t h o d can be f o u n d i n [ F o r n b e r g a n d W r i g h t (2004)].
I n F i g u r e 17.6 w e show o p t i m a l e curves for i n t e r p o l a t i o n t o t h e I D a n d 2 D

sine f u n c t i o n F 2 u s i n g Gaussians a t e q u a l l y spaced p o i n t s . T h e s e curves s h o u l d be
c o m p a r e d w i t h t h e o p t i m a l e curves o b t a i n e d for t h e same p r o b l e m v i a t r i a l a n d
e r r o r (see F i g u r e 17.1 a n d T a b l e 17.1) a n d v i a leave-one-out cross v a l i d a t i o n (see
F i g u r e 17.3 a n d T a b l e 17.3).
T h e m a i n d r a w b a c k o f t h e C o n t o u r - P a d e a l g o r i t h m is t h e fact t h a t i f N becomes
t o o large t h e n t h e r e g i o n o f i l l - c o n d i t i o n i n g a r o u n d t h e o r i g i n i n t h e c o m p l e x e-
p l a n e a n d t h e b r a n c h p o i n t s i n g u l a r i t i e s w i l l o v e r l a p . T h i s , however, i m p l i e s t h a t
t h e m e t h o d c a n o n l y be used w i t h l i m i t e d success. M o r e o v e r , as t h e g r a p h s i n
F i g u r e 17.6 a n d t h e entries i n T a b l e 17.4 show, t h e value o f N t h a t has t o be
considered "large" is u n f o r t u n a t e l y r a t h e r s m a l l . F o r t h e o n e - d i m e n s i o n a l case t h e
results for N = 17 a l r e a d y are affected b y i n s t a b i l i t i e s , a n d i n t h e t w o - d i m e n s i o n a l
e x p e r i m e n t N = 8 1 causes p r o b l e m s .
0 5 10 15 20 0 5 10 15 20
E E
Fig. 17.6 Optimal e curves based on Contour-Pade for interpolation to the sine function with
Gaussians in I D (left) and 2D (right) for various choices of N uniform points.
Table 17.4 Optimal e values based on Contour-Pade for interpolation to the sine
function with Gaussians in I D and 2 D for various choices of N uniform points.
ID 2D
N max-error cond(A) N max-error cond(A)
3 1.7605e-003 1.10 3.3386e+001 9 3.3875e-003 1.10 1.1146e+003

5 4.0380e-005 0.70 1.3852e+006 25 5.5542e-005 0.70 1.9187e+012
9 3.9703e-009 0.45 7.7731e+016 81 3.6528e-004 0.00 oo
17 1.2726e-009 0.45 1.7327e+018
17.1.5 Summary
A l l strategies p u r s u e d i n t h i s c h a p t e r have s h o w n t h a t even t h o u g h t h e bound

(15.11) b y M a d y c h seems t o i n d i c a t e t h a t t h e i n t e r p o l a t i o n e r r o r for f u n c t i o n s i n
t h e n a t i v e space o f t h e basic f u n c t i o n goes t o zero e x p o n e n t i a l l y as e 0, t h i s

does n o t seem t o be t r u e i n p r a c t i c e . E s p e c i a l l y those o p t i m a l e c u r v e s t h a t w e r e
c o m p u t e d r e l i a b l y w i t h t h e C o n t o u r - P a d e a l g o r i t h m a l l have a g l o b a l m i n i m u m for
some p o s i t i v e value o f e. I n m a n y cases t h i s o p t i m a l s v a l u e ( o r a n e close t o t h e
o p t i m a l v a l u e ) c a n be f o u n d u s i n g t h e leave-one-out cross v a l i d a t i o n a l g o r i t h m o f
P r o g r a m 17.2. F r o m n o w o n we w i l l f r e q u e n t l y use leave-one-out cross v a l i d a t i o n
t o find a n o p t i m a l shape p a r a m e t e r f o r o u r n u m e r i c a l e x p e r i m e n t s .
17.2 Non-stationary Interpolation
I n o r d e r t o i l l u s t r a t e t h e s p e c t r a l convergence p r e d i c t e d for i n f i n i t e l y s m o o t h basic

f u n c t i o n s such as Gaussians a n d g e n e r a l i z e d (inverse) m u l t i q u a d r i c s w e need t o w o r k
i n a s e t t i n g for w h i c h n e i t h e r t h e i n s t a b i l i t y due t o large p r o b l e m size o r s m a l l shape
p a r a m e t e r have a s i g n i f i c a n t effect o n o u r e x p e r i m e n t s . O t h e r w i s e , i f w e s i m p l y t a k e
a n " o p t i m a l " value o f e ( d e t e r m i n e d v i a t r i a l a n d e r r o r for a l a r g e N = 4225 p r o b l e m
i n t h e " g r a y zone", c.f. C h a p t e r 2) t h e n t h e s p e c t r a l convergence w i l l o n l y be v i s i b l e
for a l i m i t e d n u m b e r o f e x p e r i m e n t s (see T a b l e 17.5).
Table 17.5 2D non-stationary interpolation (e = 6.3) to

Franke's function with Gaussians on uniformly spaced and
Halton points.
uniform Halton
N RMS-error rate RMS-error rate
9 3.195983e-001 2.734756e-001
25 5.008591e-002 2.6738 8.831682e-002 2.3004
81 9.029664e-003 2.4717 2.401868e-002 1.7582
289 2.263880e-004 5.3178 1.589117e-003 5.0969
1089 3.323287e-008 12.7339 1.595051e-006 10.8015
4225 1.868286e-008 0.8309 9.510404e-008 4.8203
E v e n for a b a n d - l i m i t e d f u n c t i o n (see T a b l e 17.6) t h e s i t u a t i o n is n o t b e t t e r ; i n

fact worse, for t h e v a l u e o f e used.
I n F i g u r e s 17.7-17.8 we are able t o v e r i f y ( a t least t o some e x t e n t ) t h e c o n v e r -
gence estimates for n o n - s t a t i o n a r y R B F i n t e r p o l a n t s . W e o b t a i n t h e d a t a for a l l
e x p e r i m e n t s b y s a m p l i n g t h e sine f u n c t i o n f(x) = sin(7rx)/(7rx) at N uniformly
spaced p o i n t s i n t h e i n t e r v a l [ 0 , 1 ] w h e r e N r u n s f r o m 1 t o 100. E a c h p l o t shows s i x
m a x i m u m e r r o r curves ( c o r r e s p o n d i n g t o shape p a r a m e t e r s e 1 , 6 , 1 1 , 1 6 , 2 1 , 2 6 )
versus t h e n u m b e r N o f d a t a p o i n t s o n a l o g l o g scale. T h e e r r o r s are e v a l u a t e d o n a
g r i d o f 250 e q u a l l y spaced p o i n t s . I n o r d e r t o c o m p a r e these curves w i t h t h e t h e o -
r e t i c a l b o u n d s f r o m C h a p t e r 15 w e have p l o t t e d c o m p a r i s o n curves c o r r e s p o n d i n g t o
t h e t h e o r e t i c a l b o u n d s . F o r Gaussians t h e c o m p a r i s o n c u r v e is g i v e n b y t h e g r a p h
l h h
o f h i c ^ z \/ c o r r e s p o n d i n g t o s u p e r - s p e c t r a l convergence w i t h h = l/(N 1),
Table 17.6 2D non-stationary interpolation (e = 6.3) to the sine

function with Gaussians on uniformly spaced and Halton points.
uniform Halton
9 3.302644e-001 2.823150e-001
25 3.271035e-002 3.3358 1.282572e-001 1.6058
81 1.293184e-002 1.3388 3.407580e-002 1.7898
289 3.786113e-004 5.0941 1.990217e-003 5.3309
1089 3.476835e-008 13.4107 2.286014e-006 10.5905
4225 3.775365e-008 -0.1188 9.868530e-008 5.3724
Fig. 17.7 Maximum errors for non-stationary interpolation to the sine function with Gaussians
(left) and inverse multiquadrics (right) based on N uniformly spaced points in [0,1] and e
1,6,11,16,21,26.
1 / / / l
a n d for inverse m u l t i q u a d r i c s we have s p e c t r a l convergence w i t h h i e ~ . We
c a n see t h a t for a c e r t a i n range o f p r o b l e m s these r a t e s are i n d e e d o b t a i n e d (see
F i g u r e 17.7).
I n t h e case o f f u n c t i o n s w i t h f i n i t e s m o o t h n e s s (such as t h e c o m p a c t l y s u p -
p o r t e d f u n c t i o n s o f W e n d l a n d ) we c a n o n l y expect a l g e b r a i c convergence r a t e s .
F i g u r e 17.8 shows t w o m o r e sets o f m a x i m u m e r r o r curves. These plots are
2 6
based o n W e n d l a n d ' s C f u n c t i o n 1/23,1 ( r ) = ( 1 r ) + ( 4 r + 1) a n d t h e C func-
t i o n ^ 3 , 3 ( r ) = ( l - 7 - ) ^ ( 3 2 r + 2 5 r - r - 8 r + l ) . W h i l e t h e e r r o r b o u n d (15.5) p r e d i c t s
3 2
3 2 7 2
o n l y 0 ( / i / ) a n d G(h ^ ) a p p r o x i m a t i o n o r d e r , respectively. W e see t h a t a n e x t r a
s 2
factor o f h l is i n d e e d possible i n p r a c t i c e . T h i s e x t r a f a c t o r has also been c a p t u r e d
i n some o f t h e t h e o r e t i c a l w o r k o n i m p r o v e d e r r o r b o u n d s (c.f. S e c t i o n 15.2).
For less s m o o t h d a t a f u n c t i o n s we n o l o n g e r have s p e c t r a l convergence for
the infinitely s m o o t h functions, while the orders r e m a i n unchanged for t h e ba-
sic f u n c t i o n s w i t h f i n i t e s m o o t h n e s s (as l o n g as t h e d a t a f u n c t i o n lies i n the
n a t i v e space o f t h e basic f u n c t i o n ) . T h i s is i l l u s t r a t e d i n F i g u r e 17.9 where
2 2
we c o m p a r e Gaussians a n d C Wendland functions for t h e C test function
2
Fig. 17.8 Maximum errors for non-stationary interpolation to the sine function with C (left)
6
and C (right) Wendland function based on N uniformly spaced points in [0,1] and e =
1,6,11,16,21,26.
2
Fig. 17.9 Maximum errors for non-stationary interpolation to a C function with Gaussians
2
(left) and C Wendland function (right) based on N uniformly spaced points in [0,1] and e =
1,6,11,16,21,26.
2
(1 - \x - 1/2|)5.(1 + 5\x - 1 / 2 | - 27(x - 1 / 2 ) ) (c.f. the oscillatory functions o f
T a b l e 11.4). I t is i n t e r e s t i n g t o n o t e t h a t for a c e r t a i n range o f N the rate of
2
convergence for t h e C W e n d l a n d f u n c t i o n is even b e t t e r t h a n p r e d i c t e d .
17.3 Stationary Interpolation
W e b e g i n w i t h a n i l l u s t r a t i o n o f t h e fact t h a t for r a d i a l powers a n d t h i n plate

splines t h e r e is no difference i n convergence b e h a v i o r b e t w e e n t h e s t a t i o n a r y and
non-stationary regime. F i g u r e 17.10 shows t h i s p h e n o m e n o n for t h e n o r m r a d i a l
2
f u n c t i o n 5>(cc) = ||cc|| i n t h e case o f i n t e r p o l a t i o n t o d a t a s a m p l e d f r o m t h e C
3
f u n c t i o n f(x) = \x 1/21 a t u n i f o r m l y spaced p o i n t s i n [ 0 , 1 ] , M o r e o v e r , t h e left
p l o t i n F i g u r e 17.10 ( i l l u s t r a t i n g t h e n o n - s t a t i o n a r y s e t t i n g ) shows t h a t t h e shape
2
Fig. 17.10 Maximum errors for non-stationary (left) and stationary (right) interpolation to a C
function with the norm basic function based on N uniformly spaced points in [0,1].
p a r a m e t e r has no effect for t h e n o r m basic f u n c t i o n a n d o t h e r p o l y h a r m o n i c splines.

2
N o t e t h a t F i g u r e 17.10 suggests t h a t t h e n o r m basic f u n c t i o n has 0(h ) approx-
i m a t i o n order, w h i l e t h e b o u n d f r o m T h e o r e m 15.3 w i t h r = k = 0 1, n = 0,
1 2
s = 1 a n d q = oo y i e l d s o n l y OQi / ). Since t h e n o r m basic f u n c t i o n is s t r i c t l y
c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r one w e c a n use t h e same R B F expansion
as for s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s , i.e., w i t h o u t appending a constant (c.f.
T h e o r e m 9.7).
T h e discrepancy b e t w e e n t h e t h e o r e t i c a l b o u n d s o f T h e o r e m 15.3 (or T h e o -
r e m 15.4 as w e l l as t h e n a t i v e space b o u n d s o f E x a m p l e s 15.6 a n d 15.7 o f C h a p -
t e r 15) a n d those observed i n n u m e r i c a l e x p e r i m e n t s is s i m i l a r for r a d i a l cubics a n d
t h i n p l a t e splines ( w h i c h are b o t h s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e o f o r d e r
t w o ) . For cubics T h e o r e m 15.3 w i t h r = /3 = 3, k = 2, n = 0, s = 2 a n d q = oo
2
p r e d i c t s 0(h ) since t h e m e s h r a t i o p r o v i d e s a n o t h e r p o w e r o f h for u n i f o r m l y dis-
3
t r i b u t e d d a t a . T h e left p l o t o f F i g u r e 1 7 . 1 1 , however, suggests 0(h ) or better
2
a p p r o x i m a t i o n o r d e r based o n i n t e r p o l a t i o n t o t h e 2 D a n a l o g o f t h e o s c i l l a t o r y C
test f u n c t i o n F , 3 i.e., f(x) = ( 1 - \\x - ( 1 / 2 , l / 2 ) | | ) | j _ ( l + 5 | | x - ( 1 / 2 , 1 / 2 ) | | - 27\\x-
2 3 2
( 1 / 2 , 1 / 2 ) | | ) . T h e p r e d i c t e d r a t e for t h i n p l a t e splines is G(h / ) (since r = 20 = 2,
k = 2, n = 0, s 2 a n d q = oo) w h i l e t h e p l o t o n t h e r i g h t o f F i g u r e 17.11 i n d i c a t e s
2
a t least 0(h ) convergence.
For Gaussian basis f u n c t i o n s we n o t e d e a r l i e r t h a t w e s h o u l d n o t e x p e c t a n y c o n -
vergence i n t h e s t a t i o n a r y s e t t i n g . H o w e v e r , i f t h e i n i t i a l shape p a r a m e t e r is chosen
s m a l l e n o u g h ( b u t n o t t o o s m a l l ) , t h e n we c a n observe t h e a p p r o x i m a t e a p p r o x i m a -
t i o n p h e n o m e n o n , i.e., t h e r e is convergence u p t o a c e r t a i n p o i n t , a n d t h e n s a t u r a -
t i o n occurs. T h i s is d e p i c t e d i n F i g u r e 17.12. I n t h e left p l o t w e used t h e G a u s s i a n
basic f u n c t i o n w i t h different i n i t i a l shape p a r a m e t e r s (e = 0.8,1.0,1.2,1.4,1.6,1.8)
2
t o interpolate data sampled from the oscillatory C f u n c t i o n used i n t h e p r e v i o u s
i l l u s t r a t i o n at u n i f o r m l y spaced p o i n t s i n t h e u n i t square. The plot on the right
corresponds t o G a u s s i a n i n t e r p o l a t i o n o f d a t a s a m p l e d f r o m t h e 2 D sine f u n c t i o n
fix, y) = sinc(a;)sinc(y) w i t h i n i t i a l e = 0 . 1 , 0.2, 0.3, 0 . 4 , 0 . 5 , 0.6.
2
Fig. 17.11 Maximum errors for stationary interpolation to a C function with the cubic radial
basic function (left) and thin plate spline basic function (right) based on N uniformly spaced
2
points in [0, l ] .
2
Fig. 17.12 Maximum errors for stationary interpolation to the C oscillatory function (left) and
2
to the sine function (right) with Gaussians based on N uniformly spaced points in [0, l ] using
various initial e values.
If we consider the range of N values used in the experiments (N =

9, 2 5 , 8 1 , 2 8 9 , 1 0 8 9 , 4 2 2 5 ) , t h e n we see t h a t s t a t i o n a r y i n t e r p o l a t i o n w i t h Gaussians
2
does converge for t h e s m a l l e r values o f N ( a t a t r a t e b e t t e r t h a n 0(h )). However,
t h e larger t h e value o f t h e i n i t i a l e is t a k e n , t h e sooner does t h e s a t u r a t i o n occur.
I t is also a p p a r e n t t h a t i n t h e case o f i n t e r p o l a t i o n t o t h e sine f u n c t i o n s m a l l i n i -
t i a l values o f t h e shape p a r a m e t e r l e a d t o severe i l l - c o n d i t i o n i n g a n d s u b s e q u e n t
i n s t a b i l i t i e s especially for t h e t e s t s w i t h l a r g e r values o f N. W e also p o i n t o u t t h a t
t h e r a n g e o f values o f e for w h i c h w e c a n observe convergence d e p e n d s o n t h e d a t a
function / .
W e w i l l come back t o the a p p r o x i m a t e a p p r o x i m a t i o n p h e n o m e n o n i n the con-
t e x t o f q u a s i - i n t e r p o l a t i o n a n d a p p r o x i m a t e m o v i n g least squares a p p r o x i m a t i o n i n
C h a p t e r s 26 a n d 27.
I
I
I
1
g
I
Chapter 18
The Optimality of R B F Interpolation
I n t h i s c h a p t e r we w i l l see t h a t w i t h i n t h e n a t i v e H i l b e r t spaces associated w i t h

s t r i c t l y positive definite (and s t r i c t l y c o n d i t i o n a l l y positive definite) r a d i a l functions
t h e r a d i a l basis f u n c t i o n i n t e r p o l a n t p r o v i d e s t h e best approximation t o a given
d a t a f u n c t i o n . T h i s o p t i m a l i t y o f i n t e r p o l a n t s i n H i l b e r t space is t h e s u b j e c t o f t h e
t h e o r y o f optimal recovery d e s c r i b e d i n t h e l a t e 1950s b y M i c h a e l G o l o m b a n d H a n s
W e i n b e r g e r i n t h e i r p a p e r [ G o l o m b a n d W e i n b e r g e r (1959)].
18.1 The Connection to Optimal Recovery
I n [ G o l o m b a n d W e i n b e r g e r (1959)] t h e a u t h o r s s t u d i e d t h e f o l l o w i n g g e n e r a l p r o b -
lem:
Problem 1 8 . 1 . Given the values f\ = Xi(f), , / N = A

iv(/) where
{ A i , . . . , A J V } is a linearly independent set of linear functionals (called information
f u n c t i o n a l s yielding the i n f o r m a t i o n about f), how does one "best" approximate the
value A ( / ) (called a f e a t u r e of f) where X is a given linear functional and f is
unknown? Moreover, what is the total range of values for X(f)?
T h i s is a v e r y general p r o b l e m f o r m u l a t i o n t h a t allows n o t o n l y for i n t e r p o l a t i o n

of f u n c t i o n values, b u t also for o t h e r t y p e s o f d a t a (such as values o f d e r i v a t i v e s
a n d i n t e g r a l s o f / , such as averages or m o m e n t s o f / , e t c . ) , as w e l l as m e t h o d s o f
approximation other t h a n interpolation.
T h e k i n d o f p r o b l e m d e s c r i b e d a b o v e is k n o w n i n t h e l i t e r a t u r e as a n optimal
recovery problem. Besides t h e s e m i n a l w o r k b y G o l o m b a n d W e i n b e r g e r , optimal
recovery was also s t u d i e d i n d e t a i l b y M i c c h e l l i , R i v l i n a n d W i n o g r a d [ M i c c h e l l i et
al. (1976); M i c c h e l l i a n d R i v l i n ( 1 9 7 7 ) ; M i c c h e l l i a n d R i v l i n (1980); M i c c h e l l i a n d
R i v l i n (1985)].
I n a H i l b e r t space s e t t i n g t h e s o l u t i o n t o t h i s o p t i m a l recovery p r o b l e m is s h o w n
t o be t h e minimum-norm interpolant. M o r e precisely, g i v e n a H i l b e r t space 7i a n d
d a t a / i = A i ( / ) , . . . , / J V = A ^ ( / ) G R w i t h { A i , . . . , XN} C U * ( t h e d u a l o f W ) , t h e
159
m i n i m u m - n o r m i n t e r p o l a n t is t h a t f u n c t i o n g* G 7i t h a t satisfies
W ) = /;> j = h...,N,
a n d for w h i c h
\\g*\\n= min \\g\\ .

n
I t t u r n s o u t t h a t t h e r a d i a l basis f u n c t i o n i n t e r p o l a n t w i t h basic f u n c t i o n <]> satisfies

these c r i t e r i a i f 7i is t a k e n as t h e associated n a t i v e space Af<s>(Q).
W e w i l l present t h r e e o p t i m a l i t y results:
T h e r a d i a l basis f u n c t i o n i n t e r p o l a n t for a n y s t r i c t l y c o n d i t i o n a l l y p o s i t i v e def-

i n i t e f u n c t i o n $ is t h e m i n i m u m n o r m i n t e r p o l a n t f r o m j\f$(Q,).
T h e r a d i a l basis f u n c t i o n i n t e r p o l a n t p r o v i d e s t h e best a p p r o x i m a t i o n t o / i n
t h e n a t i v e space n o r m .
T h e ( c a r d i n a l f o r m o f t h e ) r a d i a l basis f u n c t i o n i n t e r p o l a n t is m o r e a c c u r a t e
(as m e a s u r e d b y t h e p o i n t w i s e e r r o r ) t h a n a n y o t h e r l i n e a r c o m b i n a t i o n o f t h e
data.
18.2 Orthogonality in Reproducing K e r n e l Hilbert Spaces
T h e proofs o f t h e first t w o " o p t i m a l i t y t h e o r e m s " r e q u i r e t h e f o l l o w i n g t w o l e m m a s .

These l e m m a s a n d t h e i r c o r o l l a r y can also be generalized t o cover t h e s t r i c t l y con-
d i t i o n a l l y p o s i t i v e d e f i n i t e case. H o w e v e r , t o keep o u r discussion t r a n s p a r e n t , we
present o n l y t h e details o f t h e s t r i c t l y p o s i t i v e d e f i n i t e case.
s
L e m m a 1 8 . 1 . Assume <& is a symmetric strictly positive definite kernel on R and
let Vf be the interpolant to f G A/$(f2) at the data sites X = {x\,... ,x^} Q ft-
Then
(Vf,Vf -g)jv*(n) = 0
for all interpolants g G j\f$>(Q), i.e., with g{xj) = f(xj), j = 1,..., N.
Proof. T h e i n t e r p o l a n t Vf is o f t h e f o r m
N
w h e r e t h e coefficients Cj are d e t e r m i n e d b y t h e i n t e r p o l a t i o n c o n d i t i o n s Vf(xi) =

f(xi), i = 1 , . . . , N. Using this representation, the s y m m e t r y of the kernel $ and
i t s r e p r o d u c i n g p r o p e r t y we have
N
c x
(Vf,v f - >AMQ) = (^2 j^(^ j)^f - g)jv*m
18. The Optimality of RBF Interpolation 161
3= 1
N
3= 1
N
3= 1
= 0
since b o t h Vf a n d g i n t e r p o l a t e / o n X.
For t h e n e x t result, r e c a l l t h e d e f i n i t i o n o f t h e space H$(X) as t h e l i n e a r s p a n
H$(X) = span{<&(-, Xj) : xj G X}

(c.f. ( 1 3 . 1 ) ) . Clearly, H$(X) is a subspace o f t h e n a t i v e space Af<z>(fl).
s
L e m m a 1 8 . 2 . Assume & is a strictly positive definite kernel on R and let Vf be
the interpolant to f G j\f<f>(fl) on X = {xi,..., XN} C fl. Then
(f-Vf,h)^ (Cl) = 0
for all h G H*{X).
Proof. A n y h G H$ (X) c a n be w r i t t e n i n t h e f o r m
3= 1
w i t h a p p r o p r i a t e coefficients c -. U s i n g t h i s r e p r e s e n t a t i o n o f h as w e l l as t h e r e p r o -
3
d u c i n g p r o p e r t y o f <3> we have
N
{f -'Pf,h) ^n
jV ) = {f -Vf^Cj^Xj))^^)
3= 1
N
3= 1
N
= X>(/-p/)te)-
3= 1
T h i s last expression, however, is zero since Vf interpolates / o n X, i.e., (/

Vf)( )Xj = Q,j = l,...,N.
T h e f o l l o w i n g P y t h a g o r e a n t h e o r e m (or "energy s p l i t t i n g " t h e o r e m ) is a n i m -

m e d i a t e consequence o f L e m m a 18.2. I t says t h a t t h e n a t i v e space "energy" o f /
can be s p l i t i n t o t h e "energy" o f t h e i n t e r p o l a n t Vf and that of the residual f Vf,
w h i c h a c c o r d i n g t o L e m m a 18.2 is o r t h o g o n a l t o t h e i n t e r p o l a n t .
C o r o l l a r y 1 8 . 1 . The orthogonality property of Lemma 18.2 implies the e n e r g y s p l i t

_
ll/llSr*(n) = 11/ ^ / l l M t ( f i ) + H?VllA/"(n)-
Proof. T h e s t a t e m e n t follows f r o m
\\f\\jV.(n) = \\f-Vf + V \\fa

f m
= <(/ - V ) f + V, f (/ - V ) f + V )^
f { a )
2
= 1 1 / - PfWir.w) + 2 ( / - V, f 7 > / W ) + \\V \\
f w n )
a n d t h e fact t h a t ( / Vf,Vf)^{Q,) = 0 b y L e m m a 18.-2 since Vf G H(X).
T h e above energy s p l i t is t h e f u n d a m e n t a l idea b e h i n d a n u m b e r o f K r y l o v -

t y p e i t e r a t i v e a l g o r i t h m s for a p p r o x i m a t e l y s o l v i n g t h e i n t e r p o l a t i o n p r o b l e m w h e n
v e r y large d a t a sets are i n v o l v e d (see, e.g., o u r discussion i n C h a p t e r 33 or t h e
papers [Faul a n d P o w e l l (1999); F a u l a n d P o w e l l (2000)] or [Schaback a n d W e n d l a n d
(2000a)]).
18.3 Optimality Theorem I
T h e f o l l o w i n g t h e o r e m presents t h e first o p t i m a l i t y p r o p e r t y f o r m u l a t e d for t h e gen-

eral case o f s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e kernels. I t is t a k e n f r o m [ W e n d -
l a n d (2005a)].
Theorem 1 8 . 1 ( O p t i m a l i t y I ) . Suppose <& G C(Q x f2) is a strictly conditionally

positive definite kernel with respect to the finite-dimensional space P C C ( f 2 ) and
that X is P-unisolvent. If the values fx,..., fx are given, then the interpolant Vf
e
is the minimum-(semi)norm interpolant to {fj}jLx> i- ->
\V W*(n)=
f s mm n ) \g\^ ( n ) .
Proof. W e consider o n l y t h e s t r i c t l y p o s i t i v e d e f i n i t e case. Consider a n a r b i t r a r y

i n t e r p o l a n t g G j\f$(Q) to / i , . . , /JV- T h e n L e m m a 18.1 gives us
( V f , V - g ) ^f { n ) = 0 .
T h i s o r t h o g o n a l i t y r e l a t i o n gives us
\Vf\j\r*(n) = (Vf,V - g + g)jv(n)

f
= (Vf,V - #)Af*(fi) +
f (Vf,g)jv*(n)
= (Vf,g)Af^(ci),
a n d t h e Cauchy-Schwarz i n e q u a l i t y y i e l d s
\Vf\Jv^(Q) < \VfW< ,(n)\gW*(ci), t
so t h a t t h e s t a t e m e n t follows.
A s i n o u r earlier use o f c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s , t h e space P

mentioned i n Theorem 18.1 is u s u a l l y t a k e n as t h e space U^-i f multivariate
p o l y n o m i a l s . A l s o , i f $ is s t r i c t l y p o s i t i v e d e f i n i t e t h e n t h e s e m i - n o r m s i n T h e o -
r e m 18.1 become n o r m s .
18. The Optimality of RBF Interpolation 163
E x a m p l e 18.1. W e said earlier t h a t t h e n a t i v e space o f t h i n p l a t e splines <fr(r) =

r 2
l o g r , r \\x\\ 2 w i t h x (x,y) GR 2
is g i v e n b y t h e B e p p o - L e v i space B L 2 ( I R ) . 2
2
N o w , t h e c o r r e s p o n d i n g s e m i - n o r m i n t h e B e p p o - L e v i space B L ( M ) is (c.f. 2 (13.6))
2
df 2
df 2
a /
I/IBL (R )
2
2
Ju2
\ dx
2
(x) + 2
dxdy
(x) + dy 2
(x) dx,
w h i c h is t h e b e n d i n g energy o f a t h i n p l a t e . B y T h e o r e m 18.1 t h e t h i n p l a t e spline

i n t e r p o l a n t m i n i m i z e s t h i s energy. T h i s explains t h e n a m e o f these f u n c t i o n s .
18.4 Optimality Theorem I I
A n o t h e r nice p r o p e r t y o f t h e r a d i a l basis f u n c t i o n i n t e r p o l a n t is t h e fact t h a t i t is

at t h e same t i m e t h e best H i l b e r t - s p a c e a p p r o x i m a t i o n t o t h e g i v e n d a t a , a n d t h u s
n o t j u s t any p r o j e c t i o n o f / b u t t h e orthogonal projection. A g a i n , we f o r m u l a t e t h e
t h e o r e m for t h e s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e case a n d p r o v i d e d e t a i l s o n l y
for t h e s t r i c t l y p o s i t i v e definite case.
T h e o r e m 1 8 . 2 ( O p t i m a l i t y I I ) . Let
N
H^(X) = {h = ^2c $(-,x )j j +p : p e P
N
and Cjq(xj) = 0 for all q G P and Xj G X},
i=i
where <E> G C(fl x fl) is a strictly conditionally positive definite kernel with respect
to the finite-dimensional space P C C(fl) and X is P-unisolvent. If only the val-
ues fx = f(xi),...,fpj = f(xjy) are given, then the interpolant Vf is the best
approximation to f from H$(X) in j\f$(fl), i.e.,
\f - P/l/V^si) < \f - h\Af*(n)
for all h G H<&(X).
Proof. W e consider o n l y t h e s t r i c t l y p o s i t i v e definite case. A s e x p l a i n e d i n Sec-

t i o n 13.2, t h e n a t i v e space j\f<s>(fl) is t h e c o m p l e t i o n o f H$(fl) w i t h respect t o t h e
O R A e
|| | | $ - n o r m so t h a t | | / | | $ = H/HAM^) ^ ^ f H&(fl). A l s o , X C fl. Therefore,
we can characterize t h e best a p p r o x i m a t i o n g* t o / f r o m H$(X) by
(/ - g \ h ) N i b m = 0 for a l l h G H*(X).
However, L e m m a 18.2 shows t h a t g* = Vf satisfies t h i s r e l a t i o n .

These o p t i m a l i t y p r o p e r t i e s o f r a d i a l basis f u n c t i o n i n t e r p o l a n t s p l a y a n i m -
p o r t a n t role i n a p p l i c a t i o n s such as i n t h e design o f s u p p o r t v e c t o r machines i n
s t a t i s t i c a l l e a r n i n g t h e o r y or t h e n u m e r i c a l s o l u t i o n o f p a r t i a l d i f f e r e n t i a l e q u a t i o n s .
T h e o p t i m a l i t y results above i m p l y t h a t one c o u l d also s t a r t w i t h some H i l b e r t

space 7i w i t h n o r m || | | ^ a n d ask t o find t h e m i n i m u m n o r m i n t e r p o l a n t (i.e., Hilbert
space best a p p r o x i m a t i o n ) t o some g i v e n d a t a . I n t h i s w a y a n y g i v e n space defines
a set o f optimal basis functions, g e n e r a t e d b y t h e r e p r o d u c i n g k e r n e l o f 7i. T h i s is
h o w D u c h o n a p p r o a c h e d t h e s u b j e c t i n his p a p e r s [ D u c h o n (1976); D u c h o n (1977);
D u c h o n (1978); D u c h o n (1980)]. M o r e recently, K y b i c , B l u a n d U n s e r [ K y b i c et al.
(2002a); K y b i c et al. (2002b)] t a k e t h i s p o i n t o f v i e w a n d e x p l a i n f r o m a s a m p l i n g
t h e o r y p o i n t o f v i e w h o w t h e t h i n p l a t e splines c a n be i n t e r p r e t e d as fundamental
solutions o f t h e d i f f e r e n t i a l o p e r a t o r d e f i n i n g t h e s e m i - n o r m i n t h e B e p p o - L e v i space
2
B L 2 ( M ) , a n d t h u s r a d i a l basis f u n c t i o n s c a n be v i e w e d as Green's functions.
18.5 Optimality Theorem I I I
T h e t h i r d o p t i m a l i t y result is i n t h e c o n t e x t o f q u a s i - i n t e r p o l a t i o n , i.e.,
T h e o r e m 18.3 ( O p t i m a l i t y I I I ) . Suppose <E> G C(Q x fl) is a strictly condition-

ally positive definite kernel with respect to the finite-dimensional space P C C(fl).
Suppose X is P-unisolvent and x G fl is fixed. Let Uj(x), j = 1,...,N, be the
values at x of the cardinal basis functions for interpolation with <3>. Then
N N
<
3= 1
for all choices of u\,..., UN G M. with ujp(xj) = p(x) for any p G P.
T h e o r e m 18.3 is p r o v e d i n [ W e n d l a n d ( 2 0 0 5 a ) ] . I t says i n p a r t i c u l a r t h a t the

m i n i m u m n o r m i n t e r p o l a n t Vf is also m o r e a c c u r a t e ( i n t h e p o i n t w i s e sense) t h a n
a n y linear c o m b i n a t i o n o f t h e g i v e n d a t a values t h a t r e p r o d u c e P.
Chapter 19
Least Squares R B F Approximation

with M A T L A B
U p t o n o w we have l o o k e d o n l y a t i n t e r p o l a t i o n . H o w e v e r , m a n y t i m e s i t m a k e s
m o r e sense t o a p p r o x i m a t e t h e g i v e n d a t a b y a least squares f i t . T h i s is e s p e c i a l l y
t r u e i f t h e d a t a are c o n t a m i n a t e d w i t h noise, o r i f t h e r e are so m a n y d a t a p o i n t s
t h a t efficiency c o n s i d e r a t i o n s force us t o a p p r o x i m a t e f r o m a space s p a n n e d b y fewer
basis f u n c t i o n s t h a n d a t a p o i n t s .
19.1 Optimal Recovery Revisited
A s we saw i n C h a p t e r 18 we c a n i n t e r p r e t r a d i a l basis f u n c t i o n i n t e r p o l a t i o n as
a c o n s t r a i n e d o p t i m i z a t i o n p r o b l e m , i.e., t h e R B F i n t e r p o l a n t a u t o m a t i c a l l y m i n -
imizes t h e n a t i v e space n o r m a m o n g a l l i n t e r p o l a n t s i n t h e n a t i v e space. W e n o w
take this p o i n t of view again, b u t s t a r t w i t h a more general f o r m u l a t i o n . L e t us
assume we are seeking a f u n c t i o n Vf of t h e f o r m
M
s
V (x)
f =^2cj&(x,Xj), x e R ,
i=i
w h e r e t h e n u m b e r M o f basis f u n c t i o n s is i n g e n e r a l less t h a n o r e q u a l t h e n u m b e r
N o f d a t a sites. W e t h e n w a n t t o d e t e r m i n e t h e coefficients c = [c\,... ,CM\ T
SO
t h a t we m i n i m i z e t h e q u a d r a t i c f o r m
T
\c Qc (19.1)
w i t h some s y m m e t r i c p o s i t i v e d e f i n i t e m a t r i x Q s u b j e c t t o t h e l i n e a r c o n s t r a i n t s
Ac = f (19.2)
T
w h e r e A is a n N x M m a t r i x w i t h f u l l r a n k , a n d t h e r i g h t - h a n d side / = [ / i , . . . , / J V ]
is g i v e n . Such a c o n s t r a i n e d q u a d r a t i c m i n i m i z a t i o n p r o b l e m c a n be c o n v e r t e d t o a
T
s y s t e m o f l i n e a r e q u a t i o n s b y i n t r o d u c i n g Lagrange multipliers A = [ A i , . . . , Ajv] ,
i.e., w e consider f i n d i n g t h e m i n i m u m o f
l T T
-c Qc-\ [Ac-f] (19.3)
165
w i t h respect t o c a n d A. Since Q is a s s u m e d t o be a p o s i t i v e d e f i n i t e m a t r i x , i t is
w e l l k n o w n t h a t t h e f u n c t i o n a l t o be m i n i m i z e d is c o n v e x , a n d t h u s has a u n i q u e
m i n i m u m . Therefore, t h e s t a n d a r d necessary c o n d i t i o n for s u c h a m i n i m u m ( w h i c h
is o b t a i n e d b y d i f f e r e n t i a t i n g w i t h respect t o c a n d A a n d finding t h e zeros o f t h o s e
d e r i v a t i v e s ) is also sufficient. T h i s leads t o
T
Qc - A X = 0
Ac - f = 0
or, i n m a t r i x f o r m ,
T
Q A ' c "0
A O A
B y a p p l y i n g ( b l o c k ) G a u s s i a n e l i m i n a t i o n t o t h i s b l o c k m a t r i x (Q is i n v e r t i b l e since
i t is assumed t o be p o s i t i v e d e f i n i t e ) w e get
1 T 1
A = (AQ~ A )~ f (19.4)
1 T 1 T 1
c = Q- A (AQ- A y f. (19.5)
In particular, i f t h e q u a d r a t i c f o r m represents t h e n a t i v e space n o r m o f t h e

c x e
i n t e r p o l a n t Vf = Y2jL\ j*&(', j)i ^- ->
M M
1=1 j=\
w i t h Qij = <fr(ccj, Xj) and c = [ c i , . . . , C M ] , T

a n d t h e l i n e a r side c o n d i t i o n s are the
interpolation conditions
Ac = f V (xi)
f = fi, z = l,...,M,
T
with A = A Q (symmetric), the same c as a b o v e a n d d a t a vector / =
T
[fi-, i / M ] , t h e n we see t h a t t h e L a g r a n g e m u l t i p l i e r s (19.4) b e c o m e
1
A = ^ l - /
a n d t h e coefficients are g i v e n b y
c = A
v i a (19.5). T h e r e f o r e , as we saw earlier, t h e m i n i m u m n o r m i n t e r p o l a n t is o b t a i n e d

b y s o l v i n g t h e i n t e r p o l a t i o n e q u a t i o n s alone.
19.2 Regularized Least Squares Approximation
Since we t o o k t h e m o r e g e n e r a l p o i n t o f v i e w t h a t Vf is g e n e r a t e d b y M basis
f u n c t i o n s , a n d N l i n e a r c o n s t r a i n t s are specified, t h e a b o v e f o r m u l a t i o n also covers
b o t h over- a n d u n d e r - d e t e r m i n e d least squares fitting where the quadratic form
19. Least Squares RBF Approximation with M A T L A B 167
T
c Qc represents a n a d d e d smoothing (or regularization) term. T h i s t e r m is n o t
r e q u i r e d t o o b t a i n a u n i q u e s o l u t i o n o f t h e s y s t e m Ac = / i n t h e over-determined
case (N > M), b u t i n t h e u n d e r - d e t e r m i n e d case such a c o n s t r a i n t is needed (c.f.
t h e s o l u t i o n o f u n d e r - d e t e r m i n e d l i n e a r systems v i a s i n g u l a r value d e c o m p o s i t i o n i n
t h e n u m e r i c a l linear algebra l i t e r a t u r e (e.g., [Trefethen and B a u (1997)])).
U s u a l l y t h e r e g u l a r i z e d least squares a p p r o x i m a t i o n p r o b l e m is f o r m u l a t e d as
minimization of
N
1
2
- c ^ c + a ^ P ^ ) - / , )
3= 1
+ UJ(AC-
T
C QC
T
f) (Ac- f). (19.6)
T
T h e q u a d r a t i c f o r m c Qc c o n t r o l s t h e smoothness o f t h e f i t t i n g f u n c t i o n a n d t h e
least squares t e r m measures t h e closeness t o t h e d a t a . T h e p a r a m e t e r UJ c o n t r o l s
t h e tradeoff between these t w o t e r m s w i t h a large value o f UJ s h i f t i n g t h e b a l a n c e
t o w a r d increased p o i n t w i s e accuracy.
T h e f o r m u l a t i o n (19.6) is used i n regularization theory (see, e.g., [ E v g e n i o u et al.
(2000); G i r o s i (1998)]). T h e same f o r m u l a t i o n is also used i n penalized least squares
f i t t i n g (see, e.g., [von G o l i t s c h e k a n d Schumaker ( 1 9 9 0 ) ] ) , t h e l i t e r a t u r e o n smooth-
ing splines [Reinsch (1967); Schoenberg (1964)], a n d i n p a p e r s b y W a h b a o n t h i n
p l a t e splines (e.g., [ K i m e l d o r f a n d W a h b a (1971); W a h b a (1979); W a h b a ( 1 9 9 0 b ) ;
W a h b a a n d L u o (1997); W a h b a a n d W e n d e l b e r g e r (1980)]). I n fact, t h e i d e a o f
s m o o t h i n g a d a t a f i t t i n g process b y t h i s k i n d o f f o r m u l a t i o n seems t o go b a c k t o
at least [ W h i t t a k e r (1923)]. I n p r a c t i c e a p e n a l i z e d least squares f o r m u l a t i o n is
especially useful i f t h e d a t a fa c a n n o t be c o m p l e t e l y t r u s t e d , i.e., t h e y are c o n t a m i -
n a t e d b y noise. T h e p r o b l e m o f m i n i m i z i n g (19.6) is also k n o w n as ridge regression
i n t h e statistics l i t e r a t u r e . T h e r e g u l a r i z a t i o n p a r a m e t e r UJ is u s u a l l y chosen u s i n g
generalized cross v a l i d a t i o n .
T
I f we r e s t r i c t ourselves t o w o r k i n g w i t h square s y m m e t r i c systems, i.e., A = A,
a n d assume t h e smoothness f u n c t i o n a l is g i v e n b y t h e n a t i v e space n o r m , i.e.,
Q = A, t h e n we o b t a i n t h e m i n i m i z e r o f t h e u n c o n s t r a i n e d q u a d r a t i c f u n c t i o n a l
(19.6) b y s o l v i n g t h e linear s y s t e m
( h')
A+ e = t
(19.7)
w h i c h is t h e result o f s e t t i n g t h e d e r i v a t i v e o f (19.6) w i t h respect t o c e q u a l t o

zero. T h u s , r i d g e regression c o r r e s p o n d s t o a d i a g o n a l s t a b i l i z a t i o n / r e g u l a r i z a t i o n
of t h e usual i n t e r p o l a t i o n s y s t e m Ac = /. T h i s a p p r o a c h is especially useful for
s m o o t h i n g o f noisy d a t a . W e present a n i m p l e m e n t a t i o n o f t h i s m e t h o d a n d some
n u m e r i c a l examples b e l o w i n Section 19.4.
19.3 L e a s t Squares A p p r o x i m a t i o n W h e n R B F C e n t e r s Differ from

D a t a Sites
W e are n o w interested i n t h e m o r e general s e t t i n g w h e r e we s t i l l sample t h e g i v e n

f u n c t i o n / o n t h e set X = {x\,..., XN} o f d a t a sites, b u t n o w i n t r o d u c e a second set
S = {^yflx at w h i c h we center t h e basis f u n c t i o n s . U s u a l l y we w i l l have M < N,
a n d t h e case M = N w i t h E = X recovers t h e t r a d i t i o n a l i n t e r p o l a t i o n s e t t i n g
discussed i n earlier chapters. T h e r e f o r e , we c a n let t h e R B F a p p r o x i m a n t be o f t h e
form
M
2/0*0 = 5>i*(a >*i)> j x e R S

- 19
( - ) 8
T h e coefficients c 3 can be f o u n d as t h e least squares s o l u t i o n o f Ac f, i.e., by

2
m i n i m i z i n g \\Qf f]] ,, where the ^2-norm
N
H/llS = [ / ( * i ) ] 2
, * *
*
is i n d u c e d b y t h e discrete i n n e r p r o d u c t
N
(f,g)=^2f{x )g(x ),
i i XiGX. (19.9)
t=i
T h i s a p p r o x i m a t i o n p r o b l e m has a u n i q u e s o l u t i o n i f t h e ( r e c t a n g u l a r ) colloca-
tion matrix A w i t h entries
A j k = $(xj,S ),
k j = l,...,N, fc = l,...,M,
has f u l l r a n k .
I f t h e centers i n H are chosen t o f o r m a subset o f t h e d a t a l o c a t i o n s X, t h e n A
does have f u l l r a n k p r o v i d e d t h e r a d i a l basis f u n c t i o n s are selected a c c o r d i n g t o o u r
p r e v i o u s chapters o n i n t e r p o l a t i o n . T h i s is t r u e , since i n t h i s case A w i l l have a n
M x M square s u b m a t r i x w h i c h is n o n - s i n g u l a r ( b y v i r t u e o f b e i n g a n interpolation
matrix).
T h e o v e r - d e t e r m i n e d l i n e a r s y s t e m Ac = / w h i c h arises i n t h e s o l u t i o n o f t h e
least squares p r o b l e m c a n be solved u s i n g s t a n d a r d a l g o r i t h m s f r o m n u m e r i c a l linear
algebra such as Q R or s i n g u l a r value d e c o m p o s i t i o n . T h e r e f o r e t h e M A T L A B code
for R B F least squares a p p r o x i m a t i o n is a l m o s t i d e n t i c a l t o t h a t for i n t e r p o l a t i o n .
P r o g r a m 19.1 presents a n e x a m p l e for least squares a p p r o x i m a t i o n i n 2 D . N o w
we define t w o sets o f p o i n t s : t h e d a t a p o i n t s (defined i n lines 3 a n d 8 ) , a n d t h e
centers (defined i n lines 4, 6 a n d 7 ) . N o t e t h a t we first l o a d t h e centers since o u r
d a t a files Data2D_1089h a n d Data2D_81u c o n t a i n a v a r i a b l e d s i t e s w h i c h we w a n t
t o use for o u r d a t a sites. L o a d i n g t h e d a t a sites first, a n d t h e n t h e centers w o u l d
lead t o u n w a n t e d o v e r w r i t i n g o f t h e values i n d s i t e s . T h e s o l u t i o n o f t h e least
squares p r o b l e m is c o m p u t e d o n l i n e 16 u s i n g backslash m a t r i x left d i v i s i o n ( \ or
m l d i v i d e ) w h i c h a u t o m a t i c a l l y p r o d u c e s a least squares s o l u t i o n . T h e s u b r o u t i n e s
P l o t S u r f a n d P l o t E r r o r 2 D are p r o v i d e d i n A p p e n d i x C .
19. Least Squares RBF Approximation with MATLAB 169
P r o g r a m 19.1. RBFApproximation2D.m
'/ RBF A p p r o x i m a t i o n ^
7. S c r i p t t h a t performs b a s i c 2D RBF l e a s t squares approximation
7, C a l l s on: D i s t a n c e M a t r i x , P l o t S u r f , P l o t E r r o r 2 D
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; ep = 1;
3 N = 1089; g r i d t y p e = 'h';
4 M = 81; grid2type = >u';
5 n e v a l = 40;
7 Load c e n t e r s
6 name = s p r i n t f ('Data2D_7.d7.s',M,grid2type) ; load(name)
7 ctrs = dsites;
7 Load d a t a p o i n t s
,
8 name = s p r i n t f ( Data2D_/,d/ s' ,N,gridtype) ; load(name)
0
7. Compute d i s t a n c e m a t r i x between d a t a s i t e s and c e n t e r s

% B u i l d c o l l o c a t i o n matrix
10 CM = rbf(ep,DM_data);
% Create right-hand s i d e v e c t o r , i . e . ,
7 e v a l u a t e the t e s t f u n c t i o n a t the d a t a p o i n t s .
7, Create n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n
7c l o c a t i o n s i n the u n i t square
13 epoints = [ x e ( : ) y e ( : ) ] ;
7. Compute d i s t a n c e m a t r i x between e v a l u a t i o n p o i n t s and c e n t e r s
7o Compute RBF l e a s t squares approximation
16 Pf = EM * (CM\rhs);
7c Compute exact s o l u t i o n , i . e . , e v a l u a t e t e s t
7c f u n c t i o n on e v a l u a t i o n p o i n t s
% Compute maximum e r r o r on e v a l u a t i o n g r i d
7, P l o t s
19 f i g u r e ; fview = [100,30]; 7. viewing angles f o r p l o t
20 c a p t i o n = s p r i n t f C 7 c d d a t a s i t e s and 7od c e n t e r s ' ,N,M) ;
21 title(caption);
22 plot(dsites(:,1),dsites(:,2),'bo',ctrs(:,1),ctrs(:,2),'r+');
23 PlotSurf(xe,ye,Pf,neval,exact,maxerr,fview);
24 PlotError2D(xe,ye,Pf.exact,maxerr,neval,fview);
O u t p u t f r o m R B F A p p r o x i m a t i o n 2 D .m is p r e s e n t e d i n F i g u r e 19.1 a n d t h e t o p
p a r t o f F i g u r e 19.2.
Fig. 19.1 1089 Halton data sites (o) and 81 uniform centers (+).
I f e = 1, t h e n t h e c o l i o c a t i o n m a t r i x is r a n k deficient w i t h M A T L A B r e p o r t i n g a
n u m e r i c a l r a n k o f 58. I n o r d e r t o have a f u l l n u m e r i c a l r a n k for t h i s p r o b l e m e needs
to be at least 2.2 ( i n w h i c h case t h e m a x i m u m e r r o r d e t e r i o r a t e s t o 5.255591e-004
i n s t e a d o f 2.173460e-007 for e = 1, c.f. t h e t o p p a r t o f F i g u r e 19.2). T h e r e is n o t
m u c h t h e o r y available for t h e case o f d i f f e r i n g centers a n d d a t a sites. W e present
w h a t is k n o w n i n t h e n e x t c h a p t e r . Some care needs t o be t a k e n w h e n c o m p u t i n g
least squares solutions based o n sets o f d i f f e r i n g centers a n d d a t a sites.
19.4 L e a s t Squares S m o o t h i n g of N o i s y Data
We present t w o strategies for d e a l i n g w i t h n o i s y d a t a , i.e., d a t a t h a t we consider

to be n o t r e l i a b l e due t o , e.g., m e a s u r e m e n t or t r a n s m i s s i o n errors. T h i s s i t u a t i o n
arises f r e q u e n t l y i n p r a c t i c e . W e s i m u l a t e a set o f n o i s y d a t a b y s a m p l i n g F r a n k e ' s
test f u n c t i o n a t a set X o f d a t a sites, a n d t h e n a d d i n g u n i f o r m l y d i s t r i b u t e d r a n d o m
noise o f various s t r e n g t h s . For t h i s e x p e r i m e n t we use t h i n p l a t e splines since t h e i r
n a t i v e space n o r m corresponds t o t h e b e n d i n g energy o f a t h i n p l a t e a n d t h u s t h e y
have a t e n d e n c y t o p r o d u c e " v i s u a l l y pleasing" s m o o t h a n d t i g h t surfaces.
Since t h e t h i n p l a t e splines have a s i n g u l a r i t y at t h e o r i g i n a l i t t l e e x t r a care
needs t o be t a k e n w i t h t h e i r i m p l e m e n t a t i o n . T h e M A T L A B s c r i p t t p s . m we use for
o u r i m p l e m e n t a t i o n o f t h i s basic f u n c t i o n is i n c l u d e d i n A p p e n d i x C as P r o g r a m C.4.
O u r first s t r a t e g y is t o c o m p u t e a s t r a i g h t f o r w a r d least squares a p p r o x i m a t i o n
t o t h e (large) set o f d a t a u s i n g a ( s m a l l ) set o f basis f u n c t i o n s as we d i d i n t h e
previous section. I n t h e s t a t i s t i c s l i t e r a t u r e t h i s a p p r o a c h is k n o w n as regression
splines. W e w i l l n o t address t h e q u e s t i o n o f h o w t o choose t h e centers for t h e basis
functions a t t h i s p o i n t .
We use a m o d i f i c a t i o n o f p r o g r a m R B F A p p r o x i m a t i o n 2 D .m t h a t a l l o w s us t o use
thin plate splines with the added linear polynomial term. These changes can be
found on lines 1, 15, 16, 19 and 24 of Program 19.2. Also, we now replace the sine
test function by Franke's function (2.2). The noise is added to the right-hand side
of the linear system on line 18. This modification adds 3 % noise to the data.
P r o g r a m 19.2. RBFApproximation2Dlinear .m
% RBFApproximation2Dlinear
% S c r i p t t h a t performs 2D RBF l e a s t squares approximation w i t h
% l i n e a r reproduction f o r n o i s y d a t a
7o C a l l s on: t p s , D i s t a n c e M a t r i x
1 rbf = @tps; ep = 1; % d e f i n e d i n tps.m (see Appendix C)
% Define Franke's f u n c t i o n as t e s t f u n c t i o n
2 f l = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x - 2 ) . ~ 2 + ( 9 * y - 2 ) . ~ 2 ) / 4 ) ;
3 2 = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x + l ) . ~ 2 / 4 9 + ( 9 * y + l ) . " 2 / 1 0 ) ) ;
4 f 3 = <3(x,y) 0 . 5 * e x p ( - ( ( 9 * x - 7 ) . ~ 2 + ( 9 * y - 3 ) . ~ 2 ) / 4 ) ;
5 f 4 = @(x,y) 0 . 2 * e x p ( - ( ( 9 * x - 4 ) . ~ 2 + ( 9 * y - 7 ) . ~ 2 ) ) ;
6 t e s t f u n c t i o n = @(x,y) f 1 ( x , y ) + f 2 ( x , y ) + f 3 ( x , y ) - f 4 ( x , y ) ;
5
7 N = 1089; g r i d t y p e = 'h ;
8 M = 81; grid2type = 'u';
9 n e v a l = 40;
% Load c e n t e r s
10 name = s p r i n t f ( 'Data2D_/ d/ s' ,M,grid2type) ; load(name)
0 0
11 c t r s = d s i t e s ;
0
12 name = s p r i n t f ( Data2D_ /od /,s' ,N .gridtype) ; load(name)
, 0
% Compute d i s t a n c e matrix between d a t a s i t e s and c e n t e r s

14 CM = rbf(ep,DM_data); % C o l l o c a t i o n matrix
% Add e x t r a columns and rows f o r l i n e a r r e p r o d u c t i o n
15 PM = [ones(N,l) d s i t e s ] ; PtM = [ones(M.l) c t r s ] ' ;
16 CM = [CM PM; [PtM z e r o s ( 3 , 3 ) ] ] ;
% Create r i g h t - h a n d s i d e v e c t o r and add n o i s e
18 r h s = r h s + 0 . 0 3 * r a n d n ( s i z e ( r h s ) ) ;
7o Add zeros f o r l i n e a r (2D) r e p r o d u c t i o n
19 r h s = [ r h s ; z e r o s ( 3 , l ) ] ;
% Create n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
/ i n the u n i t square
c
20 g r i d = l i n s p a c e ( 0 , 1 , n e v a l ) ; [xe.ye] = m e s h g r i d ( g r i d ) ;
21 e p o i n t s = [ x e ( : ) y e ( : ) ] ;
a.
1J3J
23 EM = rbf(ep,DM_eval); % Evaluation matrix

% Add columns f o r l i n e a r r e p r o d u c t i o n
24 PM = [ones(neval"2,1) e p o i n t s ] ; EM = [EM PM];
% Compute RBF l e a s t squares approximation
/, Compute exact s o l u t i o n , i . e . ,
'/, e v a l u a t e t e s t f u n c t i o n on e v a l u a t i o n p o i n t s
Plots
28 f i g u r e ; fview = [160,20]; / viewing angles f o r p l o t
0
,0 4
29 c a p t i o n = s p r i n t f ( / d data s i t e s and / d centers',N,M);
o 0
30 title(caption);
32 PlotSurf(xe,ye,Pf,neval,exact.maxerr,fview);
P r o g r a m R B F A p p r o x i a t i o n 2 D l i n e a r .m was used t o p r o d u c e t h e r i g h t p l o t i n t h e
b o t t o m p a r t o f F i g u r e 19.2 a n d t h e entries i n l i n e 2 o f T a b l e 19.1. C l e a r l y , t h i s s i m p l e
least squares a p p r o a c h p e r f o r m s m u c h b e t t e r t h a n s t r a i g h t f o r w a r d i n t e r p o l a t i o n
t o t h e n o i s y d a t a (see t h e left p l o t t h e b o t t o m p a r t o f F i g u r e 19.2 a n d l i n e 1 o f
T a b l e 19.1). M o r e o v e r , t h i s least squares a p p r o x i m a t i o n is also m u c h cheaper t o
compute. H o w e v e r , as w e p o i n t e d o u t earlier, i t is n o t clear h o w t o choose t h e
smaller set o f R B F centers, a n d w h a t is even m o r e u n s e t t l i n g , t h e r e is n o t m u c h
m a t h e m a t i c a l t h e o r y (see t h e n e x t s e c t i o n ) t o g u a r a n t e e i f (or w h e n ) t h i s a p p r o a c h
is well-posed, i.e., t h e c o l l o c a t i o n m a t r i x has f u l l r a n k .
Table 19.1 Errors and condition numbers for various least squares approximants
to noisy data.
method RMS-error max-error cond(A)
Interpolation oo 2.482624e-002 9.914837e-002 1.502900e+007

L S Q approximation NA 9.665743e-003 5.490050e-002 NA
Ridge regression 1000 1.713843e-002 7.580288e-002 2.537652e+006
A n o t h e r s t r a t e g y for s m o o t h i n g o f n o i s y d a t a is t h e r i d g e regression m e t h o d
e x p l a i n e d earlier (see (19.7)). T h i s m e t h o d is p o p u l a r i n t h e s t a t i s t i c s a n d n e u r a l
network community.
T h e n a t u r e o f t h e M A T L A B p r o g r a m for r i d g e regression is v e r y s i m i l a r t o t h a t for
RBF i n t e r p o l a t i o n . W e present a v e r s i o n for r i d g e regression w i t h t h i n p l a t e splines
y x y x
Fig. 19.2 Top: Least squares approximation (left) to 1089 data points sampled from 2D sine
function with 81 Gaussian basis functions with e = 1 and maximum error (right) false-colored
by magnitude of error. Bottom: T h i n plate spline interpolant (left) to 1089 noisy data points
sampled from Franke's function, and least squares approximation with 81 uniformly spaced thin
plate spline basis functions (right) false-colored by magnitude of error.
(including the linear term in the basis expansion) for smoothing of noisy data. T h e
smoothing parameter u) of (19.7) is defined on line 7 of Program 19.3, and the
diagonal stabilization of the (interpolation) matrix A is performed on line 17. Note
that the stabilization only affects the A part of the matrix, and not the extra rows
and columns added for polynomial precision.
P r o g r a m 19.3. TPS_RidgeRegression2D .m
% TPS_RidgeRegression2D
% S c r i p t t h a t performs 2D TPS-RBF approximation w i t h r e p r o d u c t i o n of
% l i n e a r f u n c t i o n s and smoothing v i a r i d g e r e g r e s s i o n
% C a l l s on: t p s , D i s t a n c e M a t r i x
/ Use TPS ( d e f i n e d i n tps.m, see Appendix C)
0
1 r b f = @tps; ep = 1;
/ Define Franke's f u n c t i o n a s t e s t f u n c t i o n
0
2 f l = @(x,y) 0.75*exp(-((9*x-2)."2+(9*y-2).~2)/4);
3 f 2 = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x + l ) . ~ 2 / 4 9 + ( 9 * y + l ) . ~ 2 / 1 0 ) ) ;
4 f 3 = @(x,y) 0 . 5 * e x p ( - ( ( 9 * x - 7 ) . ~ 2 + ( 9 * y - 3 ) . ~ 2 ) / 4 ) ;
5 f 4 = @(x,y) 0 . 2 * e x p ( - ( ( 9 * x - 4 ) . ~ 2 + ( 9 * y - 7 ) . ~ 2 ) ) ;
7 omega = 100; % Smoothing parameter
8 N = 1089; g r i d t y p e = 'h';
9 n e v a l = 40;
/ Load data p o i n t s
0
10 name = s p r i n t f ( 'Data2D_/od/ s' ,N,gridtype) ; load(name)

0
11 ctrs = dsites;
% Compute d i s t a n c e matrix between d a t a s i t e s and c e n t e r s
% Create n o i s y right-hand s i d e v e c t o r
14 rhs = rhs + 0.03*randn(size(rhs));
% Add z e r o s f o r 2D l i n e a r r e p r o d u c t i o n
15 rhs = [rhs; zeros(3,1)];
% Compute i n t e r p o l a t i o n m a t r i x and add d i a g o n a l r e g u l a r i z a t i o n
17 IM = IM + e y e ( s i z e ( I M ) ) / ( 2 * o m e g a ) ;
% Add e x t r a columns and rows f o r l i n e a r r e p r o d u c t i o n
18 PM = [ones(N,l) d s i t e s ] ; IM = [IM PM; [PM' z e r o s ( 3 , 3 ) ] ] ;
19 f p r i n t f (' Condition number e s t i m a t e : /e\n', condest (IM) )
7, Create n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
7o i n the u n i t square
21 epoints = [ x e ( : ) y e ( : ) ] ;
7o Compute e v a l u a t i o n m a t r i x and add columns f o r l i n e a r p r e c i s i o n
24 PM = [ o n e s ( n e v a l " 2 , D e p o i n t s ] ; EM = [EM PM] ;
7o Compute RBF i n t e r p o l a n t
25 Pf = EM * ( I M \ r h s ) ;
7o Compute exact s o l u t i o n , i . e . ,
7. e v a l u a t e t e s t f u n c t i o n on e v a l u a t i o n p o i n t s
7o Compute maximum e r r o r on e v a l u a t i o n g r i d
7o P l o t s
28 fview = [160,20]; % viewing angles f o r p l o t
T h e r e s u l t s for o u r e x a m p l e s c o m p u t e d w i t h P r o g r a m 19.3 are s h o w n i n F i g -

u r e 19.3 as w e l l as i n lines 3 - 6 o f T a b l e 1 9 . 1 . T h e s e r e s u l t s i l l u s t r a t e v e r y n i c e l y
t h e s m o o t h i n g effect o b t a i n e d b y v a r y i n g UJ. A v e r y l a r g e v a l u e o f UJ emphasizes t h e
f i t t i n g c o m p o n e n t o f t h e f u n c t i o n a l t o be m i n i m i z e d i n (19.6) r e s u l t i n g i n a r a t h e r
r o u g h surface, w h i l e a s m a l l v a l u e o f u> gives preference t o t h e s m o o t h i n g t e r m . The
" o p t i m a l " v a l u e o f UJ lies s o m e w h e r e i n t h e m i d d l e . I n p r a c t i c e one w o u l d u s u a l l y
use cross v a l i d a t i o n t o o b t a i n t h e o p t i m a l v a l u e o f UJ.
Besides t h e v i s u a l s m o o t h i n g o f t h e a p p r o x i m a t i n g surface, a s m a l l v a l u e o f UJ
also has a s t a b i l i z i n g effect o n t h e c o l l o c a t i o n m a t r i x . T h e d i a g o n a l o f t h e m a t r i x
becomes m o r e a n d m o r e d o m i n a n t . T h e c o n d i t i o n e s t i m a t e s i n T a b l e 19.1 also v e r i f y
this behavior.
Fig. 19.3 T h i n plate spline ridge regression to 1089 noisy data points sampled from Franke's
function with UJ = 1000 (top left), UJ 100 (top right), UJ = 10 (bottom left), and UJ 1 (bottom
right).
Chapter 20
Theory for Least Squares Approximation
I n t h i s c h a p t e r we give a b r i e f a c c o u n t o f t h e t h e o r e t i c a l r e s u l t s k n o w n for least

squares a p p r o x i m a t i o n w i t h r a d i a l basis f u n c t i o n s . T h e s e r e s u l t s i n c l u d e e x t e n s i o n s
o f t h e R B F i n t e r p o l a t i o n t h e o r y t o cover well-posedness for t h e s i t u a t i o n i n w h i c h
centers S a n d d a t a sites X differ. W e also p r e s e n t some r e c e n t e r r o r e s t i m a t e s for
least squares a p p r o x i m a t i o n .
20.1 Well-Posedness of R B F L e a s t S q u a r e s Approximation
T h e r e s u l t s m e n t i o n e d here are due t o Q u a k , S i v a k u m a r a n d W a r d [ S i v a k u m a r a n d

W a r d (1993); Q u a k et al. ( 1 9 9 3 ) ] . T h e first p a p e r deals w i t h discrete, t h e second
w i t h c o n t i n u o u s least squares a p p r o x i m a t i o n . I n b o t h papers the authors do not
discuss t h e c o l l o c a t i o n m a t r i x A we used i n t h e p r e v i o u s c h a p t e r , b u t r a t h e r base
t h e i r results o n t h e n o n - s i n g u l a r i t y o f t h e coefficient m a t r i x o b t a i n e d f r o m a s y s t e m
of n o r m a l equations.
I n t h e d i s c r e t e s e t t i n g t h e y use t h e i n n e r p r o d u c t (19.9) w h i c h i n d u c e s t h e i 2
n o r m , a n d t h e n discuss n o n - s i n g u l a r i t y o f t h e Gram matrix G t h a t occurs i n the

f o l l o w i n g s y s t e m o f normal equations
Gc = w, (20.1)
w h e r e t h e e n t r i e s o f G are t h e l 2 i n n e r p r o d u c t s o f t h e r a d i a l basis f u n c t i o n s , i.e.,

N
$
G jk = ($(-,^), $(,*)> = E (^> i=l
6:), j,k = l,...,M,
a n d t h e r i g h t - h a n d side v e c t o r w i n (20.1) is g i v e n b y
i=l
N o t e t h a t i n t h e i n t e r p o l a t i o n case w i t h M N a n d S = X (i.e., coinciding

centers a n d d a t a sites) w e h a v e
(^(,Xj),^(-,X ))
k = (^(,X ),^(-,X ))^(c )
j k l = $(Xj,X )
k
177
so t h a t G is j u s t t h e i n t e r p o l a t i o n m a t r i x A. T h i s provides yet another w a y of

s a y i n g t h a t t h e i n t e r p o l a t i o n m a t r i x A is also t h e s y s t e m m a t r i x for t h e n o r m a l
e q u a t i o n s i n t h e case o f best a p p r o x i m a t i o n w i t h respect t o t h e n a t i v e space n o r m
a fact a l r e a d y m e n t i o n e d e a r l i e r i n C h a p t e r 18 o n o p t i m a l recovery.
I n b o t h p a p e r s , [ S i v a k u m a r a n d W a r d (1993)] as w e l l as [ Q u a k et al. (1993)],
even t h e f o r m u l a t i o n o f t h e m a i n t h e o r e m s is v e r y t e c h n i c a l . W e t h e r e f o r e j u s t t r y
t o give a feel for t h e i r results.
E s s e n t i a l l y , t h e a u t h o r s s h o w t h a t t h e G r a m m a t r i x for c e r t a i n r a d i a l basis
f u n c t i o n s ( n o r m , (inverse) m u l t i q u a d r i c s , a n d G a u s s i a n s ) is n o n - s i n g u l a r i f t h e cen-
ters S = k 1 , . . . , M} are s u f f i c i e n t l y w e l l d i s t r i b u t e d a n d t h e d a t a p o i n t s
X = {xj,j = 1 , . . . , N} are f a i r l y e v e n l y c l u s t e r e d a b o u t t h e centers w i t h t h e d i -
ameter o f the clusters being r e l a t i v e l y s m a l l c o m p a r e d t o the separation distance of
t h e d a t a p o i n t s . F i g u r e 20.1 i l l u s t r a t e s t h i s c l u s t e r i n g idea.
o o
o o o o
o
o
o
o
o o o o
o
o
o
o
Fig. 0 . 1 Clusters of data points o around well separated centers .
O n e o f t h e k e y i n g r e d i e n t s i n t h e p r o o f o f t h e n o n - s i n g u l a r i t y o f G is t o set
u p a n i n t e r p o l a t i o n m a t r i x B for w h i c h t h e basis f u n c t i o n s are c e n t e r e d a t c e r t a i n
r e p r e s e n t a t i v e s o f t h e c l u s t e r s o f k n o t s a b o u t t h e d a t a sites. One t h e n splits the
matrix B ( w h i c h is n o n - s y m m e t r i c i n g e n e r a l ) i n t o a p a r t t h a t is s y m m e t r i c and
one t h a t is a n t i - s y m m e t r i c , a s t a n d a r d s t r a t e g y i n l i n e a r a l g e b r a , i.e., B B\ + B 2
w h e r e B\ and B 2 are defined b y

T
B + B ,
B\ (symmetric),
T
B B
B 2 = (anti-symmetric).
T h e n , l o w e r e s t i m a t e s for t h e n o r m o f these t w o p a r t s are f o u n d a n d u s e d t o c o n c l u d e

t h a t , u n d e r c e r t a i n r e s t r i c t i o n s , G is n o n - s i n g u l a r .
A s a b y - p r o d u c t o f this a r g u m e n t a t i o n the a u t h o r s o b t a i n a p r o o f for t h e n o n -
s i n g u l a r i t y o f interpolation m a t r i c e s for t h e case i n w h i c h t h e centers o f t h e basis
20. Theory for Least Squares Approximation 179
f u n c t i o n s are chosen different f r o m t h e d a t a sites, n a m e l y as s m a l l perturbations

thereof.
20.2 E r r o r B o u n d s for Least Squares A p p r o x i m a t i o n
I n t h e case o f basis f u n c t i o n s c e n t e r e d a t t h e p o i n t s o f a n i n f i n i t e l a t t i c e de B o o r ,
D e V o r e a n d R o n [ d e B o o r et a/.(1994b); R o n (1992)] discussed L 2 - a p p r o x i m a t i o n
o r d e r s for r a d i a l basis f u n c t i o n s .
M o r e recently, [ W a r d (2004)] p r o v i d e d e r r o r b o u n d s for least squares a p p r o x i -
m a t i o n at s c a t t e r e d centers a n d i n f i n i t e d o m a i n s . H i s w o r k is closely l i n k e d t o t h e
r e s u l t s i n [ N a r c o w i c h et al. (2005)] discussed earlier i n C h a p t e r 15.
A t y p i c a l r e s u l t is t h a t t h e c o n t i n u o u s least squares e r r o r for a p p r o x i m a t i o n
based o n t h e c o m p a c t l y s u p p o r t e d W e n d l a n d f u n c t i o n s is
min ||/ - <2/|U a ( n ) < C M > n | */T||/||wj(fi).
s
H e r e fl C R is a b o u n d e d L i p s c h i t z d o m a i n w h i c h satisfies a n i n t e r i o r cone c o n d i -
t i o n , X C fl is t h e set o f centers, H<&(X) span{$(- X j ) , xj X}, k is such t h a t
r = k + a w i t h 0 < a < 1, a n d r > s/2 measures t h e decay o f t h e F o u r i e r t r a n s f o r m
o f <&, i.e.,
ci(l + I M H ) - - < < c (l + |MH)- ,

2
T
| M | oo, UJ G s
R,
w i t h p o s i t i v e c o n s t a n t s c\ a n d c so t h a t t h e n a t i v e space o f $ is g i v e n b y t h e S o b o l e v
2
s
space W J ( R ) - T h i s e r r o r b o u n d is o f t h e same o r d e r as t h e one for i n t e r p o l a t i o n
(c.f. T h e o r e m 15.3). W e refer t h e reader t o [ W a r d (2004)] for m o r e d e t a i l s .
is
Chapter 21
Adaptive Least Squares Approximation
I n t h i s c h a p t e r w e m e n t i o n some s t r a t e g i e s for s o l v i n g t h e least squares p r o b l e m i n

a n a d a p t i v e fashion.
21.1 Adaptive Least Squares using K n o t Insertion
A classical t e c h n i q u e used t o i m p r o v e t h e q u a l i t y o f a g i v e n i n i t i a l a p p r o x i m a t i o n
based o n a l i n e a r c o m b i n a t i o n o f c e r t a i n basis f u n c t i o n s is t o a d a p t i v e l y increase
t h e n u m b e r o f basis f u n c t i o n s used for t h e f i t . I n o t h e r w o r d s , one refines t h e space
f r o m w h i c h t h e a p p r o x i m a t i o n is c o m p u t e d . Since e v e r y r a d i a l basis f u n c t i o n is
associated w i t h one p a r t i c u l a r center ( o r knot), t h i s c a n b e achieved b y a d d i n g
2
n e w k n o t s t o t h e e x i s t i n g ones. T h i s i d e a was e x p l o r e d for m u l t i q u a d r i c s o n M in
[Franke et al. (1994); F r a n k e et al. ( 1 9 9 5 ) ] , a n d for r a d i a l basis f u n c t i o n s o n spheres
m [Fasshauer (1995a)].
W e w i l l n o w describe a n a l g o r i t h m t h a t a d a p t i v e l y adds k n o t s t o a r a d i a l basis
function a p p r o x i m a n t i n order t o i m p r o v e the i 2 error.
L e t us assume w e are g i v e n a large n u m b e r , N, o f d a t a a n d we w a n t t o f i t t h e m
w i t h a r a d i a l basis e x p a n s i o n t o w i t h i n a g i v e n t o l e r a n c e . T h e idea is t o s t a r t w i t h
v e r y few i n i t i a l k n o t s , a n d t h e n t o r e p e a t e d l y i n s e r t a k n o t a t t h a t d a t a l o c a t i o n
whose i 2 e r r o r c o m p o n e n t is largest. T h i s is d o n e as l o n g as t h e least squares e r r o r
exceeds a g i v e n t o l e r a n c e . T h e f o l l o w i n g a l g o r i t h m m a y be used.
A l g o r i t h m 21.1. K n o t insertion
(1) L e t d a t a sites X {x\,..., CCJV}, d a t a / * , i 1 , . . . , N, a n d a t o l e r a n c e tol be

given.
(2) Choose M i n i t i a l k n o t s S = { i , . . . , M } -
(3) C a l c u l a t e t h e least squares f i t
3=1
181
w i t h i t s associated e r r o r
N
W h i l e e > tol do
(4) " W e i g h t " each d a t a p o i n t Xi, i = 1 , . . . , N, according t o its error compo-

n e n t , i.e., let
Wi = \fi-Qf(xi)\, i = l,...,N.
(5) F i n d the data point x u 0 S w i t h m a x i m u m weight w u a n d i n s e r t i t as a

knot, i.e.,
E = EU{x} and M = M + 1.
(6) Recalculate fit a n d associated error.
A MATLAB implementation of the knot i n s e r t i o n a l g o r i t h m is p r o v i d e d in

R B F K n o t I n s e r t 2 D . m ( P r o g r a m 2 1 . 1 ) . T h i s p r o g r a m is a l i t t l e m o r e t e c h n i c a l than
p r e v i o u s ones since we need t o a v o i d a d d i n g t h e same p o i n t m u l t i p l e t i m e s . This
w o u l d lead t o a s i n g u l a r s y s t e m . I n M A T L A B w e c a n easily check t h i s w i t h the
c o m m a n d i s m e m b e r (see l i n e 2 8 ) . W e also t a k e a d v a n t a g e o f t h e s o r t c o m m a n d t o
h e l p us f i n d ( p o s s i b l y several) k n o t s w i t h l a r g e s t e r r o r c o n t r i b u t i o n . T h e a d d i t i o n
of these d a t a sites t o t h e set o f centers is a c c o m p l i s h e d o n lines 2 6 - 3 2 . Evaluation
o f t h e a p p r o x i m a n t (see lines 3 4 - 3 6 ) is n o t r e q u i r e d u n t i l a l l o f t h e k n o t s have b e e n
inserted.
Program 2 1 . 1 . RBFKnotInsert2D.m
% RBFKnotInsert2D
% Script that performs 2D RBF l e a s t squares approximation
% v i a knot insertion
% Calls on: DistanceMatrix
1 rbf = @(e,r) exp(-(e*r).~2); ep = 5.5;
/ D e f i n e
0 F r a n k e ' s f u n c t i o n as testfunction
2 f l = <3(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x - 2 ) . ~ 2 + ( 9 * y - 2 ) . ~ 2 ) / 4 ) ;
3 f2 = @(x,y) 0.75*exp(-((9*x+l).~2/49+(9*y+l).~2/10));
4 f3 = @(x,y) 0.5*exp(-((9*x-7).~2+(9*y-3).~2)/4);
5 f4 = @(x,y) 0.2*exp(-((9*x-4).~2+(9*y-7)."2));
6 testfunction = @(x,y) fI(x,y)+f2(x,y)+f3(x,y)-f4(x,y);
7 N = 289; gridtype = 'h';
8 M = 1; % Number o f initial centers
9 neval = 40;
10 grid = linspace(0,1,neval); [xe.ye] = meshgrid(grid);
11 epoints = [xe(:) ye(:)];
12 t o l = le-5; % Tolerance; stopping criterion
21. Adaptive Least Squares Approximation 183
13 name = s p r i n t f ( ' D a t a 2 D _ % d % s ' , N , g r i d t y p e ) ; load(name)
% Take f i r s t M "data s i t e s " as c e n t e r s
14 ctrs = dsites(1:M,:);
% Compute exact s o l u t i o n , i . e . , e v a l u a t e t e s t f u n c t i o n
% on e v a l u a t i o n p o i n t s
% Create r i g h t - h a n d s i d e v e c t o r , i . e . ,
% evaluate the t e s t function a t the data points.
lG rhs = testfunction(dsites(:,1),dsites(:,2));
17 rms_res = 999999;
18 w h i l e (rms_res > t o l )
% Compute l e a s t squares f i t
21 coef = CM\rhs;
/ Compute r e s i d u a l
0
22 r e s i d u a l = abs(CM*coef - r h s ) ;
23 [sresidual,idx] = sort(residual);
24 Ires = length(residual);
25 rms_res = n o r m ( r e s i d u a l ) / s q r t ( I r e s ) ;
7, Add p o i n t ( s )
26 i f (rms_res > t o l )
27 addpoint = i d x ( I r e s ) ; % T h i s i s t h e p o i n t we add
/ I f a l r e a d y used, t r y next p o i n t
0
28 while any(ismember(ctrs,dsites(addpoint,:),'rows'))
29 I r e s = l r e s - 1 ; addpoint = i d x ( l r e s ) ;
30 end
31 ctrs = [ctrs; dsites(addpoint,:)];
32 end
33 end
/ Compute e v a l u a t i o n m a t r i x
0
36 Pf = EM*coef; % Compute RBF l e a s t squares approximation
37 maxerr = max(abs(Pf - e x a c t ) ) ; rms.err = n o r m ( P f - e x a c t ) / n e v a l ;
38 f p r i n t f ('RMS e r r o r : /e\n', r m s . e r r )
0
39 f i g u r e ; / P l o t d a t a s i t e s and c e n t e r s
0
41 PlotSurf(xe,ye,Pf,neval,exact,maxerr,[160,20] ) ;
W e p o i n t o u t t h a t we have t o solve one l i n e a r least squares p r o b l e m i n each

iteration. W e do t h i s u s i n g t h e s t a n d a r d M A T L A B b a c k s l a s h (or m l d i v i d e ) QR-
based solver (see l i n e 2 1 ) . T h e size o f these p r o b l e m s increases a t each s t e p w h i c h

m e a n s t h a t a d d i t i o n o f n e w k n o t s becomes i n c r e a s i n g l y m o r e expensive. T h i s is
u s u a l l y n o t such a b i g d e a l . B o t h [ F r a n k e et al. ( 1 9 9 4 ) ; F r a n k e et al. (1995)] a n d
[Fasshauer (1995a)] f o u n d t h a t t h e d e s i r e d a c c u r a c y was u s u a l l y achieved w i t h f a i r l y
few a d d i t i o n a l k n o t s a n d t h u s t h e a l g o r i t h m is q u i t e fast.
I f t h e i n i t i a l k n o t s are chosen t o l i e a t d a t a sites (as we d i d i n o u r MATLAB
implementation), then the collocation m a t r i x A i n the k n o t insertion algorithm w i l l
always have f u l l r a n k . T h i s is g u a r a n t e e d since w e o n l y a d d d a t a sites as n e w k n o t s ,
a n d we m a k e sure i n step (5) o f t h e a l g o r i t h m t h a t n o m u l t i p l e k n o t s are c r e a t e d
(which w o u l d obviously lead t o a r a n k deficiency).
Instead of deciding which point t o add based o n residuals one c o u l d also
pick the new point by looking at the power function, since t h e dependence o f
the a p p r o x i m a t i o n error o n the data sites is e n c o d e d i n the power function.
T h i s s t r a t e g y is used t o b u i l d so-called greedy adaptive algorithms that inter-
polate successively m o r e a n d m o r e d a t a (see [Schaback a n d W e n d l a n d ( 2 0 0 0 a ) ;
Schaback a n d W e n d l a n d ( 2 0 0 0 b ) ] or C h a p t e r 3 3 ) . T h e p o w e r f u n c t i o n is also e m -
p l o y e d i n [De M a r c h i et al. (2005)] t o c o m p u t e a n o p t i m a l set o f R B F centers
i n d e p e n d e n t o f t h e specific d a t a values.
21.2 A d a p t i v e Least Squares using K n o t Removal
T h e idea o f k n o t r e m o v a l was p r i m a r i l y m o t i v a t e d b y t h e need for d a t a r e d u c t i o n ,

but i t c a n also be used for t h e p u r p o s e o f a d a p t i v e a p p r o x i m a t i o n (for a s u r v e y
o f k n o t r e m o v a l see, e.g., [Lyche (1992)]). T h e basic i d e a is t o s t a r t w i t h a g o o d
fit (e.g., a n i n t e r p o l a t i o n t o t h e d a t a ) , a n d t h e n successively reduce t h e n u m b e r o f
k n o t s used ( a n d t h e r e f o r e basis f u n c t i o n s ) u n t i l a c e r t a i n g i v e n t o l e r a n c e is reached.
Specifically, t h i s m e a n s w e w i l l s t a r t w i t h a n i n i t i a l fit a n d t h e n use some k i n d
o f w e i g h t i n g s t r a t e g y for t h e k n o t s , so t h a t w e c a n r e p e a t e d l y r e m o v e t h o s e c o n -
t r i b u t i n g least t o t h e a c c u r a c y o f t h e fit. T h e f o l l o w i n g a l g o r i t h m was suggested
i n [Fasshauer (1995a)] for a d a p t i v e least squares a p p r o x i m a t i o n o n spheres a n d
performs t h i s task.
A l g o r i t h m 21.2. K n o t removal
(1) L e t d a t a p o i n t s X = {x\,..., C C J V } , d a t a fi, i = 1 , . . . , N, a n d a t o l e r a n c e tol

be g i v e n .
(2) Choose M i n i t i a l k n o t s E = { i , . . . , M } -
(3) C a l c u l a t e a n i n i t i a l fit
M
x
Qf( ) = Y2 j( ,j)
c x
w i t h i t s associated least squares e r r o r
i=l
W h i l e e < tol do
(4) " W e i g h t " each k n o t j, j = 1 , . . . , M, a c c o r d i n g t o i t s least squares e r r o r ,
i.e., form
3 ' = 3 \ { f c } ,
and calculate the weights
i=l
where
M-l
Q}(x)= J 2 c
M x
^ j )
is t h e a p p r o x i m a t i o n based o n t h e r e d u c e d set o f k n o t s E*.

(5) F i n d t h e k n o t M w i t h lowest w e i g h t < tol a n d p e r m a n e n t l y r e m o v e i t ,
i.e.,
S = E\{ M } and M M 1.
(6) R e c a l c u l a t e f i t a n d associated e r r o r .
W e present a M A T L A B i m p l e m e n t a t i o n o f a k n o t r e m o v a l a l g o r i t h m t h a t is
s l i g h t l y m o r e efficient. I t s w e i g h t i n g s t r a t e g y is based o n t h e leave-one-out cross
v a l i d a t i o n a l g o r i t h m (see [ R i p p a (1999)] a n d C h a p t e r 17). T h e code is g i v e n i n
R B F K n o t R e m o v a l 2 D . m ( P r o g r a m 2 1 . 2 ) . T h i s p r o g r a m is s i m i l a r t o t h e k n o t inser-
tion program. I n fact, i t is a l i t t l e s i m p l e r since we do n o t have t o w o r r y a b o u t
multiple knots.
P r o g r a m 2 1 . 2 . RBFKnotRemove2D.m
% RBFKnotRemove2D
% Script that performs 2D RBF l e a s t squares approximation
% v i a knot removal
1 r b f = @(e,r) exp(-(e*r).~2); ep = 5.5;
% D e f i n e F r a n k e ' s f u n c t i o n as testfunction
2 f l = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x - 2 ) . ~ 2 + ( 9 * y - 2 ) . ~ 2 ) / 4 ) ;
3 f 2 = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x + l ) . ~ 2 / 4 9 + ( 9 * y + l ) . " 2 / 1 0 ) ) ;
4 f 3 = @(x,y) 0 . 5 * e x p ( - ( ( 9 * x - 7 ) . ~ 2 + ( 9 * y - 3 ) . " 2 ) / 4 ) ;
5 f 4 = @(x,y) 0 . 2 * e x p ( - ( ( 9 * x - 4 ) . ~ 2 + ( 9 * y - 7 ) . ~ 2 ) ) ;
6 testfunction = @(x,y) f 1 ( x , y ) + f 2 ( x , y ) + f 3 ( x , y ) - f 4 ( x , y ) ;
7 N = 289; g r i d t y p e = 'h';
8 M = 289; 7. Number of i n i t i a l c e n t e r s
9 n e v a l = 40;
11 epoints = [xe(:) y e ( : ) ] ;
12 t o l = 5 e - l ; % T o l e r a n c e ; stopping c r i t e r i o n
,
13 name = s p r i n t f (Data2D_/,d/,s' ,N,gridtype) ; load(name)
% Take f i r s t M "data s i t e s " as c e n t e r s
14 c t r s = d s i t e s d :M, : ) ;
7. Compute exact s o l u t i o n , i . e . , e v a l u a t e t e s t f u n c t i o n
% on e v a l u a t i o n p o i n t s
7 C r e a t e r i g h t - h a n d s i d e v e c t o r , i . e . ,
7, e v a l u a t e the t e s t f u n c t i o n a t t h e d a t a p o i n t s .
17 minres = 0;
18 w h i l e (minres < t o l )
7o Compute c o l l o c a t i o n m a t r i x
7o Compute r e s i d u a l
21 invCM = pinv(CM); EF = (invCM*rhs)./diag(invCM);
22 residual = abs(EF);
23 [ s r e s i d u a l , i d x ] = s o r t ( r e s i d u a l ) ; minres = r e s i d u a l ( 1 ) ;
7. Remove p o i n t
24 i f (minres < t o l )
25 ctrs = [ c t r s ( l : i d x ( l ) - l , : ) ; ctrs(idx(l)+l:M,:)];
26 M = M-l;
27 end
28 end
7o E v a l u a t e f i n a l l e a s t squares f i t
33 Pf = EM*(CM\rhs);
34 maxerr = max(abs(Pf - e x a c t ) ) ; rms_err = n o r m ( P f - e x a c t ) / n e v a l ;
35 f p r i n t f ('RMS e r r o r : 7.e\n' , r m s _ e r r )
36 figure; '/ P l o t d a t a s i t e s and c e n t e r s
37 plot(dsites(:,1),dsites(:,2),'bo',ctrs(:,1),ctrs(:,2),\r+');
38 c a p t i o n = s p r i n t f ( ' 7 o d d a t a s i t e s and 7od c e n t e r s ' , N, M) ;
39 title(caption);
40 PlotSurf(xe,ye,Pf,neval,exact.maxerr,[160,20]);
A g a i n we w o u l d l i k e t o c o m m e n t o n t h e a l g o r i t h m . A s far as c o m p u t a t i o n a l
t i m e s are concerned, A l g o r i t h m 21.2 as l i s t e d above is much slower t h a n t h e M A T -
L A B i m p l e m e n t a t i o n P r o g r a m 21.2 based o n t h e L O O C V i d e a since t h e w e i g h t for
every k n o t is d e t e r m i n e d b y t h e s o l u t i o n o f a least squares p r o b l e m , i.e., i n every
i t e r a t i o n one needs t o solve M least squares p r o b l e m s . T h e M A T L A B p r o g r a m r u n s
c o n s i d e r a b l y faster, b u t u s u a l l y i t is s t i l l slower t h a n t h e k n o t i n s e r t i o n a l g o r i t h m .
T h i s is clear since w i t h t h e k n o t r e m o v a l s t r a t e g y one s t a r t s w i t h l a r g e p r o b l e m s
t h a t get successively s m a l l e r , whereas w i t h k n o t i n s e r t i o n one begins w i t h s m a l l
p r o b l e m s t h a t c a n be solved q u i c k l y .
T h e o n l y w a y t h e k n o t r e m o v a l a p p r o a c h w i l l be b e n e f i c i a l is w h e n t h e n u m b e r
o f e v a l u a t i o n s o f t h e c o n s t r u c t e d a p p r o x i m a n t is m u c h l a r g e r t h a n i t s a c t u a l c o m p u -
t a t i o n . T h i s is so since, for c o m p a r a b l e tolerances, one w o u l d e x p e c t k n o t r e m o v a l
t o r e s u l t i n fewer k n o t s t h a n k n o t i n s e r t i o n . H o w e v e r , o u r e x a m p l e s s h o w t h a t t h i s
is n o t necessarily t r u e .
I f t h e i n i t i a l k n o t s are chosen a t t h e d a t a sites t h e n , a g a i n , t h e r e w i l l be n o
p r o b l e m s w i t h t h e c o l l o c a t i o n m a t r i x b e c o m i n g r a n k deficient.
I n [Fasshauer (1995a); Fasshauer ( 1 9 9 5 b ) ] some o t h e r a l t e r n a t i v e s t o t h i s k n o t
r e m o v a l s t r a t e g y were considered. O n e o f t h e m is t h e r e m o v a l o f c e r t a i n g r o u p s o f
k n o t s a t one t i m e i n o r d e r t o speed u p t h e process. A n o t h e r is based o n c h o o s i n g
t h e w e i g h t s based o n t h e size o f t h e coefficients Cj i n t h e e x p a n s i o n o f Q / , i.e., to
remove t h a t k n o t whose associated coefficient is smallest.
A f u r t h e r v a r i a t i o n o f t h e a d a p t i v e a l g o r i t h m s was c o n s i d e r e d i n b o t h [Franke
et al. (1994)] a n d i n [Fasshauer ( 1 9 9 5 a ) ] . I n s t e a d o f t r e a t i n g o n l y t h e coefficients
of t h e e x p a n s i o n o f Qf as p a r a m e t e r s i n t h e m i n i m i z a t i o n process, one c a n also
i n c l u d e t h e k n o t l o c a t i o n s t h e m s e l v e s a n d p o s s i b l y a ( v a r i a b l e ) shape p a r a m e t e r .
T h i s however, leads t o nonlinear least squares p r o b l e m s . W e w i l l n o t discuss t h i s
t o p i c f u r t h e r here.
Buhmann, Derrien, and Le Mehaute [ B u h m a n n et al. (1995); Le Mehaute
(1995)] also discuss k n o t r e m o v a l . T h e i r a p p r o a c h is based o n a n a priori esti-
mate for t h e e r r o r m a d e w h e n r e m o v i n g a c e r t a i n k n o t . These e s t i m a t e s d e p e n d o n

t h e specific choice o f r a d i a l basis f u n c t i o n , a n d o n l y cover t h e inverse m u l t i q u a d r i c
type, i.e.,
2 1 3
<p(r) = (1 + r ) ~ , 0 < p < s / 2 .
Iske (see [iske (1999a); Iske ( 2 0 0 4 ) ] ) suggests a n a l t e r n a t i v e k n o t r e m o v a l s t r a t -

egy for least squares a p p r o x i m a t i o n . H i s r e m o v a l h e u r i s t i c s are based o n so-called
thinning algorithms. I n p a r t i c u l a r , i n each i t e r a t i o n a p o i n t is r e m o v e d i f i t belongs
t o a p a i r o f p o i n t s i n S w i t h m i n i m a l s e p a r a t i o n d i s t a n c e . T h e t h i n n i n g phase o f t h e
a l g o r i t h m is t h e n e n h a n c e d b y a n exchange phase i n w h i c h p o i n t s c a n be "swapped
b a c k i n " i f t h i s process reduces t h e fill-distance o f S. This strategy maintains a
r e l a t i v e l y stable m e s h r a t i o .
21.3 Some Numerical Examples
For t h e f o l l o w i n g examples we consider F r a n k e ' s t e s t f u n c t i o n ( 2 . 2 ) . A l l final fits are

e v a l u a t e d o n a g r i d o f 40 x 40 e q u a l l y spaced p o i n t s i n t h e u n i t s q u a r e . R M S a n d
m a x i m u m e r r o r s are also c o m p u t e d o n t h i s g r i d . T h e p r o g r a m s RBFKnot I n s e r t 2D .m
( P r o g r a m 21.1) a n d RBFKnotRemove2D.m ( P r o g r a m 21.2) were used t o c o m p u t e t h e
results.
I n the t o p and m i d d l e parts o f F i g u r e 21.1 we compare the k n o t i n s e r t i o n a n d
k n o t r e m o v a l a l g o r i t h m s . I n b o t h cases w e use a reference set o f 289 H a l t o n d a t a
sites a n d G a u s s i a n basic f u n c t i o n s scaled w i t h a shape p a r a m e t e r e = 5.5. Using
t h e k n o t i n s e r t i o n a l g o r i t h m w i t h tol l e - 0 0 4 as i n P r o g r a m 21.1 w e select a subset
o f 154 d a t a sites as centers for t h e basis f u n c t i o n s . T h e s e p o i n t s are d i s p l a y e d o n
t h e left side o f t h e t o p p a r t o f F i g u r e 2 1 . 1 a l o n g w i t h t h e f i t o n t h e r i g h t . T h e k n o t
i n s e r t i o n a l g o r i t h m is i n i t i a l i z e d w i t h a single r a n d o m k n o t i n t h e u n i t s q u a r e .
Table 21.1 Total of knots selected, errors and runtimes for adaptive
least squares approximants.
Method N M RMS-error max-error time
Knot insert 289 154 1.33461 le-003 2.871526e-002 2.66

Knot remove 289 153 1.424598e-003 3.961593e-002 35.09
Knot insert 4225 163 1.198695e-004 1.137886e-003 48.75
I n t h e m i d d l e p a r t o f F i g u r e 2 1 . 1 w e s h o w t h e r e s u l t s for t h e same c o n f i g u r a t i o n ,
i.e., Gaussians w i t h e = 5.5 a n d N = 289 i n i t i a l k n o t s , for t h e k n o t removal
algorithm. T h i s t i m e we t a k e tol = 0.5, a n d t h e k n o t r e m o v a l a l g o r i t h m begins
w i t h a n i n t e r p o l a n t t o a l l 289 d a t a values. I n T a b l e 2 1 . 1 w e c a n c o m p a r e t h e e r r o r s
a n d r u n t i m e s for t h e t w o a l g o r i t h m s . I t is clear t h a t t h e k n o t i n s e r t i o n a l g o r i t h m
is m u c h m o r e efficient for t h i s e x a m p l e .
A n o t h e r a d v a n t a g e o f t h e k n o t i n s e r t i o n a l g o r i t h m is revealed i n t h e b o t t o m
p a r t o f F i g u r e 21.1 a n d l i n e 3 o f T a b l e 2 1 . 1 . W e s t i l l use Gaussians w i t h shape
p a r a m e t e r e = 5.5. H o w e v e r , n o w we t a k e N = 4225 H a l t o n p o i n t s as o u r d a t a
set. T h i s p r o v i d e s m a n y m o r e c a n d i d a t e s as centers for basis f u n c t i o n s . T h e chosen
centers are d i s p l a y e d o n t h e left o f t h e b o t t o m p a r t o f F i g u r e 2 1 . 1 , a n d i t is clear
t h a t t h i s center s e l e c t i o n is closely a d a p t e d t o t h e features o f t h e d a t a f u n c t i o n ,
n a m e l y t h e peaks a n d valley. T h e rest o f t h e k n o t s are l o c a t e d a l o n g t h e boundary
o f t h e d o m a i n . M o r e o v e r , we see t h a t i t is possible t o o b t a i n a m u c h m o r e a c c u r a t e
fit w i t h r o u g h l y t h e same n u m b e r o f basis f u n c t i o n s . W h i l e t h e r u n t i m e for t h i s
e x a m p l e is c o n s i d e r a b l y longer t h a n t h a t for t h e o t h e r k n o t i n s e r t i o n e x a m p l e i t
is c o m p a r a b l e t o t h e t i m e r e q u i r e d for t h e k n o t r e m o v a l a l g o r i t h m w i t h o n l y 289
d a t a sites. R u n n i n g t h e k n o t r e m o v a l a l g o r i t h m w i t h 4225 d a t a sites w o u l d be
p r o h i b i t i v e l y expensive.
10.025
1
0.8 o c
0.8
o o 10.02
o 1 0.6*
0.6 o o
o 8
O o 0.4. |0.015 2
Q ^
8
o a 8
o 0.2.
0.4 o
0- 10.01
BO
P o C
o
0.2 ffi o 10.005
8

) a,
SL_
8
0.2 0.4 0.6 0.8
x
o b o o
o e o
0.8 o
o
0.6 o
, 8
o 18 o
0.4
o o
o 8
0.2
0.2 0.4 0.6 0.8

x
1 1
Fig. 21.1 Top: 289 data sites and 154 knots (left) for least squares approximation to Franke's
function (right) using knot insertion algorithm with tol = le-004. Middle: 289 data sites and 153
knots (left) for least squares approximation to Franke's function (right) using knot removal algo-
rithm with tol = 0.5. Bottom: 4225 data sites and 163 knots (left) for least squares approximation
to Franke's function (right) using knot insertion algorithm with tol = le-004.
J
Chapter 22
Moving Least Squares Approximation
A n a l t e r n a t i v e t o r a d i a l basis f u n c t i o n i n t e r p o l a t i o n a n d a p p r o x i m a t i o n is t h e so-
called moving least squares ( M L S ) m e t h o d . A s w e w i l l see b e l o w , i n t h i s m e t h o d t h e
a p p r o x i m a t i o n Vf t o / is o b t a i n e d b y s o l v i n g m a n y ( s m a l l ) l i n e a r systems, i n s t e a d
o f v i a s o l u t i o n o f a single b u t l a r g e l i n e a r s y s t e m as w e d i d i n t h e previous
chapters.
22.1 Discrete Weighted Least Squares Approximation
I n o r d e r t o m o t i v a t e t h e m o v i n g least squares m e t h o d we b e g i n b y discussing d i s c r e t e

w e i g h t e d least squares a p p r o x i m a t i o n f r o m t h e space o f m u l t i v a r i a t e p o l y n o m i a l s .
T h u s , we consider d a t a (xi, f(xi)), i = 1 , . . . , N, w h e r e X{ e fl C R s
a n d f(xi) 6 M.
w i t h a r b i t r a r y s > 1. T h e a p p r o x i m a t i o n space is o f t h e f o r m
U = s p a n { p i , . . . ,p }, m m<N,
w i t h multivariate polynomials p m 6 11^ o f degree a t m o s t d.

W e i n t e n d t o find t h e b e s t d i s c r e t e w e i g h t e d least squares a p p r o x i m a t i o n f r o m
U t o some g i v e n f u n c t i o n / , i.e., w e need t o d e t e r m i n e t h e coefficients Cj i n
m
u(x) = ^ CjPj(x), X G R , s
3=1
such t h a t
11/ - u\\ , 2 w -> min.
H e r e t h e n o r m is defined v i a t h e d i s c r e t e (pseudo) i n n e r p r o d u c t
i=l
w i t h scalar w e i g h t s uii = w(xi), i = 1,... ,N. T h e i n d u c e d n o r m is t h e n o f t h e f o r m
li/lli^ = E[/(^)] M^)- 2
i=l
191
I t is w e l l k n o w n t h a t t h e best a p p r o x i m a t i o n u f r o m U t o / is c h a r a c t e r i z e d by
f - u w U (f-u,p )w k = 0, k = l,...,m,
m
(f ~ ^2cjPj,Pk)w = 0, k = l,...,m,
j=i
m
c
^2 j{PjiPk)w = (f,Pk)w, k = l,...,m,
3= 1
^ G c = f. p (22.1)
Here t h e G r a m m a t r i x G has entries Gjk = (j>j,Pk)w a n d t h e r i g h t - h a n d side vec-

T
t o r is f p = [{f,Pi) ,---,(f,Pm)w] -
w W e refer t o (22.1) as t h e normal equations
associated w i t h t h i s p r o b l e m .
A n o t h e r w a y t o t h i n k o f t h i s p r o b l e m w o u l d be as a p u r e linear a l g e b r a p r o b l e m .
T o t h i s end, define t h e Nxm m a t r i x A w i t h e n t r i e s A^ = pj(xi), a n d t h e vectors c =
T T
[ci,..., c] m a n d / = [f(xi),..., f(xisi)) . W i t h t h i s n o t a t i o n we seek a s o l u t i o n o f
t h e ( o v e r d e t e r m i n e d , since N > m) l i n e a r s y s t e m Ac = / . T h e s t a n d a r d w e i g h t e d
T
least squares s o l u t i o n is g i v e n b y t h e s o l u t i o n o f t h e n o r m a l e q u a t i o n s A WAc =
T
A Wf, w h e r e W is t h e d i a g o n a l w e i g h t i n g m a t r i x W = d i a g ( t u i , . . . ,WN). T h i s ,
however, is e x a c t l y w h a t is w r i t t e n i n ( 2 2 . 1 ) , i.e., t h e m a t r i x G is o f t h e form
T T
G = A WA, a n d for t h e r i g h t - h a n d side v e c t o r we have f p = A Wf.
22.2 S t a n d a r d I n t e r p r e t a t i o n of M L S A p p r o x i m a t i o n
Several equivalent f o r m u l a t i o n s exist for t h e m o v i n g least squares a p p r o x i m a t i o n

scheme. I n o r d e r t o m a k e a c o n n e c t i o n w i t h t h e discussion o f t h e d i s c r e t e w e i g h t e d
least squares a p p r o x i m a t i o n j u s t p r e s e n t e d we s t a r t w i t h t h e s t a n d a r d f o r m u l a t i o n
o f M L S a p p r o x i m a t i o n . T h e B a c k u s - G i l b e r t f o r m u l a t i o n t o be presented i n t h e f o l -
l o w i n g section w i l l have a closer c o n n e c t i o n t o p r e v i o u s c h a p t e r s since i t c o r r e s p o n d s
to a linearly constrained quadratic m i n i m i z a t i o n problem.
T h e general m o v i n g least squares m e t h o d first appeared i n the approximation
t h e o r y l i t e r a t u r e i n t h e p a p e r [Lancaster a n d Salkauskas (1981)] whose a u t h o r s also
p o i n t e d o u t t h e c o n n e c t i o n t o t h e earlier m o r e specialized w o r k [Shepard (1968);
M c L a i n (1974)]. W e n o w present a d e s c r i p t i o n o f M L S a p p r o x i m a t i o n t h a t is s i m -
i l a r t o t h e discussion i n L a n c a s t e r a n d Salkauskas' o r i g i n a l p a p e r a n d m o s t closely
resembles w h a t is f o u n d i n m u c h o f t h e o t h e r l i t e r a t u r e o n M L S a p p r o x i m a t i o n .
W e consider t h e f o l l o w i n g a p p r o x i m a t i o n p r o b l e m . A s s u m e we are g i v e n d a t a
values f(xi), i = 1 , . . . , N, o n some set X = {x\,..., XN} C I R o f d i s t i n c t d a t a
s
sites, w h e r e / is some ( s m o o t h ) f u n c t i o n , as w e l l as a n a p p r o x i m a t i o n space tl =

s p a n { w i , . . . ,u } m w i t h m < N. I n a d d i t i o n , we define a w e i g h t e d i 2 inner product
N
s
(/, 9)w y = ] T f(xi)g(xi)w(xi, y), y R fixed, (22.2)
22. Moving Least Squares Approximation 193
w h e r e n o w t h e w e i g h t f u n c t i o n s Wi = w(xi, ) , i = 1 , . . . , N, vary w i t h the point

y. Note t h a t the definition o f this inner p r o d u c t n a t u r a l l y introduces a second
v a r i a b l e , y, i n t o t h e discussion o f t h e p r o b l e m . This two-variable formulation of
MLS a p p r o x i m a t i o n w i l l be essential t o u n d e r s t a n d i n g t h e c o n n e c t i o n b e t w e e n t h e
various formulations.
A s i n t h e p r e v i o u s sections we w i s h t o find t h e best a p p r o x i m a t i o n u f r o m hi t o
/. However, we focus o u r i n t e r e s t o n best a p p r o x i m a t i o n at the point y, i.e., with
respect t o t h e n o r m i n d u c e d b y ( 2 2 . 2 ) . I n o r d e r t o keep t h e discussion as s i m p l e
as possible we w i l l r e s t r i c t o u r discussion t o t h e m u l t i v a r i a t e p o l y n o m i a l case, i.e.,
s
U = H d w i t h basis { p i , . . . ,p }- m A s always, t h e space of s-variate p o l y n o m i a l s
s d
of degree d has d i m e n s i o n m = ( ^ ) . W e emphasize, however, t h a t e v e r y t h i n g t h a t
is said b e l o w also goes t h r o u g h for a m o r e general linear a p p r o x i m a t i o n space Li.
Since we j u s t i n t r o d u c e d t h e second v a r i a b l e y i n t o o u r f o r m u l a t i o n we w i l l n o w
l o o k for t h e best a p p r o x i m a t i o n u i n t h e f o r m
m
c x x e R S 22 3
u(x, )y = J2 i(y)Pj( -2/)> >y - ( -)
3= 1
W e c a n t h i n k o f x as t h e global variable a n d y as t h e local variable. T h u s , expressing

t h e p o l y n o m i a l basis f u n c t i o n s i n t h i s f o r m is r e m i n i s c e n t o f a T a y l o r e x p a n s i o n .
T h i s shift t o t h e l o c a l e v a l u a t i o n p o i n t y also adds s t a b i l i t y t o n u m e r i c a l c o m p u t a -
tions.
F o r t h e p u r p o s e o f final e v a l u a t i o n o f o u r a p p r o x i m a t i o n w e i d e n t i f y t h e g l o b a l
a n d t h e l o c a l v a r i a b l e , i.e., w e have
m
s
V {x)
f = u{x,x) - ^2cj(x)pj(0), x GR . (22.4)
3= 1
Since for t h e p o l y n o m i a l a p p r o x i m a t i o n space w i t h s t a n d a r d m o n o m i a l basis w e

have pi(x) = 1, a n d Pj(0) = 0 for j > 1 we get t h e s t a n d a r d M L S a p p r o x i m a t i o n
i n t h e final f o r m
s
V {x) s = ci(x), xeM . (22.5)
N o t e , however, t h a t x has b e e n i d e n t i f i e d w i t h t h e fixed l o c a l p o i n t y, a n d t h e r e f o r e

i n general we s t i l l need t o r e c o m p u t e t h e coefficient C\ every t i m e t h e e v a l u a t i o n
p o i n t changes. E x a m p l e s for some c o m m o n choices o f s a n d d w i l l be p r o v i d e d i n
the next chapter.
A s i n t h e s t a n d a r d least squares case, t h e coefficients Cj(y) i n (22.3) are f o u n d
b y ( l o c a l l y ) m i n i m i z i n g t h e w e i g h t e d least squares e r r o r | | / u(-,y)\\ , Wy i.e.,
N
2
[ffri) - <Xi, y)} w(xi, y) (22.6)
i=l
is m i n i m i z e d over t h e coefficients i n t h e e x p a n s i o n (22.3) o f u(-,y). N o t e , however,
t h a t due t o t h e d e f i n i t i o n o f t h e i n n e r p r o d u c t (22.2) whose w e i g h t s " m o v e " w i t h
t h e l o c a l p o i n t y, t h e coefficients Cj i n (22.3) d e p e n d also o n y. A s a consequence

one has t o solve t h e normal equations
m
YCj(y)(pj(-y),Pk(--y))w v = (f,Pk(--y))w , v k = i,...,m, (22.7)
anew each t i m e t h e p o i n t y is c h a n g e d . I n m a t r i x n o t a t i o n ( 2 2 . 7 ) becomes
G(y)c(y) = f (y).p (22.8)
Here t h e p o s i t i v e d e f i n i t e G r a m m a t r i x G(y) has e n t r i e s
G(y)jk = (j>j{- - y),Pk(- - y))w v
N
= ^Pjixi - y)p (xik - y)w(xi,y), (22.9)
i=l
T
a n d t h e coefficient v e c t o r is o f t h e f o r m c(y) = [c\(y),..., c (y)] .
m O n the right-
h a n d side o f (22.8) we have t h e v e c t o r f (y) p = [ ( / , P i ( - - l / ) ) w , , ( / , P m ( -
w -
T
y))w ]v o f p r o j e c t i o n s o f t h e d a t a o n t o t h e basis f u n c t i o n s .
Several c o m m e n t s are c a l l e d for. F i r s t , t o ensure i n v e r t i b i l i t y o f t h e G r a m m a t r i x
we need t o i m p o s e a s m a l l r e s t r i c t i o n o n t h e set X o f d a t a sites. N a m e l y , X needs
t o be (i-unisolvent (c.f. D e f i n i t i o n 6.1). I n t h i s case t h e G r a m m a t r i x is s y m m e t r i c
a n d p o s i t i v e d e f i n i t e since t h e p o l y n o m i a l basis is l i n e a r l y i n d e p e n d e n t a n d the
w e i g h t s are p o s i t i v e . Second, t h e fact t h a t t h e coefficients c - d e p e n d o n t h e p o i n t3
y, a n d t h u s for every e v a l u a t i o n o f Vf a G r a m s y s t e m ( w i t h different m a t r i x G(y))

needs t o be solved, i n i t i a l l y scared p e o p l e a w a y f r o m t h e m o v i n g least squares
a p p r o a c h . H o w e v e r , for s m a l l values o f m , i.e., s m a l l p o l y n o m i a l degree d a n d s m a l l
space dimensions s, i t is possible t o solve t h e G r a m s y s t e m a n a l y t i c a l l y , a n d t h u s
a v o i d s o l v i n g l i n e a r systems a l t o g e t h e r . W e f o l l o w t h i s a p p r o a c h a n d present some
e x a m p l e s w i t h e x p l i c i t f o r m u l a s i n C h a p t e r 23 a n d use t h e m for o u r numerical
experiments later. M o r e o v e r , i f one chooses t o use c o m p a c t l y s u p p o r t e d weight
f u n c t i o n s , t h e n o n l y a few t e r m s are " a c t i v e " i n t h e s u m d e f i n i n g t h e entries o f
G(y) (c.f. (22.9)).
22.3 T h e B a c k u s - G i l b e r t A p p r o a c h to M L S A p p r o x i m a t i o n
T h e c o n n e c t i o n b e t w e e n t h e s t a n d a r d m o v i n g least squares f o r m u l a t i o n a n d B a c k u s -
G i l b e r t t h e o r y was p o i n t e d o u t i n [Bos a n d Salkauskas (1989)]. M a t h e m a t i c a l l y , i n
t h e B a c k u s - G i l b e r t a p p r o a c h one considers a quasi-interpolant of the form
N
V (x)
f = J2f(xi)*i(x), (22-10)
i=l
T
w h e r e / = [f(xi),..., /(ccjv)] represents t h e g i v e n d a t a .
Q u a s i - i n t e r p o l a t i o n is a g e n e r a l i z a t i o n o f t h e i n t e r p o l a t i o n idea. I f we use a l i n e a r
f u n c t i o n space s p a n { < & i , . . . , < & } t o a p p r o x i m a t e g i v e n d a t a { / ( c c i ) , . . . , / ( a j j v ) } ,
N
t h e n we saw earlier t h a t we c a n d e t e r m i n e coefficients c i , . . . , CJV such t h a t

N
w(ac) = ^2ci$i(x)
i=l
i n t e r p o l a t e s t h e d a t a , i.e., u{xi) = f(xi), i = 1 , . . . , N. I n p a r t i c u l a r , i f t h e basis
functions <&j are c a r d i n a l f u n c t i o n s , i.e., <&i(xj) = 5{j, i,j = 1,...,N, then the
coefficients are g i v e n b y t h e d a t a , i.e., Ci = f(xi), i = 1,..., N.
Now, for a general q u a s i - i n t e r p o l a n t we t a k e generating functions î, i =
1 , . . . , N ( w h i c h c a n be t h e same as t h e basis f u n c t i o n s <j) a n d f o r m t h e e x p a n s i o n
(22.10). T h i s e x p a n s i o n w i l l i n g e n e r a l n o l o n g e r i n t e r p o l a t e t h e d a t a , b u t i t w i l l
represent some f o r m o f a p p r o x i m a t i o n . I n o r d e r t o ensure t h a t a q u a s i - i n t e r p o l a n t
achieves a c e r t a i n a p p r o x i m a t i o n p o w e r one u s u a l l y requires t h a t t h e generating
functions r e p r o d u c e p o l y n o m i a l s o f a c e r t a i n degree. T h e same a p p r o a c h w i l l be
followed here, also (cf. ( 2 2 . 1 3 ) ) . T h e m a j o r a d v a n t a g e o f q u a s i - i n t e r p o l a t i o n over
i n t e r p o l a t i o n is t h e fact t h a t we n o longer have t o solve a ( p o t e n t i a l l y v e r y large)
s y s t e m o f linear e q u a t i o n s t o d e t e r m i n e t h e coefficients Cj. I n s t e a d , t h e y are g i v e n
directly by the data. W e w i l l n o w discuss a scheme t h a t tells us h o w t o choose
" g o o d " g e n e r a t i n g f u n c t i o n s \T/j.
As before, we consider t h e m o r e g e n e r a l f o r m u l a t i o n
N
u(x, y) = f(xi)î(x, y),
i=l
w i t h a global variable x a n d a l o c a l v a r i a b l e y. To o b t a i n the Backus-Gilbert
a p p r o x i m a n t we i d e n t i f y x a n d y, i.e.,
N
x K x x 2 2 n
V (x)
f = u(x, x) = J2 f( i) ( i> )i ( - )
1=1
w h e r e we n o w i n t r o d u c e d t h e n o t a t i o n K(xi, x) = î(x,x) w i t h a kernel K.

F r o m t h e discussion above a n d f r o m T h e o r e m 18.3 we k n o w t h a t t h e quasi-
i n t e r p o l a n t t h a t m i n i m i z e s t h e p o i n t - w i s e e r r o r is g i v e n i f t h e g e n e r a t i n g functions
î(-,y) are c a r d i n a l f u n c t i o n s (for fixed y).
I n t h e B a c k u s - G i l b e r t f o r m u l a t i o n o f t h e m o v i n g least squares m e t h o d one does
n o t a t t e m p t t o m i n i m i z e t h e p o i n t w i s e e r r o r , b u t i n s t e a d seeks for a fixed refer-
ence p o i n t y t o f i n d t h e values o f t h e g e n e r a t i n g f u n c t i o n s î(x, y) at t h e fixed
p o i n t x as t h e m i n i m i z e r s o f
k t ^ v ) ^ (22.12)
subject t o t h e p o l y n o m i a l r e p r o d u c t i o n constraints
N
x s
J2p( i - y)*i(x, y) = p(x - y), for all p e Il , d (22.13)
w h e r e 11^ is t h e space o f s-variate p o l y n o m i a l s o f t o t a l degree at m o s t d w i t h d i -

d s
m e n s i o n m = ( ^ ) - I f we a g a i n express t h e basis p o l y n o m i a l s b y p i , . . . , p , t h e n m
we c a n r e f o r m u l a t e (22.13) i n m a t r i x - v e c t o r f o r m as
A{y)V(x t y) = p(x - y), (22.14)
where A {y)
3i = p {xi
3 - y), j = 1, . . . , m , i = 1,...,N, V(x,y) =
T
[&i(x, y),..., ^^(x, y)] is t h e v e c t o r o f values o f t h e g e n e r a t i n g f u n c t i o n s , a n d
T
P [Pi) , P m ] is t h e v e c t o r o f basis p o l y n o m i a l s . T h e c o r r e s p o n d i n g m a t r i x -
vector f o r m u l a t i o n o f (22.12) is
T
l* (x,y)Q(y)*(x,y), (22.15)
where
a n d t h e w(xi, ) are p o s i t i v e weight functions ( a n d t h u s for a n y fixed y t h e m a t r i x

Q(y) is p o s i t i v e d e f i n i t e ) .
I n t h e above f o r m u l a t i o n t h e r e is n o e x p l i c i t e m p h a s i s o n nearness o f fit as t h i s
is i m p l i c i t l y o b t a i n e d b y t h e q u a s i - i n t e r p o l a t i o n "ansatz" a n d t h e closeness o f t h e
g e n e r a t i n g f u n c t i o n s t o t h e p o i n t w i s e o p t i m a l d e l t a f u n c t i o n s . T h i s is achieved b y
t h e above p r o b l e m f o r m u l a t i o n i f t h e w(xi, ) are w e i g h t f u n c t i o n s t h a t decrease w i t h
distance f r o m t h e o r i g i n . T h e s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s used earlier
are c a n d i d a t e s for these w e i g h t f u n c t i o n s . H o w e v e r , s t r i c t p o s i t i v e definiteness is
n o t r e q u i r e d at t h i s p o i n t , so t h a t , e.g., ( r a d i a l or t e n s o r p r o d u c t ) S - s p l i n e s c a n
also be used. A s m e n t i o n e d earlier, t h e p o l y n o m i a l r e p r o d u c t i o n c o n s t r a i n t is a d d e d
so t h a t t h e q u a s i - i n t e r p o l a n t w i l l achieve a desired a p p r o x i m a t i o n o r d e r . T h i s w i l l
become clear i n C h a p t e r 25.
I n p u r e linear algebra n o t a t i o n t h e B a c k u s - G i l b e r t a p p r o a c h corresponds t o
finding t h e m i n i m u m n o r m s o l u t i o n o f a n u n d e r d e t e r m i n e d linear s y s t e m , i.e., we
w a n t t o solve t h e p o l y n o m i a l r e p r o d u c t i o n c o n s t r a i n t s
A(y)^(x,y) =p(x-y)
w i t h m x N ( m < N) s y s t e m m a t r i x . T h e n o r m o f t h e s o l u t i o n v e c t o r is a w e i g h t e d
n o r m t h a t varies w i t h t h e (fixed) reference p o i n t y a n d is m e a s u r e d as i n ( 2 2 . 1 5 ) .
I n o t h e r w o r d s , t h e B a c k u s - G i l b e r t f o r m u l a t i o n guarantees t h a t we find t h e "best"
system of generating functions w i t h local p o l y n o m i a l r e p r o d u c t i o n properties, where
"best" is m e a s u r e d i n t e r m s o f t h e n o r m ( 2 2 . 1 5 ) .
T h e q u a d r a t i c f o r m (22.12) (or e q u i v a l e n t l y ( 2 2 . 1 5 ) ) c a n also be i n t e r p r e t e d as
a smoothness f u n c t i o n a l . I t s use is also m o t i v a t e d b y p r a c t i c a l a p p l i c a t i o n s . In
t h e B a c k u s - G i l b e r t t h e o r y , w h i c h was d e v e l o p e d i n t h e c o n t e x t o f geophysics (see
[Backus a n d G i l b e r t ( 1 9 6 8 ) ] ) , i t is desired t h a t t h e g e n e r a t i n g f u n c t i o n s are as
close as possible t o t h e i d e a l c a r d i n a l f u n c t i o n s ( i . e . , d e l t a f u n c t i o n s ) . Therefore,
one needs t o m i n i m i z e t h e i r " s p r e a d " . The polynomial reproduction constraints
are a g e n e r a l i z a t i o n o f a n o r i g i n a l n o r m a l i z a t i o n c o n d i t i o n w h i c h c o r r e s p o n d s t o
reproduction of constants only.
For any c o m b i n a t i o n o f a f i x e d ( e v a l u a t i o n ) p o i n t x a n d a fixed (reference)
p o i n t y the c o m b i n a t i o n of ( 2 2 . 1 2 ) and ( 2 2 . 1 3 ) (or e q u i v a l e n t l y ( 2 2 . 1 5 ) a n d (22.14))
present j u s t a n o t h e r c o n s t r a i n e d q u a d r a t i c m i n i m i z a t i o n p r o b l e m o f t h e f o r m dis-
cussed i n p r e v i o u s chapters.
According to our earlier work we use Lagrange multipliers X(x,y) =
T
[Xi(x, y),...,X (x,y)]
m ( d e p e n d i n g o n x a n d y), a n d t h e n k n o w t h a t (c.f. (19.4)
a n d (19.5))
1 T 1
X(x,y) = (A(y)Q- (y)A (y)y p(x-y) (22.17)
1 T
*(x,y) = Q- (y)A (y)X(x,y). (22.18)
E q u a t i o n ( 2 2 . 1 8 ) tells us h o w t o c o m p u t e t h e g e n e r a t i n g f u n c t i o n s for ( 2 2 . 1 1 ) , i.e.,

i f we w r i t e ( 2 2 . 1 8 ) c o m p o n e n t w i s e then
m
Vi(x,y) = w(xi,y)^T\ (x,y)p (x j j i - y), i = l,...,N. (22.19)
J=I
Therefore, once t h e values o f t h e L a g r a n g e m u l t i p l i e r s Xj(x, y), j = 1 , . . . , N, have
been d e t e r m i n e d we have e x p l i c i t f o r m u l a s for t h e values o f t h e g e n e r a t i n g f u n c t i o n s .
I n p a r t i c u l a r , we get
N
V/(x) = u(x,x) = y^f(xi)^fi(x,x)
^2f( i)w(xi,x)Y2^j( ^ )Pj( i

N m
x x x x x
~ )i i = l,...,N,
i=i j=i
N
= x K x x
^,f( i) ( ii )
i=l
T
w i t h kernels K(xi, x) = w(xi, x)\ (x, x)p(xi x).
I n general, finding t h e L a g r a n g e m u l t i p l i e r s involves s o l v i n g a ( s m a l l ) l i n e a r
s y s t e m t h a t changes as soon as t h e reference p o i n t y changes (see ( 2 2 . 1 7 ) ) . U s i n g
e q u a t i o n ( 2 2 . 1 7 ) , t h e L a g r a n g e m u l t i p l i e r s are o b t a i n e d as t h e s o l u t i o n o f a G r a m
system
G(y)X(x,y)=p(x-y), (22.20)
where t h e entries o f t h e m x m m a t r i x G(y) are t h e w e i g h t e d 2 inner products

N
Gjk(y) = (pj(- - y),Pk(- - y))w y = E^"^ ~ x

y)Pk( i - y)w{xi,y). (22.21)
N o t e t h a t t h i s is i d e n t i c a l t o t h e m a t r i x G(y) needed for t h e d e t e r m i n a t i o n o f t h e

coefficients c(y) i n t h e s t a n d a r d M L S a p p r o a c h (c.f. (22.8) and (22.9)). Equation
(22.20) shows us t h a t for a fixed reference p o i n t y t h e L a g r a n g e m u l t i p l i e r s
are p o l y n o m i a l s , i.e., Xj(-, y), j 1,... ,m, are p o l y n o m i a l s .
22.4 Equivalence of the T w o Formulations o f M L S A p p r o x i m a t i o n
We n o w show t h a t t h e t w o f o r m u l a t i o n s o f m o v i n g least squares a p p r o x i m a t i o n

d e s c r i b e d i n t h e p r e v i o u s t w o sections are e q u i v a l e n t , i.e., we s h o w t h a t Vf(x)
c o m p u t e d v i a (22.4) a n d (22.11) are t h e same. T h e a p p r o x i m a n t i n t h e s t a n d a r d
m o v i n g least squares f o r m u l a t i o n (22.4) establishes u(x,y) i n the form
m
T
u(x,y) = ^2cj(y)pj(x-y) = p \x - y)c{y),
T T
w h e r e p = [ p i , . . . ,p ] m and c = [ c i , . . . , c m ] .
I n (22.8) we w r o t e t h e n o r m a l e q u a t i o n s for t h i s a p p r o a c h as
G(y)c(y) = f (y).
p
N o t e t h a t t h e r i g h t - h a n d side v e c t o r f (y) p c a n be w r i t t e n as
T
f (y)
P = - y))w ,- v , < / , P m ( - - y))w ]
v
= A(y)Q-\y)f (22.22)
1
w i t h t h e m a t r i x Q~ (y) used i n t h e B a c k u s - G i l b e r t f o r m u l a t i o n . T h i s i m p l i e s
1
c{y) = G- {y)A{y)Q-\y)f.
T h u s , u s i n g t h e s t a n d a r d a p p r o a c h , we get
T T
u(x, y) = p (x - y)c(y) = p (x - y)G-\y)A{y)Q-\y)f. (22.23)
For t h e B a c k u s - G i l b e r t a p p r o a c h , o n t h e o t h e r h a n d , t h e "ansatz" is o f t h e f o r m
(22.11)
N
T
u(x,y) =J2f(x )t> (x,y)
i i = V (x,y)f,
T T
w h e r e as before ^(x,y) = [$i(x,y),.. .,fy (x,y)]
N and / = [ / ( x ) , . . . ,
x f(x )] .
N
For t h i s a p p r o a c h we d e r i v e d (see (22.17) a n d ( 2 2 . 1 8 ) )

1
X(x, y) = G~ {y)p(x - y)
T
V(x,y) = Q-\y)A {y)\(x,y),
1 T
w h e r e G(y) = A(y)Q~ (y)A \y) (see (22.21) o r ( 2 2 . 9 ) ) . T h e r e f o r e , we n o w o b t a i n
for t h e B a c k u s - G i l b e r t a p p r o x i m a n t
T 1 T 1 T
u(x,y) = V (x,y)f = [Q- (y)A (y)G- (y)p(x-y)] f
w h i c h , b y t h e s y m m e t r y o f Q(y) a n d G(y), is t h e same as ( 2 2 . 2 3 ) . C l e a r l y , we also

have equivalence w h e n we e v a l u a t e e i t h e r r e p r e s e n t a t i o n at y = x, i.e., consider
Vf(x) = u{x, x).
T h e equivalence o f t h e t w o approaches shows t h a t t h e m o v i n g least squares
a p p r o x i m a n t has a l l o f t h e f o l l o w i n g p r o p e r t i e s :
I t reproduces a n y p o l y n o m i a l o f degree at m o s t d i n s variables e x a c t l y .

r
I t p r o d u c e s t h e best l o c a l l y w e i g h t e d least squares f i t .

V i e w e d as a q u a s i - i n t e r p o l a n t , t h e g e n e r a t i n g f u n c t i o n s are as close as pos-
sible t o t h e o p t i m a l c a r d i n a l basis f u n c t i o n s a m o n g a l l f u n c t i o n s t h a t produce
p o l y n o m i a l s o f t h e desired degree i n t h e sense t h a t (22.12) is m i n i m i z e d .
Since p o l y n o m i a l s are i n f i n i t e l y s m o o t h , e i t h e r o f t h e r e p r e s e n t a t i o n s o f Vf
shows t h a t i t s s m o o t h n e s s is d e t e r m i n e d b y t h e s m o o t h n e s s o f t h e w e i g h t func-
t i o n ^ ) Uli = w(xi, ).
I n particular, the standard m o v i n g least squares m e t h o d w i l l r e p r o d u c e the

p o l y n o m i a l basis f u n c t i o n s p i , . . . , p m even t h o u g h t h i s is n o t e x p l i c i t l y enforced b y
the m i n i m i z a t i o n (solution of the n o r m a l equations). Moreover, the more general
"ansatz" w i t h ( n o n - p o l y n o m i a l ) a p p r o x i m a t i o n space U a l l o w s us t o b u i l d m o v i n g
least squares a p p r o x i m a t i o n s t h a t also r e p r o d u c e a n y o t h e r f u n c t i o n t h a t is i n c l u d e d
i n U. T h i s c a n be v e r y b e n e f i c i a l for t h e s o l u t i o n o f p a r t i a l d i f f e r e n t i a l e q u a t i o n s w i t h
k n o w n s i n g u l a r i t i e s (see, e.g., t h e p a p e r s [ B a b u s k a a n d M e l e n k ( 1 9 9 7 ) ; B e l y t s c h k o
et al. ( 1 9 9 6 ) ] ) .
22.5 D u a l i t y a n d B i - O r t h o g o n a l Bases
F r o m the Backus-Gilbert f o r m u l a t i o n we k n o w t h a t
1
G(y)X(x,y)=p(x-y) X(x, y) = G' (y)p(x - y). (22.24)
B y t a k i n g m u l t i p l e r i g h t - h a n d sides p(x y) w i t h x X = {x\,... ,x^} w e get

1
[ A ( x i , y ) , . . .,X(x ,y)]
N = G~ {y)\p{x 1 - y ) , . . . ,p{x N - y)}
or
A(y) = G-\y)A(y), (22.25)
w h e r e A(y) is t h e p o l y n o m i a l m a t r i x (22.14) f r o m a b o v e a n d t h e m x N matrix

A(y) has entries A {y) 3i = Xj(xi,y).
The standard M L S f o r m u l a t i o n , o n t h e o t h e r h a n d , gives us (see (22.8) a n d
(22.22))
1 1
G{y)c{y) = f (y)
p <=> c(y) = G- {y)A{y)Q- {y)f. (22.26)
B y c o m b i n i n g (22.25) w i t h ( 2 2 . 2 6 ) w e get
1
c ( y ) = G-\y)A{y)Q-\y)f = A(y)Q~ (y)f = f {y),
x
w h e r e f\(y) is d e f i n e d a n a l o g o u s l y t o f (y) p {c.f. (22.22)). Componentwise this

gives us
Cjiv) = (f,Xj(-,y)) , Wy j = l,...,m,

a n d therefore,
3= 1
I t is also possible t o f o r m u l a t e t h e m o v i n g least squares m e t h o d b y u s i n g t h e

L a g r a n g e m u l t i p l i e r s o f t h e B a c k u s - G i l b e r t a p p r o a c h as basis f u n c t i o n s for t h e ap-
p r o x i m a t i o n space U. T h e n , u s i n g t h e same a r g u m e n t a t i o n as above, w e e n d u p
with
m
u(x,y) = Y^d (y)\ (x,y) j j (22.28)
3=1
with
d
j(y) = (fiPji- -y))w , y j = l,...,m.
W e c a n v e r i f y t h i s b y a p p l y i n g (22.20) a n d (22.22) t o ( 2 2 . 2 3 ) , i.e.,
T 1 1
u{x, y) = p (x - y)G- (y)A(y)Q- (y)f.
We then obtain
T
u(x,y) = X (x,y)f (y),
p
w h i c h corresponds t o (22.28).
T h e c a l c u l a t i o n s j u s t p r e s e n t e d s h o w t h a t t h e L a g r a n g e m u l t i p l i e r s f o r m a basis
t h a t is dual t o t h e p o l y n o m i a l basis. I n p a r t i c u l a r , i f we r e c a l l t h e M L S a p p r o x i m a n t
i n its standard representation
m
u{x,y) = ^2(f,Xj(;y))w Pj(x y - y)

3=1
a n d let / = p {- k y), then the p o l y n o m i a l reproduction property of the method

ensures
m
^2(Pk(- -y),\j(-,y)) Pj{x-y) Wy =p (x-y).
k
3= 1
T h i s , however, i m p l i e s
(Pfc(- - y),Xj(-,y)) Wy = S, jk j,k = l,...,m. (22.29)
T h e r e f o r e we have t w o sets o f basis f u n c t i o n s t h a t are bi-orthogonal w i t h respect
t o t h e special w e i g h t e d i n n e r p r o d u c t ( - , - ) W y o n t h e set X. We will illustrate this
d u a l i t y i n C h a p t e r 24.
E a r l i e r we d e r i v e d t h e r e p r e s e n t a t i o n (c.f. (22.18))
T
9{x,y) =Q-\y)A (y)\{x,y)
for t h e g e n e r a t i n g f u n c t i o n s i n t h e B a c k u s - G i l b e r t f o r m u l a t i o n . Since t h e L a g r a n g e
1
m u l t i p l i e r s are g i v e n b y X(x,y) = G~ (y)p(x y) (see ( 2 2 . 2 0 ) ) we get
T 1 - 1 T
*(x,y) = Q-\y)A {y)G- {y)p{x - y) = Q (y)A (y)p(x - y)
due t o ( 2 2 . 2 5 ) . T h u s , t h e B a c k u s - G i l b e r t r e p r e s e n t a t i o n is g i v e n also b y t h e d u a l
representation
N .f .
T
u(x, y) = f{xi)w(xi, y)\ {xi, y)p(x - y). (22.30)
B y also c o n s i d e r i n g t h e t w o d u a l e x p a n s i o n s (22.28) a n d (22.30) we n o w h a v e

four a l t e r n a t i v e r e p r e s e n t a t i o n s for t h e m o v i n g least squares q u a s i - i n t e r p o l a n t . This
is s u m m a r i z e d i n t h e f o l l o w i n g t h e o r e m .
T h e o r e m 2 2 . 1 . Let f : Cl R be some function whose values on the d-unisolvent

s
set of points X = {xi}fL 1 C R are given as data. Let p\,.. ,Pm be a basis for H d
withpi(x) = I, let {w(xi, ) , . . . ,W(XN, )} & e a s e

^ of positive weight functions, and
let Xj, j = 1 , . . . ,m, be the Lagrange multipliers defined by (22.17). Furthermore,
consider the generating functions
Vi(x,y) =w(x ,y)^2x (x,y)pj(x

i j i -y), i = l,...,N,
3=1
or in dual form
m
Vi(x,y) = w(x ,y)^2x (x ,y)p (x

i j i j -y), i = l,...,N.
3= 1
The best local least squares approximation to f on X in the sense of (22.6) is

given by Vf(x) = u(x,x), where
x x
u{x,y) = E(/> j(->y)) Pj( v - v)
3= 1
m
= ^2(f:Pj(--y))w Xj(x,y) v
3= 1
N
X i x
= Y l ^ ) ^ ^ ^ y y
i=l
This results in the four representations
m
Vf{x) = ^(f^j^xVw.PjiO) = (/, A i ( - , x ) ) W m = a(x)

3= 1
m
x
= *52{f>Pj(- - ))w^Xj(x,x)
3= 1
N m
= ^2f(x )w(xi,x)^2x (x,x)p (x

i j j i - x)
i=l j=l
N m N
= ^2f{x )w(x ,x)^2X (x ,x)p (0)
i i j i j = ^ f(Xj)w(Xj, x)\i(Xj, x).
i=l 3= 1 i=l
N o t e t h a t t h e first t w o e x p a n s i o n s for Vf{x) i n T h e o r e m 22.1 c a n be v i e w e d as

generalizations o f ( f i n i t e ) e i g e n f u n c t i o n o r F o u r i e r series expansions.
22.6 Standard M L S Approximation as a C o n s t r a i n e d Quadratic

Optimization Problem
F i n a l l y , i t is also possible t o f o r m u l a t e t h e s t a n d a r d M L S a p p r o a c h as a c o n -
strained quadratic minimization problem. To this end we (artificially) introduce
the quadratic functional
m m
C C G
T
c (y)G(y)c(y) = EE ^) ^) ^)
j=l k=l
m m
= ^2^2cj(y)c (y)(pj(- k - y),p (-k - y))w v
3= 1 k=l
w h i c h s h o u l d be i n t e r p r e t e d as t h e ( y - d e p e n d e n t ) n a t i v e space n o r m o f t h e a p p r o x -
m
i m a n t u(x,y) = ^^Cj(y)pj(x y). T h e G r a m s y s t e m (22.8) c a n be w r i t t e n i n
3= 1
m a t r i x - v e c t o r f o r m as
1
G(y)c{y)=A(y)Q- (y)f "
w h e r e Q(y) is t h e d i a g o n a l m a t r i x o f w e i g h t f u n c t i o n s (22.16) a n d A(y) is t h e m a t r i x

o f p o l y n o m i a l s (22.14) used e a r l i e r .
M i n i m i z a t i o n o f t h e q u a d r a t i c f o r m s u b j e c t t o t h e l i n e a r side c o n d i t i o n s is e q u i v -
alent t o m i n i m i z a t i o n o f (for f i x e d y)
T T
\c {y)G{y)c{y) - / x ( y ) [G(y)c(y) - A(y)Q-\y)f] , (22.31)
w h e r e fjt(y) is a v e c t o r o f L a g r a n g e m u l t i p l i e r s .
T h e solution of the linear system resulting f r o m the m i n i m i z a t i o n p r o b l e m
(22.31) gives us
T 1 1 T
/ * ( y ) = {G{y)G-\y)G {y)Y A{y)Q- {y)f = G- {y)A{y)Q-\y)f
1 T
c(y) = G- (y)G (y) x(y) t = //(</)
so t h a t as i n t h e case o f r a d i a l basis f u n c t i o n i n t e r p o l a t i o n t h e quadratic

T
f u n c t i o n a l c (y)G(y)c(y) is a u t o m a t i c a l l y m i n i m i z e d b y s o l v i n g o n l y t h e Gram
s y s t e m G(y)c(y) = f (y).
p
22.7 Remarks
I n t h e s t a t i s t i c s l i t e r a t u r e t h e m o v i n g least squares a p p r o a c h is k n o w n as local
(polynomial) regression. G o o d sources o f i n f o r m a t i o n are t h e b o o k [Fan a n d G i -

j b e l s (1996)] a n d t h e r e v i e w a r t i c l e [ C l e v e l a n d a n d L o a d e r (1996)] a c c o r d i n g to
w h i c h t h e basic ideas o f l o c a l regression c a n b e t r a c e d b a c k at least t o w o r k d o n e
i n t h e late 1 9 t h c e n t u r y b y [ G r a m ( 1 8 8 3 ) ; W o o l h o u s e (1870); D e F o r e s t ( 1 8 7 3 ) ;
D e Forest (1874)]. I n p a r t i c u l a r , i n t h e s t a t i s t i c s l i t e r a t u r e one learns t h a t t h e use
of least squares a p p r o x i m a t i o n is j u s t i f i e d w h e n t h e d a t a / i , . , /AT are n o r m a l l y

d i s t r i b u t e d , whereas, i f t h e noise i n t h e d a t a is n o t G a u s s i a n , t h e n o t h e r c r i t e r i a
s h o u l d be used. See, e.g., t h e s u r v e y a r t i c l e [ C l e v e l a n d a n d L o a d e r (1996)] for m o r e
details.
W e close b y e s t a b l i s h i n g a c o n n e c t i o n b e t w e e n t h e p o l y n o m i a l r e p r o d u c t i o n
constraints and certain moment conditions. Recall the p o l y n o m i a l reproduction
c o n s t r a i n t s (22.14) i n t h e B a c k u s - G i l b e r t f o r m u l a t i o n
A(y)W(x,y) =p(x - y).
B y s e t t i n g y = x we get
A(x)V{x, x) = p(0). (22.32)
Since we defined t h e kernels K{xi, x) = î(x, x) earlier, a n d since we assume t h a t

t h e p o l y n o m i a l s basis is such t h a t p\{x) = 1 a n d Pj(0) = 0 for j > 1 we get f r o m
(22.32)
N
^ P f c ( i - x)K{xi, x) = <5i , fc k = 1 , . . . , m.
T h i s comprises a set o f discrete moment conditions for t h e k e r n e l K. Since t h e r e

are o n l y m c o n d i t i o n s for t h e N k e r n e l values K(xi, x) a t a fixed p o i n t x, t h e k e r n e l
is n o t u n i q u e l y d e t e r m i n e d b y these m o m e n t c o n d i t i o n s . I f , however, we a d d t h e
least n o r m c o n s t r a i n t f r o m t h e B a c k u s - G i l b e r t f o r m u l a t i o n , i.e., we m i n i m i z e
t h e n we get t h e s t a n d a r d m i n i m u m n o r m s o l u t i o n for u n d e r d e t e r m i n e d least squares

problems. W e also p o i n t o u t t h a t we d e r i v e d earlier (see T h e o r e m 22.1) t h a t the
k e r n e l K(-,x) is o f t h e f o r m
K(-,X) = X (-,x)w(;X)
1
w i t h p o l y n o m i a l t e r m A i ( - , x) a n d w e i g h t f u n c t i o n w{-, x). T h u s , we have a u n i q u e

s o l u t i o n once t h e w e i g h t f u n c t i o n w(-, x) has b e e n chosen.
T h e i n t e r p r e t a t i o n o f M L S a p p r o x i m a t i o n w i t h t h e h e l p o f m o m e n t m a t r i c e s is
used i n t h e e n g i n e e r i n g l i t e r a t u r e (see, e.g., [ L i a n d L i u ( 2 0 0 2 ) ] ) , a n d also plays
a n essential role w h e n c o n n e c t i n g m o v i n g least squares a p p r o x i m a t i o n t o t h e m o r e
efficient concept o f approximate approximation [Maz'ya and Schmidt (2001)]. For
a discussion o f a p p r o x i m a t e m o v i n g least squares a p p r o x i m a t i o n see [Fasshauer
(2002c); Fasshauer ( 2 0 0 2 d ) ; Fasshauer ( 2 0 0 3 ) ; Fasshauer (2004)] or C h a p t e r 26.
lISJ
Chapter 23
Examples of M L S Generating Functions
23.1 Shepard's M e t h o d
T h e m o v i n g least squares m e t h o d i n t h e case d = 0 ( a n d therefore m = 1) w i t h

pi{x) = 1 is u s u a l l y referred t o as Shepard's method [Shepard ( 1 9 6 8 ) ] . I n the
statistics l i t e r a t u r e Shepard's m e t h o d is also k n o w n as a kernel method (see, e.g., t h e
papers f r o m t h e 1950s a n d 60s [ R o s e n b l a t t (1956); P a r z e n (1962); N a d a r a y a (1964);
W a t s o n ( 1 9 6 4 ) ] ) . U s i n g o u r n o t a t i o n we have
V (x)
f = (x).
Cl
T h e G r a m " m a t r i x " (22.9) consists o f o n l y one element

N
G(x) = (pi(- - aj),pi(- - x)) Wm = ^2w(xi,x)
i=l
so t h a t
G(x)c(x) = f (x)
p
implies
N
Y2f(xi)w(xi,x)
ci(x) = ^ . (23.1)
Y^w(xj,x)
3= 1
I f we c o m p a r e (23.1) w i t h the Backus-Gilbert quasi-interpolation formulation
(22.11) we i m m e d i a t e l y see t h a t t h e g e n e r a t i n g f u n c t i o n s are g i v e n b y
{ X i x )
*<(x,x)= ' . (23.2)
^2w{xj,x)
3= 1
Since
f x ^ - f ;
i=l i=l
N
wix
" I
x)
\
= i (23.3)
2^w(xj,x)
3= 1
205
i n d e p e n d e n t o f x, Shepard's m e t h o d is also k n o w n as a partition of unity method.

T h e weight functions can take m a n y forms. I n p r a c t i c e one u s u a l l y takes a
single basic w e i g h t f u n c t i o n w w h i c h is t h e n s h i f t e d t o t h e d a t a sites. Often the
basic w e i g h t f u n c t i o n is also a r a d i a l f u n c t i o n ( i n e i t h e r t h e E u c l i d e a n or m a x i m u m
n o r m ) . W e w i l l use r a d i a l f u n c t i o n s as basic w e i g h t s i n o u r n u m e r i c a l experiments
below, i.e.,
Vj(Xi, X) = W(\\X - Xi\\)
w h e r e w is one o f o u r ( s t r i c t l y p o s i t i v e d e f i n i t e ) r a d i a l basic f u n c t i o n s . B o t h c o m -
p a c t l y s u p p o r t e d a n d g l o b a l l y s u p p o r t e d f u n c t i o n s w i l l be used.
T h e d u a l S h e p a r d basis is defined b y (see ( 2 2 . 2 4 ) )
G{y)X(x,y) = p(x - y)
so t h a t
Ai(cc,a;) -
N
^2 w{x ,x)
/ i
i=l
a n d (c.f. (22.28))
V (x)
f = di(a;)Ai(a;,aj) (23.4)
with
N
di(x) = (f,pi(- - x)) Wm = ^2f(xi)w(xi,x).

i=l
T h e e x p l i c i t d u a l r e p r e s e n t a t i o n o f t h e S h e p a r d a p p r o x i m a n t is, o f course, i d e n t i c a l
t o (23.1).
T h e g e n e r a t i n g f u n c t i o n s (22.19) are defined as
a? a?
î(x, x) = w(xi, x)Xi(x, x)pi(xi - x) = ^( ' ) ;
^ w { x 3 , x )
3= 1
w h i c h matches o u r earlier e x p r e s s i o n ( 2 3 . 2 ) . T h i s once m o r e gives rise t o t h e w e l l -

k n o w n q u a s i - i n t e r p o l a t i o n f o r m u l a for S h e p a r d ' s m e t h o d
N
V (x)
f = ^2f(x )V (x,x)i i
i=l
w ( X u X )
= /(*<) N
Y2w(xj,x)
3= 1
W e n o w also have b i - o r t h o g o n a l i t y o f t h e basis a n d d u a l basis, i.e.,
(Xi(-,x),pi(- - x)) Wa! = 1.

23. Examples of MLS Generating Functions 207
Indeed
N
( A I ( - , S B ) , P I ( - - x)) Ww = y^ X (x ,x)w(x ,x)
j 1 i i
i=l
w(xj, x)
23.2 M L S Approximation with Nontrivial Polynomial

Reproduction
W h i l e i t is always possible t o s i m p l y solve t h e l o c a l G r a m systems (22.8) or (22.20)

and therefore i m p l i c i t l y c o m p u t e t h e g e n e r a t i n g f u n c t i o n s î, i = l,...,N, i t is
often o f interest t o have e x p l i c i t f o r m u l a s for ^ j . W e saw i n t h e p r e v i o u s s e c t i o n
t h a t i n t h e case o f r e p r o d u c t i o n o f c o n s t a n t s we a r r i v e at S h e p a r d ' s m e t h o d w h i c h
is v a l i d i n d e p e n d e n t o f t h e space d i m e n s i o n . H o w e v e r , i f t h e degree d o f p o l y n o m i a l
d s
r e p r o d u c t i o n is n o n t r i v i a l (i.e., d > 0 ) , t h e n t h e size m = ( ^ ) o f t h e G r a m systems,
and therefore t h e r e s u l t i n g f o r m u l a s for t h e g e n e r a t i n g f u n c t i o n s w i l l d e p e n d o n t h e
space d i m e n s i o n s.
W e n o w present t h r e e e x a m p l e s w i t h e x p l i c i t f o r m u l a s for t h e M L S g e n e r a t i n g
functions.
To s i m p l i f y t h e n o t a t i o n i n t h e f o l l o w i n g e x a m p l e s we define t h e moments
N
a\ < 2d,
i=l
w i t h a a m u l t i - i n d e x . These m o m e n t s arise as entries i n t h e G r a m m a t r i x G(y)

i f we use a m o n o m i a l basis a n d i d e n t i f y t h e reference p o i n t y w i t h t h e e v a l u a t i o n
point x.
E x a m p l e 2 3 . 1 . W e t a k e s = 1, d = 1, a n d therefore m = 2. T h e set o f d a t a sites

is g i v e n b y X = {x\,..., x^}. W e choose t h e s t a n d a r d m o n o m i a l basis
U = s p a n { p i ( x ) = 1, P2(x) = x}.
T h e n t h e G r a m m a t r i x (22.20) is o f t h e f o r m
( p i ( - - x),pi(- - x))
Wx ( p i ( - - x),p (- 2 - x)) Wx
G(x) =
( p ( - - x),p (-
2 x - x)) Wx ( p ( - - x),p (-
2 2 - x)) Wx
w x x X
I2i=l ( ii ) Y2i=l( i ~ x)w(Xi,x)
X _ x 2
YliLl (* x)w(Xi , X) YliLl (i ~ x) w(Xi , X)
a n d t h e r i g h t - h a n d side o f t h e G r a m s y s t e m is g i v e n b y
>i(0)" "1"
p(x x)
_P2(0)_ 0
Therefore, solution o f t h e G r a m system v i a Cramer's rule immediately yields
2
Xi(x,x) - ^
M0M2 - /i? '
Ml
X (x,x)
2
a n d a c c o r d i n g t o (22.19) t h e g e n e r a t i n g f u n c t i o n s for t h e B a c k u s - G i l b e r t quasi-

i n t e r p o l a n t ( M L S a p p r o x i m a n t ) (22.11) a r e g i v e n b y
#i(x,x) = w(xi,x) [Xi(x,x) + X (x,x)(xi

2 x)\, i = l,...,N,
w h e r e t h e w(xi, ) a r e a r b i t r a r y ( p o s i t i v e ) w e i g h t f u n c t i o n s .
E x a m p l e 2 3 . 2 . W e r e m a i n i n t h e u n i v a r i a t e case (s = 1) b u t increase t h e degree

o f p o l y n o m i a l r e p r o d u c t i o n t o d = 2 so t h a t m = 3. A g a i n w e t a k e t h e s t a n d a r d
m o n o m i a l basis, i.e.,
2
U = s p a n { p i ( x ) = 1 , p (x) 2 = x, p (x) 3 = x }.
T h e 3 x 3 G r a m system
~/i 0 Ml M2~ ~Xt(x, "1"

Ml M2 M3 X (x, 2 X) 0
Jl
2 p,3 / i _ 4 _X3(X, _0.
c a n t h e n b e solved a n a l y t i c a l l y (e.g., using M a p l e ) , a n d one obtains t h e Lagrange

multipliers
M2M4 - Mg
Xi(x,x) =
D
M2M3 - MlM4
X (x,x)
2 =
D
MlM3 - 1*2
X (x,x)
3 =
D
w h e r e D = 2[L\[X ÎZ 2 M0M3 A*l

MiM4 + MoM2M4- N o w t h e g e n e r a t i n g functions
are g i v e n b y
î(x,x) = w(xi,x) [X (x,x)

x + X (x,x)(xi 2 x) + X (x,x)(xi
3
2
- x) ] , z= l, N.
Example 2 3 . 3 . R e p r o d u c t i o n o f l i n e a r p o l y n o m i a l s i n 2 D also leads t o a 3 x 3

G r a m s y s t e m (since s = 2, d = 1 , a n d t h e r e f o r e m = 3 ) . N o w t h e m o n o m i a l basis
is g i v e n b y
U = s p a n { p i ( x , y ) = 1, p (x,y) 2 = x, p (x,y) 3 = y},

23. Examples of MLS Generating Functions 209
2
w h e r e x = (x, y) G K , a n d t h e G r a m s y s t e m l o o k s l i k e
Moo Mio Moi ~\i(x, x)] "1"

Mio M20 M i l X {x,
2 X)
= 0
_Moi M i l M02. _A (a;, 3
x)\ .0.
T h e L a g r a n g e m u l t i p l i e r s i n t h i s case t u r n o u t t o be
Xi(x,x) = [n\ x - M20M02] ,
A (a;, x) =
2 [M10M02 - M01M11] >
A (aj,aj) =
3 [M20M01 - M10M11] >
T h
w i t h D = M10M02 + M20M01 M00M20M02 - 2 / i i o M o i M u + MooMii-
_
e generating
functions i = 1 , . . . , i V , for t h i s e x a m p l e are o f t h e f o r m
Vi(x,x) = w(xi,x) [Ai(a;,cc) + \ (x,x)(xi

2 - x) + X (x,x)(yi
3 - y)].
W h i l e we c a n use a s y m b o l i c m a n i p u l a t i o n p r o g r a m such as M a p l e t o solve t h e

G r a m s y s t e m a n a l y t i c a l l y for o t h e r choices o f d a n d s, i t is clear t h a t t h e expressions
for t h e g e n e r a t i n g f u n c t i o n s q u i c k l y b e c o m e v e r y u n w i e l d y . I t m a y be t o l e r a b l e t o
c o n t i n u e t h i s a p p r o a c h for a 4 x 4 G r a m s y s t e m c o r r e s p o n d i n g t o r e p r o d u c t i o n
of l i n e a r p o l y n o m i a l s i n 3 D (or c u b i c p o l y n o m i a l s i n I D ) , b u t a l r e a d y t h e case o f
q u a d r a t i c r e p r o d u c t i o n i n 2 D ( w i t h m = 6) is m u c h t o o c o m p l e x t o p r i n t here,
a n d r e p r o d u c t i o n o f q u a d r a t i c s i n 3 D ( o r cubics i n 2 D ) requires even 10 L a g r a n g e
multipliers.
Chapter 24
M L S Approximation with MATLAB
24.1 A p p r o x i m a t i o n w i t h Shepard's M e t h o d
I n t h i s section w e i n v e s t i g a t e a p p r o x i m a t i o n w i t h S h e p a r d ' s m e t h o d
{ X i X ) s
PA*) = E/W ' , * GK .
3= 1
We w i l l l o o k at t h r e e sets o f e x p e r i m e n t s . T h e first t w o sets w i l l e m p l o y g l o b a l

e 35-35 2
G a u s s i a n w e i g h t s , i.e., w(xi,x) = e~ H * ! ! , i n K , w h i l e t h e t h i r d set o f ex-
2
p e r i m e n t s is based o n W e n d l a n d ' s c o m p a c t l y s u p p o r t e d C w e i g h t s w(xi,x) =
4 s
(1 - e\\x - Xi\\) + (4e\\x - Xi\\ + 1) i n R for s = 1, 2 , . . . , 6. N o t e t h a t t h e W e n d -
l a n d w e i g h t s are s t r i c t l y p o s i t i v e d e f i n i t e basic f u n c t i o n s o n l y for s < 3 so t h a t i t
w o u l d n o t be a d v i s a b l e t o use t h e m for R B F i n t e r p o l a t i o n i n h i g h e r d i m e n s i o n s .
For M L S a p p r o x i m a t i o n , however, s t r i c t p o s i t i v e definiteness is n o t r e q u i r e d . H e r e
we o n l y ask t h a t t h e w e i g h t s be p o s i t i v e o n t h e i r s u p p o r t .
I n o u r 2 D e x p e r i m e n t s w e t a k e F r a n k e ' s f u n c t i o n (2.2) as o u r test f u n c t i o n a n d
s a m p l e i t a t u n i f o r m l y spaced p o i n t s i n t h e u n i t square. I n T a b l e 2 4 . 1 we l i s t t h e
R M S - e r r o r s a n d c o m p u t e d convergence r a t e s for b o t h s t a t i o n a r y a n d n o n - s t a t i o n a r y
approximation. C l e a r l y , n o n - s t a t i o n a r y a p p r o x i m a t i o n does n o t converge. This
b e h a v i o r is exactly opposite t h e convergence b e h a v i o r o f Gaussians i n t h e RBF
i n t e r p o l a t i o n s e t t i n g . T h e r e we have convergence i n t h e n o n - s t a t i o n a r y s e t t i n g , b u t
not i n t h e s t a t i o n a r y s e t t i n g (c.f. T a b l e 17.5 a n d F i g u r e 1 7 . 1 2 ) . T h e i n i t i a l shape
parameter for t h e s t a t i o n a r y s e t t i n g ( a n d t h e f i x e d v a l u e i n t h e non-stationary
s e t t i n g ) is e = 3. T h i s v a l u e is s u b s e q u e n t l y m u l t i p l i e d b y a f a c t o r o f t w o for each
h a l v i n g o f t h e f i l l - d i s t a n c e . W e c a n see t h a t S h e p a r d ' s m e t h o d seems t o have a
s t a t i o n a r y a p p r o x i m a t i o n o r d e r o f a t least 0(h). T h i s w i l l be v e r i f i e d t h e o r e t i c a l l y
in the next chapter.
T h e first t w o s t a t i o n a r y S h e p a r d a p p r o x i m a t i o n s (based o n 9 a n d 25 p o i n t s ,
r e s p e c t i v e l y ) are d i s p l a y e d i n t h e t o p p a r t o f F i g u r e 2 4 . 1 . I t is a p p a r e n t that
t h e S h e p a r d m e t h o d has a s m o o t h i n g effect. I n o r d e r t o emphasize t h i s f e a t u r e
we p r o v i d e S h e p a r d a p p r o x i m a t i o n s t o n o i s y d a t a ( w i t h 3% noise a d d e d t o t h e
211
Table 24.1 2D stationary and non-stationary Shepard ap-

proximation with Gaussian weight function.
stationary non-stationary
9 1.835110e-001 1.835110e-001
25 5.885159e-002 1.6407 1.303771e-001 0.4932
81 2.299502e-002 1.3558 1.311538e-001 -0.0086
289 6.726166e-003 1.7735 1.315894e-001 -0.0048
1089 2.113604e-003 1.6701 1.320564e-001 -0.0051
4225 8.065893e-004 1.3898 1.323576e-001 -0.0033
F r a n k e d a t a ) i n t h e b o t t o m p a r t o f F i g u r e 2 4 . 1 . O n t h e left we d i s p l a y t h e S h e p a r d
a p p r o x i m a t i o n based o n G a u s s i a n w e i g h t s w i t h e = 48 ( c o r r e s p o n d i n g t o t h e e n t r y
i n l i n e 5 o f T a b l e 2 4 . 1 ) . T h e R M S - e r r o r for t h i s a p p r o x i m a t i o n is 1.820897e-002.
T h e p l o t o n t h e r i g h t o f t h e b o t t o m p a r t o f F i g u r e 24.1 is t h e r e s u l t o f r e d u c i n g e
t o 12, a n d t h u s i n c r e a s i n g t h e s m o o t h i n g effect since " w i d e r " w e i g h t f u n c t i o n s are
used, i.e., m o r e n e i g h b o r i n g d a t a are i n c o r p o r a t e d i n t o t h e l o c a l regression fit. T h e
c o r r e s p o n d i n g R M S - e r r o r is 2.481218e-002. N o t e t h a t even t h o u g h t h e R M S - e r r o r
is larger for t h e p l o t o n t h e r i g h t , t h e surface is v i s u a l l y s m o o t h e r . These examples
s h o u l d be c o m p a r e d t o t h e d a t a s m o o t h i n g e x p e r i m e n t s i n C h a p t e r 19.
T h e M A T L A B code for t h e 2 D e x p e r i m e n t s is Shepard2D. m l i s t e d as Pro-
g r a m 2 4 . 1 . I t is a l i t t l e s i m p l e r t h a n t h e R B F i n t e r p o l a t i o n code used earlier since
we d o n o t need t o assemble a n i n t e r p o l a t i o n m a t r i x a n d n o l i n e a r s y s t e m needs t o
be solved. T h e e v a l u a t i o n m a t r i x is assembled i n t w o steps. F i r s t , o n l i n e 14, we
compute the standard R B F evaluation m a t r i x . I n order t o implement the Shepard
scaling we n o t e t h a t a l l o f t h e entries i n r o w i o f t h e e v a l u a t i o n m a t r i x are scaled
w x x
w i t h t h e same s u m , ] C j L i ( ji i)- T h e r e f o r e , o n l i n e 16, we p e r f o r m t h e e x t r a
S h e p a r d scaling w i t h t h e h e l p o f repmat a n d a v e c t o r o f ones w h i c h gives us a l l t h e
necessary sums i n t h e d e n o m i n a t o r . F o r t h e e x a m p l e w i t h n o i s y d a t a we a d d
f = f + 0.03*randn(size(f));
after l i n e 1 1 .
P r o g r a m 24.1. Shepard2D. m
7, Shepard2D
7, S c r i p t t h a t performs 2D Shepard approximation w i t h g l o b a l weights
7. C a l l s on: D i s t a n c e M a t r i x , P l o t S u r f , P l o t E r r o r 2 D
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; ep = 5.5;
7o Define F r a n k e ' s f u n c t i o n as t e s t f u n c t i o n
2 f l = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x - 2 ) . ~ 2 + ( 9 * y - 2 ) . ~ 2 ) / 4 ) ;
3 f 2 = <3(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x + l ) . ~ 2 / 4 9 + ( 9 * y + l ) . ~ 2 / 1 0 ) ) ;
4 f 3 = @(x,y) 0 . 5 * e x p ( - ( ( 9 * x - 7 ) . ~ 2 + ( 9 * y - 3 ) . ~ 2 ) / 4 ) ;
24- MLS Approximation with MATLAB 213
Fig. 24.1 Top: Shepard approximation with stationary Gaussian weights to Franke's function
2
using 9 (left) and 25 (right) uniformly spaced points in [0, l ] . Bottom: Shepard approximation
with Gaussian weights to noisy Franke's function using e = 48 (left) and = 12 (right) on 1089
2
uniformly spaced points in [0, l ] .
5 f 4 = <3(x,y) 0 . 2 * e x p ( - ( ( 9 * x - 4 ) .~2+(9*y-7) ."2)) ;

7 N = 1089; g r i d t y p e = 'u';
8 n e v a l = 40;
7. Load data p o i n t s
9 name = s p r i n t f ('Data2D_7od7.s' ,N,gridtype) ; load(name)
10 ctrs = dsites;
7o C r e a t e v e c t o r of f u n c t i o n ( d a t a ) v a l u e s
11 f = testfunction(dsites(:,1),dsites(:,2));
7 Create n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
7o i n t h e u n i t square
13 epoints = [xe(:) y e ( : ) ] ;
V. Compute e v a l u a t i o n m a t r i x
16 EM = EM./repmat(EM*ones(N,1),1,N); % Shepard n o r m a l i z a t i o n
/ Compute q u a s i - i n t e r p o l a n t
0
17 Pf = EM*f;
"It Compute e x a c t s o l u t i o n , i . e . ,
X e v a l u a t e t e s t f u n c t i o n on e v a l u a t i o n p o i n t s
% Compute e r r o r s on e v a l u a t i o n g r i d
21 fprintf('RMS error: %e\n', rms_err)
22 fprintf('Maximum e r r o r : %e\n', maxerr)
% Plot interpolant
23 PlotSurf(xe,ye,Pf,neval,exact,maxerr,[160,20]);
% P l o t absolute e r r o r
24 PlotError2D(xe,ye,Pf,exact,maxerr,neval,[160,20]);
T h e n e x t g r o u p o f e x p e r i m e n t s ( r e p o r t e d i n T a b l e s 24.2 a n d 24.3) s h o u l d b e
c o m p a r e d t o t h e d i s t a n c e m a t r i x fits o f C h a p t e r 1. T h e d a t a are a g a i n s a m p l e d
f r o m t h e test f u n c t i o n
s
s
f (x)
s = 4 Y[x (l-x ), d d x = (xi,...,x )3 G [0, l ] ,s
d=l
w h i c h is p a r a m e t r i z e d b y t h e space d i m e n s i o n s a n d c o d e d i n t h e M A T L A B s u b r o u -
t i n e t e s t f u n c t i o n . m (see P r o g r a m C l ) .
Since we are using the compactly supported weights w(xi,x)
(1 e\\x Xi\\)^_ (4e\\x Xi\\ + 1) w e c a n keep t h e e v a l u a t i o n m a t r i x sparse a n d
are therefore able t o deal w i t h m u c h larger d a t a sets. A g a i n , w e e m p l o y a s t a t i o n -
a r y a p p r o x i m a t i o n scheme, i.e., t h e scale p a r a m e t e r e for t h e s u p p o r t o f t h e w e i g h t
f u n c t i o n s is ( i n v e r s e l y ) p r o p o r t i o n a l t o t h e f i l l - d i s t a n c e . I n fact, we t a k e e = 2 + 1
fc
k s
(see line 4 o f P r o g r a m 24.2), w h e r e A; also d e t e r m i n e s t h e n u m b e r = (2 + l ) of
s
d a t a p o i n t s used (c.f. l i n e 3). T h e s e p o i n t s are t a k e n t o be H a l t o n p o i n t s i n [0, l ]
(c.f. l i n e 6).
I n Tables 24.2 a n d 24.3 w e list R M S - e r r o r s ( c o m p u t e d o n v a r i o u s g r i d s o f e q u a l l y
spaced p o i n t s , i.e., m o s t l y o n t h e b o u n d a r y o f t h e u n i t c u b e i n h i g h e r d i m e n s i o n s )
s
for a series o f a p p r o x i m a t i o n p r o b l e m s i n K . for s = l,2,...,6. A g a i n , t h e a p p r o x -
i m a t i o n o r d e r for S h e p a r d ' s m e t h o d seems t o be r o u g h l y 0(h) independent of the
space d i m e n s i o n s.
T h e M A T L A B code Shepard_CS. m is l i s t e d as P r o g r a m 24.2. T h i s t i m e t h e eval-
u a t i o n m a t r i x is a sparse m a t r i x w h i c h we also c o m p u t e i n t w o steps. T h e s t a n d a r d
RBF e v a l u a t i o n m a t r i x is c o m p u t e d as for o u r e a r l i e r C S R B F c o m p u t a t i o n s on
lines 10 a n d 11 u s i n g t h e s u b r o u t i n e DistanceMatrixCSRBF.m (see C h a p t e r 12).
T h i s t i m e t h e S h e p a r d s c a l i n g is p e r f o r m e d o n l i n e 12 w i t h t h e h e l p o f a d i a g o n a l
m a t r i x stored i n the spdiags f o r m a t .
Table 24.2 Shepard fit in 1 D - 3 D with compactly supported weight function.
ID 2D 3D
1 3 2.844398e-001 9 2.388314e-001 27 1.912827e-001

2 5 1.665656e-001 25 1.271941e-001 125 1.249589e-001
3 9 8.796073e-002 81 7.107988e-002 729 6.898182e-002
4 17 3.970488e-002 289 3.944228e-002 4913 3.718816e-002
5 33 1.738869e-002 1089 3.109722e-002 35937 2.838763e-002
6 65 7.535727e-003 4225 1.888361e-002 274625 9.837855e-003
7 129 3.418925e-003 16641 2.831294e-002
8 257 1.615694e-003 66049 2.667914e-003
9 513 7.872903e-004 263169 2.352736e-003
10 1025 3.884881e-004
11 2049 1.940611e-004
12 4097 9.699922e-005
Table 24.3 Shepard fit in 4 D - 6 D with compactly supported weight function.
4D 5D 6D
1 81 1.310311e-001 243 8.936253e-002 729 6.372675e-002

2 625 8.943469e-002 3125 6.344921e-002 15625 3.910649e-002
3 6561 4.599027e-002 59049 3.492958e-002 531441 2.234389e-002
4 83521 2.523581e-002
P r o g r a m 24.2. Sh.epard_CS.rn
7, Shepard.CS
% S c r i p t t h a t performs Shepard approximation f o r a r b i t r a r y
% space dimensions s u s i n g s p a r s e m a t r i c e s
/
0C a l l s on: DistanceMatrixCSRBF, MakeSDGrid, t e s t f u n c t i o n
% Uses: h a l t o n s e q ( w r i t t e n by D a n i e l Dougherty from
7, MATLAB C e n t r a l F i l e Exchange)
% Wendland C2 weight f u n c t i o n
1 r b f = @(e,r) r . ~ 4 . * ( 5 * s p o n e s ( r ) - 4 * r ) ;
2 s = 6; % Space dimension s
% Number of Halton d a t a p o i n t s
3 k = 3; N = ( 2 ~ k + l ) ~ s ;
4 ep = 2~k+l; % S c a l e parameter f o r b a s i s f u n c t i o n
5 n e v a l = 4; M = n e v a l ~ s ;
7o Compute d a t a s i t e s as H a l t o n p o i n t s
6 dsites = haltonseq(N,s);
7 ctrs = dsites;
7o Create v e c t o r of f u n c t i o n (data) v a l u e s
8 f = testfunction(s,dsites);
/. C r e a t e n e v a l ~ s e q u a l l y spaced e v a l u a t i o n l o c a t i o n s i n
% the s-dimensional u n i t cube
9 epoints = MakeSDGrid(s,neval);
% Compute e v a l u a t i o n m a t r i x , i . e . ,
% matrix of v a l u e s of g e n e r a t i n g functions
10 DM_eval = DistanceMatrixCSRBF(epoints,ctrs,ep);
% Shepard scaling
12 EM = spdiags(l./(EM*ones(N,l)),0,M,M)*EM;
% Compute q u a s i - i n t e r p o l a n t
13 Pf = EM*f;
% Compute e x a c t s o l u t i o n , i . e . ,
/ e v a l u a t e t e s t f u n c t i o n on e v a l u a t i o n p o i n t s
14 exact = t e s t f u n c t i o n ( s , e p o i n t s ) ;
16 rms_err = norm(Pf-exact)/sqrt(M);
17 fprintf('RMS error: %e\n', rms_err)
18 f p r i n t f ('Maximum e r r o r : /e\n', maxerr) 0
24.2 M L S Approximation with Linear Reproduction
I n g e n e r a l one w o u l d e x p e c t a m o v i n g least squares m e t h o d t h a t r e p r o d u c e s l i n e a r

p o l y n o m i a l s t o be m o r e a c c u r a t e t h a n one t h a t r e p r o d u c e s o n l y c o n s t a n t s s u c h as
2
S h e p a r d ' s m e t h o d . W e i l l u s t r a t e t h i s i n T a b l e 24.4 w h e r e w e a g a i n use t h e C com-
p a c t l y s u p p o r t e d w e i g h t f u n c t i o n s w(xi,x) = (1 ej|c JCi||)^_ (4e||c Xi\\ + 1) i n
a s t a t i o n a r y a p p r o x i m a t i o n s e t t i n g w i t h a base e = 3 for N = 9 d a t a p o i n t s ( a n d
t h e n scaled i n v e r s e l y p r o p o r t i o n a l t o t h e fill-distance) t o a p p r o x i m a t e d a t a sam-
p l e d f r o m F r a n k e ' s f u n c t i o n a t u n i f o r m l y s p a c e d p o i n t s i n [0, l ] . W e c a n see t h a t
2
t h e M L S m e t h o d w i t h l i n e a r r e p r o d u c t i o n has a n u m e r i c a l a p p r o x i m a t i o n o r d e r o f
2
0(h ). T h i s w i l l be j u s t i f i e d t h e o r e t i c a l l y i n t h e n e x t c h a p t e r .
I n s t e a d o f s o l v i n g a G r a m s y s t e m for each e v a l u a t i o n p o i n t as suggested i n (22.8)
or (22.20) we use t h e e x p l i c i t f o r m u l a s for t h e L a g r a n g e m u l t i p l i e r s a n d generating
f u n c t i o n s g i v e n for t h e 2D case i n E x a m p l e 23.3. T h e s e f o r m u l a s are i m p l e m e n t e d
i n M A T L A B i n t h e s u b r o u t i n e L i n e a r S c a l i n g 2 D _ C S .m a n d l i s t e d i n P r o g r a m 24.3.
I n p a r t i c u l a r , t h i s s u b r o u t i n e is w r i t t e n t o d e a l w i t h c o m p a c t l y s u p p o r t e d w e i g h t
f u n c t i o n s a n d t h u s uses sparse m a t r i c e s . A s i n t h e a s s e m b l y o f sparse i n t e r p o l a t i o n
m a t r i c e s we m a k e use o f kd-trees.
W e b u i l d a fcd-tree o f a l l o f t h e centers f o r t h e w e i g h t f u n c t i o n s a n d find for
each e v a l u a t i o n p o i n t t h o s e c e n t e r s whose s u p p o r t o v e r l a p s t h e e v a l u a t i o n p o i n t .
T h e i n p u t t o L i n e a r S c a l i n g 2 D _ C S .m are e p o i n t s ( a n N x s m a t r i x r e p r e s e n t i n g a
s
set o f N d a t a sites i n M ) , c t r s ( a n M x s m a t r i x r e p r e s e n t i n g a set o f M centers
Table 24.4 2D stationary M L S ap-

proximation to Franke's function at
2
uniformly spaced points in [ 0 , 1 ]
with linear precision using com-
pactly supported weight function.
N RMS-error rate
9 1.789573e-001
25 7.089522e-002 1.3359
81 2.691693e-002 1.3972
289 7.516377e-003 1.8404
1089 1.944252e-003 1.9508
4225 4.903575e-004 1.9873
16641 1.228639e-004 1.9968
66049 3.072866e-005 1.9994
263169 7.658656e-006 2.0044
1050625 1.921486e-006 1.9949
s
for t h e w e i g h t f u n c t i o n s i n R ) , r b f ( a n a n o n y m o u s o r i n l i n e f u n c t i o n d e f i n i n g t h e
R B F w e i g h t f u n c t i o n ) , a n d ep ( t h e scale p a r a m e t e r t h a t d e t e r m i n e s t h e size o f t h e
support of the weight functions). A s always, w i d e functions result f r o m a small
value o f ep, i.e., t h e size o f t h e s u p p o r t o f t h e w e i g h t f u n c t i o n is g i v e n b y 1/ep. The
o u t p u t o f L i n e a r S c a l i n g 2 D _ C S .m is a n N x M sparse m a t r i x P h i t h a t c o n t a i n s t h e
M L S generating functions centered at the p o i n t s given b y c e n t e r a n d evaluated at
the e v a l u a t i o n p o i n t s i n e p o i n t s . N o t e t h a t t h e c o m p a c t l y s u p p o r t e d basic f u n c t i o n
needs t o be s u p p l i e d i n i t s s t a n d a r d ( u n s h i f t e d ) f o r m , e.g., as
rbf = _(e,r) max(spones(r)-e*r,0)."4.*(4*e*r+spones(r));
2 r
for t h e C W e n d l a n d f u n c t i o n < ^ , i ( ) = ( 1 - r)\(4r
3 + 1).
For each e v a l u a t i o n p o i n t (see t h e l o o p over i f r o m l i n e 10 t o l i n e 33) w e c o m p u t e
the six different m o m e n t s ( e n t r i e s o f t h e G r a m m a t r i x G, c.f. E x a m p l e 23.3) r e q u i r e d
for t h e c o m p u t a t i o n o f t h e L a g r a n g e m u l t i p l i e r s o n lines 1 4 - 2 0 . T h e n t h e values o f
the L a g r a n g e m u l t i p l i e r s a n d o f t h e g e n e r a t i n g f u n c t i o n s a t t h e ith e v a l u a t i o n p o i n t
are c o m p u t e d o n lines 2 1 - 2 4 . T h e f i n a l sparse m a t r i x P h i is assembled o n l i n e 35.
P r o g r a m 2 4 . 3 . L i n e a r S c a l i n g 2 D _ C S .m
% Phi = LinearScaling2D_CS(epoints,ctrs,rbf,ep)
% Forms a s p a r s e m a t r i x o f scaled generating functions f o r MLS
% approximation with linear reproduction.
7. U s e s : k-D t r e e p a c k a g e b y Guy S h e c h t e r from
7. MATLAB C e n t r a l F i l e Exchange
1 function Phi = LinearScaling2D_CS(epoints,ctrs,rbf,ep)
2 [N,s] = s i z e ( e p o i n t s ) ; [M,s] = size(ctrs);
3 alpha = [ 0 0; 10; 0 1 ; 1 1 ; 20; 0 2];
7o B u i l d k - D t r e e for centers
4 [tmp,tmp,Tree] = kdtree ( c t r s , [ ] ) ;
% F o r each e v a l . p o i n t , f i n d c e n t e r s whose support o v e r l a p i t
5 support = 1/ep; mu = z e r o s ( 6 ) ;
% Modify t h e f o l l o w i n g l i n e f o r optimum performance
6 v e c l e n g t h = round(support*N*M/4);
7 rowidx = z e r o s ( 1 . v e c l e n g t h ) ; c o l i d x = z e r o s ( 1 , v e c l e n g t h ) ;
8 validx = zeros(1.veclength);
9 i s t a r t = 1; i e n d = 0;
10 f o r i = 1:N
11 [pts,dist,idx] = kdrangequery(Tree.epoints(i,:).support);
12 newlen = l e n g t h ( i d x ) ;
7o Vector of b a s i s f u n c t i o n s
13 Phi_i = rbf(ep,dist>);
% Compute a l l 6 moments f o r i - t h e v a l u a t i o n p o i n t
14 for j=l:6
15 x _ t o _ a l p h a = 1;
16 f o r coord=l:s
17a x_to_alpha = x_to_alpha . * ( c t r s ( i d x , c o o r d ) - . . .
17b repmat(epoints(i,coord),newlen,1))."alpha(j,coord);
18 end
19 mu(j) = P h i _ i * x _ t o _ a l p h a ;
20 end
21 Ll=(mu(4)~2-mu(5)*mu(6)); L2=(mu(2)*mu(6)-mu(3)*mu(4));
22 L3=(mu(5)*mu(3)-mu(2)*mu(4));
23a s c a l i n g = Ll*repmat(1.newlen,1) + ...
23b L2*(ctrs(idx,l)-repmat(epoints(i,1),newlen,1))+...
23c L3*(ctrs(idx,2)-repmat(epoints(i,2),newlen,1));
24a denom = mu(2)~2*mu(6)+mu(5)*mu(3)~2-mu(l)*mu(5)*mu(6)-...
24b 2*mu(2)*mu(3)*mu(4)+mu(l)*mu(4)"2;
25 i f (denom ~= 0)
26 s c a l i n g = scaling/denom;
27 i e n d = i e n d + newlen;
28 r o w i d x ( i s t a r t : i e n d ) = repmat(i,1,newlen);
29 c o l i d x ( i s t a r t : i e n d ) = idx';
30 validx(istart:iend) = Phi_i.*scaling';
31 i s t a r t = i s t a r t + newlen;
32 end
33 end
34 f i l l e d = f i n d ( r o w i d x ) ; % only those a c t u a l l y f i l l e d
35 Phi = s p a r s e ( r o w i d x ( f i l l e d ) , c o l i d x ( f i l l e d ) , v a l i d x ( f i l l e d ) , N , M )
% Free memory
36 c l e a r rowidx c o l i d x v a l i d x ; k d t r e e ( [ ] , [ ] .Tree) ;
W i t h a l l t h e d i f f i c u l t c o d i n g r e l e g a t e d t o t h e s u b r o u t i n e L i n e a r S c a l i n g 2 D _ C S .m
t h e m a i n p r o g r a m LinearMLS2D_CS .m is r a t h e r s i m p l e . I t is l i s t e d i n P r o g r a m 24.4.
T h e v e r s i o n o f t h e code l i s t e d here is a d a p t e d t o r e a d a d a t a file t h a t c o n t a i n s b o t h
d a t a sites ( i n t h e v a r i a b l e d s i t e s ) a n d f u n c t i o n values ( i n t h e v a r i a b l e r h s ) . For
t h e e x a m p l e d i s p l a y e d i n F i g u r e 24.2 we used a n a c t u a l set o f 1:250,000-scale D i g i t a l
E l e v a t i o n M o d e l ( D E M ) d a t a f r o m t h e U . S . G e o l o g i c a l Survey, n a m e l y t h e d a t a set
Dubuque-E w h i c h contains elevation d a t a f r o m a region i n northeastern Iowa, n o r t h -
w e s t e r n I l l i n o i s , a n d s o u t h w e s t e r n W i s c o n s i n . T h e d a t a set is a v a i l a b l e o n t h e w o r l d -
wide web at http://edcsgs9.cr.usgs.gov/glis/hyper/guide/l_dgr_demfig/-
states/IL.html. T h e o r i g i n a l d a t a set c o n t a i n s 1201 x 1201 u n i f o r m l y spaced
m e a s u r e m e n t s r a n g i n g i n h e i g h t f r o m 178 m e t e r s t o 426 m e t e r s . We converted
t h e D E M d a t a f o r m a t w i t h t h e u t i l i t y DEM2XYZN t h a t c a n be d o w n l o a d e d from
http://data.geocomm.com/dem/dem2xyzn/, selected o n l y t h e n o r t h e a s t e r n quad-
r a n t o f 601 x 601 = 3 6 1 2 0 1 e l e v a t i o n values, a n d scaled t h e x a n d y coordinates
t h a t o r i g i n a l l y defined a square w i t h a side l e n g t h o f a b o u t 35 m i l e s t o t h e u n i t
square. T h e r e s u l t i n g d a t a set Data2D_DubuqueNE is i n c l u d e d o n t h e enclosed C D .
2
A n M L S a p p r o x i m a t i o n w i t h l i n e a r r e p r o d u c t i o n based o n t h e C c o m p a c t l y sup-
p o r t e d W e n d l a n d w e i g h t used earlier was e v a l u a t e d o n a g r i d o f 60 x 60 e q u a l l y
spaced p o i n t s a n d is d i s p l a y e d i n F i g u r e 24.2. W e use a shape p a r a m e t e r o f e = 30
t o d e t e r m i n e t h e s u p p o r t size o f t h e w e i g h t f u n c t i o n s .
P r o g r a m 2 4 . 4 . LinearMLS2D_CS .m
% LinearMLS2D_CS
/ S c r i p t t h a t
0 performs MLS a p p r o x i m a t i o n w i t h linear reproduction
% u s i n g sparse matrices
% Calls on: LinearScaling2D_CS
1 r b f = @(e,r) max(spones(r)-e*r,0).~4.*(4*e*r+spones(r));
2 ep = 3 0 ; n e v a l = 6 0 ;
% Load d a t a p o i n t s and rhs
,
3 load( Data2D_DubuqueNE');
4 ctrs = dsites;
% Create n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n locations
/ i n t h e
0 unit square
5 grid = linspace(0,1,neval); [xe,ye] = meshgrid(grid);
/ Compute e v a l u a t i o n m a t r i x
0
7 EM = L i n e a r S c a l i n g 2 D _ C S ( e p o i n t s , c t r s , r b f , e p ) ;
% Compute MLS a p p r o x i m a t i o n (rhs read from data file)
8 Pf = EM*rhs;
9 figure % P l o t MLS f i t
10 surfplot = surf(xe,ye.reshape(Pf,neval,neval));
11 set(surfplot,'FaceColor','interp','EdgeColor','none')
12 view([15,35]); c a m l i g h t ; l i g h t i n g gouraud; colormap summer
400 - r
350-
300-
250-
200
0
0.8 0
Fig. 24.2 M L S approximation using compactly supported weight for elevation data in northwest-
ern Illinois.
A m u c h s i m p l e r ( b u t also m u c h s l o w e r ) i m p l e m e n t a t i o n o f M L S a p p r o x i m a t i o n
w i t h l i n e a r r e p r o d u c t i o n is g i v e n as P r o g r a m 24.5. I n t h i s i m p l e m e n t a t i o n w e solve
t h e l o c a l least squares p r o b l e m for each e v a l u a t i o n p o i n t x, i.e., w e o b t a i n t h e M L S
a p p r o x i m a t i o n at the p o i n t x as
m
T h e s o l u t i o n o f these s y s t e m s (for each e v a l u a t i o n p o i n t x) is c o m p u t e d o n l i n e 18

o f t h e p r o g r a m , w i t h t h e G r a m m a t r i x b u i l t as
T
G{x) = P (x)W(x)P(x),
and the polynomial matrix P with entries Pij(x) = Pj(xi x) and diago-
nal weight matrix computed on lines 16 and 17, respectively. T h e subroutine
LinearScaling2D_CS .m is not required by this program. Compare this program
with Program 24.1.
P r o g r a m 24.5. LinearMLS2D_GramSolve .m
7. LinearMLS2D_GramSolve
7o S c r i p t t h a t performs MLS approximation w i t h l i n e a r reproduction
7o by s o l v i n g l o c a l Gram systems
Swap*
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; ep = 5.5;
Define Franke's f u n c t i o n as t e s t f u n c t i o n
2 f l = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x - 2 ) . ~ 2 + ( 9 * y - 2 ) . ~ 2 ) / 4 ) ;
3 f2 = _(x,y) 0.75*exp(-((9*x+l).~2/49+(9*y+l).~2/10));
4 f 3 = @(x,y) 0 . 5 * e x p ( - ( ( 9 * x - 7 ) . ~ 2 + ( 9 * y - 3 ) . ~ 2 ) / 4 ) ;
5 f 4 = _(x,y) 0.2*exp(-((9*x-4).~2+(9*y-7).~2));
7 N = 1089; g r i d t y p e = 'u';
8 n e v a l = 40;
9 name = s p r i n t f ('Data2D_y d%s' ,N,gridtype) ; load(name)
o
10 ctrs = dsites;
% Create v e c t o r of f u n c t i o n ( d a t a ) v a l u e s
11 f = testfunction(dsites(:,1),dsites(:,2));
% Create n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
% i n the u n i t square
13 epoints = [ x e ( : ) y e ( : ) ] ;
% Compute MLS approximation w i t h s h i f t e d b a s i s polynomials
14 Pf = z e r o s ( n e v a l " 2 , 1 ) ;
15 f o r j=l:neval~2
16 P = [ones(N,l) d s i t e s - r e p m a t ( e p o i n t s ( j , : ) , N , 1 ) ] ;
17 W = diag(rbf(ep,DistanceMatrix(epoints(j,:),ctrs)));
18 c = (P'*W*P)\(P'*W*f);
19 Pf(j) = c(l);
20 end
% Compute exact s o l u t i o n , i . e . ,
/ e v a l u a t e t e s t f u n c t i o n on e v a l u a t i o n p o i n t s
23 rms_err = n o r m ( P f - e x a c t ) / n e v a l ;
/ P l o t i n t e r p o l a n t
0
24 PlotSurf(xe,ye,Pf,neval,exact,maxerr,[160,20] ) ;
% P l o t absolute e r r o r
25 PlotError2D(xe,ye,Pf,exact,maxerr,neval,[160,20]);
A l t e r n a t i v e l y , we c o u l d have c o d e d lines 1 4 - 2 0 w i t h a n u n s h i f t e d polynomial

basis (c.f. t h e discussion i n C h a p t e r 2 2 ) , i n w h i c h case we w o u l d have
14 P = [ones(N.l) d s i t e s ] ;
15 Pf = z e r o s ( n e v a l ~ 2 , 1 ) ;
16 for j=l:neval~2
17 W = diag(rbf(ep,DistanceMatrix(epoints(j,:),ctrs)));
18 c = (P'*W*P)\(P'*W*_);
19 P f ( j ) = [1 e p o i n t s ( j , : ) ] * c ;
20 end
i n P r o g r a m 24.5. T h i s r e q u i r e s s e t t i n g u p t h e m a t r i x P o n l y once. However, the

c o m p u t a t i o n s are n u m e r i c a l l y n o t as stable. Also, additional computation of the
a p p r o x i m a n t o n l i n e 19 is r e q u i r e d v i a a d o t p r o d u c t .
24.3 P l o t s of B a s i s - D u a l B a s i s P a i r s
W e n o w p r o v i d e some p l o t s o f t h e p o l y n o m i a l basis f u n c t i o n s p for t h e s t a n d a r d M L S

approach, t h e d u a l basis f u n c t i o n s A ( L a g r a n g e m u l t i p l i e r s ) , a n d t h e generating
functions f r o m t h e B a c k u s - G i l b e r t a p p r o a c h for a o n e - d i m e n s i o n a l e x a m p l e w i t h
X b e i n g t h e set o f 11 e q u a l l y spaced p o i n t s i n [0,1]. We take m 3, i.e., we
consider t h e case t h a t ensures r e p r o d u c t i o n o f q u a d r a t i c p o l y n o m i a l s . T h e w e i g h t
f u n c t i o n is t a k e n t o be t h e s t a n d a r d G a u s s i a n r a d i a l f u n c t i o n scaled w i t h a s h a p e
p a r a m e t e r e = 5. I n t h e o n l y e x c e p t i o n i n t h i s b o o k , these p l o t s were c r e a t e d with
M a p l e u s i n g t h e p r o g r a m MLSDualBases .mws i n c l u d e d i n A p p e n d i x C.
2
T h e t h r e e basis p o l y n o m i a l s pi(x) = 1, p (%)2 = x, a n d pz(x) = x are s h o w n
i n F i g u r e 24.3, w h e r e a s t h e d u a l basis f u n c t i o n s A i , A , a n d A3 are d i s p l a y e d i n
2
F i g u r e 24.4.
Fig. 24.3 Plot of the three polynomial basis functions for moving least squares approximation
with quadratic reproduction on [0,1].
I n F i g u r e 24.5 w e p l o t t h r e e M L S g e n e r a t i n g f u n c t i o n s ( s o l i d ) t o g e t h e r w i t h t h e
c o r r e s p o n d i n g g e n e r a t i n g f u n c t i o n s f r o m t h e a p p r o x i m a t e m o v i n g least squares a p -
p r o a c h ( d a s h e d ) d e s c r i b e d i n C h a p t e r 26. T h e g e n e r a t i n g f u n c t i o n s for t h e a p p r o x -
i m a t e M L S a p p r o a c h are L a g u e r r e - G a u s s i a n s (c.f. S e c t i o n 4 . 2 ) . W h i l e t h e standard
MLS g e n e r a t i n g f u n c t i o n s reflect t h e fact t h a t t h e d a t a is g i v e n o n a f i n i t e i n t e r v a l ,
t h e g e n e r a t i n g f u n c t i o n s for a p p r o x i m a t e M L S a p p r o x i m a t i o n are a l l j u s t shifts o f
24- MLS Approximation with M A T L A B 223
Fig. 24.4 Plot of the three dual basis functions for moving least squares approximation with
quadratic reproduction for 11 equally spaced points in [0,1].
t h e one f u n c t i o n
X y\ l|o> Wll
T>h' 2
TV Vh
t o t h e center p o i n t s y. H e r e we i d e n t i f y t h e scale p a r a m e t e r V w i t h o u r shape

p a r a m e t e r e for t h e w e i g h t f u n c t i o n v i a e = F o r t h i s e x a m p l e w i t h 11 p o i n t s
i n [0,1] we have h = 1/10, so t h a t e = 5 corresponds t o a v a l u e o f T> = 4.
I n t h e center o f t h e i n t e r v a l , w h e r e t h e influence o f t h e b o u n d a r y is m i n i m a l ,
t h e t w o t y p e s o f g e n e r a t i n g f u n c t i o n s are a l m o s t i d e n t i c a l (see t h e r i g h t p l o t i n
F i g u r e 24.5).
i Kf_-+-n | i i i i | i i i i |
0.0 0.35 0.5 0.75 1.0
Fig. 24.5 Standard M L S generating functions (solid) and approximate M L S generating functions
(dashed) centered at three of the 11 equally spaced points in [0,1].
I f t h e d a t a p o i n t s are n o l o n g e r e q u a l l y spaced, t h e L a g r a n g e f u n c t i o n s a n d
g e n e r a t i n g f u n c t i o n s are also less u n i f o r m . F i g u r e s 24.6 a n d 24.7 i l l u s t r a t e t h i s
dependence o n t h e d a t a d i s t r i b u t i o n for 11 H a l t o n p o i n t s i n [0,1].
F i n a l l y , we p r o v i d e p l o t s o f M L S g e n e r a t i n g f u n c t i o n s for t h e case o f r e p r o d u c -
t i o n o f linear p o l y n o m i a l s i n 2D (see F i g u r e 24.8). These p l o t s were c r e a t e d w i t h t h e
MATLAB program LinearMLS2D_CS .m (see P r o g r a m 24.4) b y p l o t t i n g c o l u m n j o f
t h e e v a l u a t i o n m a t r i x EM c o r r e s p o n d i n g t o t h e values o f t h e j t h g e n e r a t i n g f u n c t i o n .
W e used t h e C 2
W e n d l a n d w e i g h t s w(xi,x) = (1 e\\x cci||)+ (4_r||ic c_j|| + 1)
w i t h e 5.
Fig. 24.6 Plot of the three dual basis functions for moving least squares approximation with
quadratic reproduction for 11 Halton points in [0,1].
Fig. 24.7 Standard M L S generating functions (solid) and approximate M L S generating functions
(dashed) centered at three of the 11 Halton points in [0,1].
Fig. 24.8 M L S generating functions for linear reproduction centered at two of 289 uniformly
2
spaced data sites in [0, l ] .
Chapter 25
Error Bounds for

Moving Least Squares Approximation
25.1 A p p r o x i m a t i o n O r d e r of M o v i n g L e a s t S q u a r e s
Since t h e m o v i n g least squares a p p r o x i m a n t s c a n be w r i t t e n as quasi-interpolants

we c a n use s t a n d a r d techniques t o derive t h e i r point-wise error estimates. The
s t a n d a r d a r g u m e n t proceeds as f o l l o w s . Let / be a g i v e n ( s m o o t h ) f u n c t i o n t h a t
generates t h e d a t a , i.e., fi = f(xi),..., fx = / ( c c / v ) , a n d l e t p be a n a r b i t r a r y
p o l y n o m i a l . M o r e o v e r , assume t h a t t h e m o v i n g least squares a p p r o x i m a n t is g i v e n
in the form
N
V (x)
f = Y f(x )y (x,x)
/ i i
i=l
w i t h the generating functions * i satisfying the p o l y n o m i a l r e p r o d u c t i o n p r o p e r t y
N
^2p(xi)î(x,x) = p(x), fbrallpeLLj,
i=l
as described i n C h a p t e r 22. T h e n , d u e t o t h e p o l y n o m i a l r e p r o d u c t i o n p r o p e r t y o f
t h e g e n e r a t i n g f u n c t i o n s , w e get
N
\f(x) - V (x)\f < \f(x) - p(x)\ + \p(x) - f{xi)î{x, x)\
i=l
N N
= \f(x)-p(x)\ + \^2p(xi)î(x,x) -J2f( i)î(x,x)\.
x
i=l i=l
C o m b i n a t i o n o f t h e t w o s u m a n d t h e d e f i n i t i o n o f t h e discrete m a x i m u m n o r m y i e l d
N
\f(x) - V (x)\
f < \f(x) - p(x)\ + \PM ~ aOI
i=l
N
<II/-Pl (25.1)
W e see t h a t i n o r d e r t o refine t h e e r r o r e s t i m a t e w e n o w h a v e t o answer t w o ques-

tions:
225
H o w w e l l d o p o l y n o m i a l s a p p r o x i m a t e / ? T h i s c a n b e achieved w i t h s t a n d a r d
T a y l o r expansions.
N
A r e t h e g e n e r a t i n g f u n c t i o n s b o u n d e d ? T h e expression \î(x, x)\ is k n o w n
i=l
as t h e (value o f t h e ) Lebesgue function, a n d f i n d i n g a b o u n d for t h e Lebesgue
f u n c t i o n is t h e m a i n t a s k t h a t remains.
B y t a k i n g t h e p o l y n o m i a l p above t o b e t h e T a y l o r p o l y n o m i a l o f t o t a l degree d
for / a t x, t h e r e m a i n d e r t e r m i m m e d i a t e l y y i e l d s a n e s t i m a t e o f t h e f o r m
d + 1 a
1 1 / - Plloo < C h x max\D f{% \ct\ = d + l . (25.2)
T h u s , i f we c a n establish a u n i f o r m b o u n d for t h e Lebesgue f u n c t i o n , t h e n (25.1)

a n d (25.2) w i l l r e s u l t i n
d+1
\f(x)-V (x)\<Ch m^\D<*f(Z)\,
f |a| =d+l,
w h i c h shows t h a t m o v i n g least squares a p p r o x i m a t i o n w i t h p o l y n o m i a l r e p r o d u c t i o n

d+1
of degree d has a p p r o x i m a t i o n o r d e r G(h ).
For Shepard's m e t h o d , i.e., m o v i n g least squares a p p r o x i m a t i o n w i t h c o n s t a n t
r e p r o d u c t i o n (i.e., m = 1 o r d = 0 ) , we saw above t h a t t h e g e n e r a t i n g f u n c t i o n s a r e
of t h e f o r m
w(xi, x)
î(x,x)
N
a n d f o r m a p a r t i t i o n o f u n i t y (see ( 2 3 . 3 ) ) . T h e r e f o r e t h e Lebesgue f u n c t i o n a d m i t s
the uniform bound
N
J2\î( , )\ x x
= !
i=l
T h i s shows t h a t S h e p a r d ' s m e t h o d has a p p r o x i m a t i o n o r d e r 0(h).
B o u n d i n g t h e Lebesgue f u n c t i o n i n t h e general case is m o r e i n v o l v e d a n d is
t h e s u b j e c t o f t h e papers [ L e v i n (1998); W e n d l a n d (2001a)]. A s i n d i c a t e d above,
d+1
t h i s results i n a p p r o x i m a t i o n o r d e r 0(h ) f o r a m o v i n g least squares m e t h o d
t h a t reproduces p o l y n o m i a l s o f degree d. I n b o t h p a p e r s t h e a u t h o r s assumed t h a t
t h e w e i g h t f u n c t i o n is c o m p a c t l y s u p p o r t e d , a n d t h a t t h e s u p p o r t size is scaled
p r o p o r t i o n a l t o t h e fill distance. T h e following theorem paraphrases t h e results o f
[ L e v i n (1998); W e n d l a n d (2001a)].
T h e o r e m 2 5 . 1 . Let ficl . s
If f e C (fl), d+1
{xi : i = 1,..., N} C fl are quasi-
uniformly distributed with fill distance h, the weight functions Wi(x) w(xi, x) are
compactly supported with support size pi = ch (c = c o n s t . ) , and if polynomials in
s
Ii d are reproduced according to (22.13), then the scale of MLS approximations
) i
^ (*) = E / ( * 0 * ( ^ ) , (25.3)
25. Error Bounds for Moving Least Squares Approximation 227
with generating functions ^(x x^) = \j/j(x,cc) determined via (22.17)-(22.19)

satisfies
d+1 a
I I Z - ^ l l o o < Ch max\D f()\, \a\ = d + l .
I t s h o u l d be possible t o a r r i v e at s i m i l a r e s t i m a t e s i f t h e w e i g h t f u n c t i o n o n l y
decays fast e n o u g h (see, e.g., t h e s u r v e y [ d e B o o r ( 1 9 9 3 ) ] ) .
A s i d e f r o m t h i s c o n s t r a i n t o n t h e w e i g h t f u n c t i o n ( w h i c h essentially c o r r e s p o n d s
t o a s t a t i o n a r y a p p r o x i m a t i o n scheme), t h e choice o f w e i g h t f u n c t i o n w does n o t
p l a y a role i n d e t e r m i n i n g t h e a p p r o x i m a t i o n o r d e r o f t h e m o v i n g least squares
m e t h o d . A s n o t e d earlier, i t o n l y d e t e r m i n e s t h e s m o o t h n e s s of Vf. For example,
i n t h e p a p e r [ C l e v e l a n d a n d L o a d e r (1996)] f r o m t h e s t a t i s t i c s l i t e r a t u r e o n l o c a l
regression t h e a u t h o r s s t a t e t h a t o f t e n "the choice [of w e i g h t f u n c t i o n ] is n o t t o o
c r i t i c a l " , a n d t h e use o f t h e so-called tri-cube
3 3 s
Wi(x) = {1 - \\x - Xi\\ ) , + x e R ,
is suggested. O f course, m a n y o t h e r w e i g h t f u n c t i o n s such as ( r a d i a l ) i3-splines or
any o f t h e (bell-shaped) r a d i a l basis f u n c t i o n s s t u d i e d earlier c a n also be used. I f
t h e w e i g h t f u n c t i o n is c o m p a c t l y s u p p o r t e d , t h e n t h e g e n e r a t i n g f u n c t i o n s \T/j w i l l
be so, t o o . T h i s leads t o c o m p u t a t i o n a l l y efficient m e t h o d s since t h e G r a m m a t r i x
G(x) w i l l be sparse.
An i n t e r e s t i n g q u e s t i o n is also t h e size o f t h e s u p p o r t o f t h e different l o c a l
w e i g h t f u n c t i o n s . O b v i o u s l y , a f i x e d s u p p o r t size for a l l w e i g h t f u n c t i o n s is possible.
H o w e v e r , t h i s w i l l cause serious p r o b l e m s as soon as t h e d a t a are n o t u n i f o r m l y
d i s t r i b u t e d . T h e r e f o r e , i n t h e a r g u m e n t s i n [ L e v i n (1998); W e n d l a n d (2001a)] t h e
a s s u m p t i o n is m a d e t h a t t h e d a t a are at least q u a s i - u n i f o r m l y d i s t r i b u t e d . A n o t h e r
choice for t h e s u p p o r t size o f t h e i n d i v i d u a l w e i g h t f u n c t i o n s is based o n t h e n u m b e r
of nearest neighbors, i.e., t h e s u p p o r t size is chosen so t h a t each o f t h e l o c a l w e i g h t
f u n c t i o n s c o n t a i n s t h e same n u m b e r o f centers i n i t s s u p p o r t . A t h i r d p o s s i b i l i t y is
suggested i n [Schaback (2000b)] w h e r e t h e a u t h o r proposes t o use a n o t h e r m o v i n g
least squares a p p r o x i m a t i o n based o n ( e q u a l l y spaced) a u x i l i a r y p o i n t s t o d e t e r m i n e
a s m o o t h f u n c t i o n 5 such t h a t at each e v a l u a t i o n p o i n t x t h e r a d i u s o f t h e s u p p o r t
of t h e w e i g h t f u n c t i o n is g i v e n b y 5(x). H o w e v e r , convergence e s t i m a t e s for these
l a t t e r t w o choices do n o t e x i s t .
Sobolev e r r o r estimates are p r o v i d e d for m o v i n g least squares a p p r o x i m a t i o n
w i t h compactly supported weight functions i n [ A r m e n t a n o (2001)]. T h e rates ob-
t a i n e d i n t h a t p a p e r are n o t i n t e r m s o f t h e f i l l d i s t a n c e b u t i n s t e a d i n t e r m s o f t h e
s u p p o r t size p o f t h e w e i g h t f u n c t i o n . M o r e o v e r , i t is assumed t h a t for g e n e r a l s
s d
a n d m = ( ^ ) t h e l o c a l L a g r a n g e f u n c t i o n s are b o u n d e d . A s m e n t i o n e d above, t h i s
is t h e h a r d p a r t , a n d i n [ A r m e n t a n o (2001)] s u c h b o u n d s are o n l y p r o v i d e d i n t h e
case s = 2 w i t h d = 1 or d = 2. H o w e v e r , i f c o m b i n e d w i t h t h e general b o u n d s for
t h e Lebesgue f u n c t i o n p r o v i d e d b y W e n d l a n d , t h e p a p e r [ A r m e n t a n o (2001)] y i e l d s
t h e f o l l o w i n g estimates:
d+1 a
\f(x) -V (x)\
f <Cp max\D f()\, \a\=d+l,
but also
| V ( / - P / ) ( x ) | <Cp d
max irr/COl, | | = d + 1.
I n t h e weaker ( l o c a l ) L 2 - n o r m w e have
H/-^/llL (B,nQ)2 <Cp d + l

\f\ w * + i {
and
l|V(/-7>/)|| L a ( B j nn) < C / | / | '(B.nn)'
where the balls B 3 provide a finite cover o f t h e d o m a i n fl, i.e., fl C [JjBj, and
d
t h e n u m b e r o f o v e r l a p p i n g b a l l s is b o u n d e d . H e r e W {fl) is t h e S o b o l e v space o f
f u n c t i o n s whose d e r i v a t i v e s u p t o o r d e r d are i n L 2 (c.f. C h a p t e r 13).
W e close t h i s c h a p t e r b y p o i n t i n g o u t t h a t e a r l y e r r o r e s t i m a t e s for some special
cases were p r o v i d e d i n [ F a r w i g ( 1 9 8 7 ) ; F a r w i g ( 1 9 9 1 ) ] .
.1-
Chapter 26
Approximate Moving Least Squares

Approximation
26.1 High-order Shepard Methods via Moment Conditions
W h i l e w e m e n t i o n e d earlier t h a t t h e w e i g h t f u n c t i o n does n o t have a n effect o n

t h e a p p r o x i m a t i o n o r d e r results for S h e p a r d ' s m e t h o d (cf. T h e o r e m 25.1), wee n o w
present some h e u r i s t i c c o n s i d e r a t i o n s for o b t a i n i n g h i g h e r - o r d e r S h e p a r d methods.
T h i s w i l l result i n c e r t a i n c o n d i t i o n s o n t h e m o m e n t s o f t h e w e i g h t f u n c t i o n w.
R e c a l l t h a t u s i n g o u r s h i f t e d m o n o m i a l r e p r e s e n t a t i o n we o b t a i n V/(x) = c\(x)
(see (22.5)) for a n y degree d. T h e r e f o r e , i f we c a n f i n d w e i g h t f u n c t i o n s w(xi,-)
such t h a t t h e first r o w ( a n d first c o l u m n ) o f t h e G r a m m a t r i x G(x) becomes
T T
[Y,Mxi,x),0,...,0] = [ < 1 , 1 ) _ , 0 , . . . , 0 ] , then Cl (x) = </, l>t__/<l, l)t__ via
(22.8). T h u s ,
i = i T,j=iw{xj,x)
d+1
b u t n o w b y c o n s t r u c t i o n t h e m e t h o d has a p p r o x i m a t i o n o r d e r 0(h ) (in-
s t e a d o f t h e m e r e 0(h) o f S h e p a r d ' s m e t h o d w h i c h p e r m i t s t h e use o f arbitrary
weights).
We therefore use t h e discrete moments (c.f. S e c t i o n 23.2)
N
Ha = Y2(
i=l
Xi
~ x
) a w x
( i , ) , x
x&R ,s
0 < | a | < d , (26.1)
w h e r e a is a m u l t i - i n d e x , a n d t h e n d e m a n d
/i = < W 0 < \a\ < d. (26.2)
A s m e n t i o n e d i n C h a p t e r 23 these m o m e n t s p r o v i d e t h e entries o f t h e G r a m m a t r i x
G(x). T h e c o n d i t i o n / i o = 1 ensures t h a t t h e w e i g h t s w(xi, ) f o r m a partition of
unity, a n d we e n d u p w i t h a q u a s i - i n t e r p o l a n t o f t h e f o r m
N
Vf(x) = y^J(xi)w(xi,x).
i=l
We can summarize our heuristics i n
229
s d+1
Proposition 2 6 . 1 . Let fl C R. If f e C (fl), the data sites {xi : i =
1, , N} C fl are quasi-uniformly distributed with fill distance h, the weight func-
tions w(xi, ) = w(xi x) are compactly supported with support size pi = ch
(c = c o n s t . ) , and the discrete moment conditions
Pa 0~<x0, 0 < \cx\ < d,
are satisfied, then
d+1
has approximation order 0(h ).
T h e discrete m o m e n t c o n d i t i o n s i n P r o p o s i t i o n 2 6 . 1 lead us t o t h e following

i n t e r p r e t a t i o n o f t h e w e i g h t f u n c t i o n w:
s
w(x,y) =w (x,y)q(x
0 -y), q e U, d (26.3)
w h e r e wo is a n e w ( a r b i t r a r y ) w e i g h t f u n c t i o n , a n d q is a p o l y n o m i a l o f degree d
o r t h o n o r m a l i n t h e sense o f t h e i n n e r p r o d u c t (22.2) w i t h respect t o WQ. This view
is also t a k e n b y L i u a n d c o - w o r k e r s (see, e.g., [ L i a n d L i u (2002)]) w h e r e q is c a l l e d
t h e correction function.
T h e discrete m o m e n t c o n d i t i o n s i n P r o p o s i t i o n 26.1 are d i f f i c u l t t o satisfy an-
a l y t i c a l l y , as is t h e c o n s t r u c t i o n o f d i s c r e t e m u l t i v a r i a t e o r t h o g o n a l p o l y n o m i a l s as
needed for (26.3). I n o r d e r t o o b t a i n q u a s i - i n t e r p o l a n t s for a r b i t r a r y space d i m e n -
sions a n d s c a t t e r e d d a t a sites we consider t h e c o n c e p t o f a p p r o x i m a t e approxima-
t i o n i n t h e n e x t section. T h e r e one satisfies c o n t i n u o u s m o m e n t c o n d i t i o n s w h i l e
e n s u r i n g t h a t t h e discrete m o m e n t c o n d i t i o n s are a l m o s t satisfied.
26.2 Approximate Approximation
W e n o w give t h e m a i n r e s u l t s o n a p p r o x i m a t e a p p r o x i m a t i o n relevant t o o u r w o r k .
T h e concept o f approximate approximation was first i n t r o d u c e d b y M a z ' y a i n t h e
e a r l y 1990s (see [ M a z ' y a ( 1 9 9 1 ) ; M a z ' y a ( 1 9 9 4 ) ] ) . T o keep t h e discussion t r a n s p a r e n t
we r e s t r i c t ourselves t o r e s u l t s for r e g u l a r center d i s t r i b u t i o n s . H o w e v e r , i r r e g u l a r l y
spaced d a t a are also a l l o w e d b y t h e t h e o r y (see, e.g., [ M a z ' y a a n d S c h m i d t ( 2 0 0 1 ) ;
L a n z a r a et al. (2006)]) a n d h a v e b e e n i n v e s t i g a t e d i n p r a c t i c e (see, e.g., [Fasshauer
(2004)] or [ L a n z a r a et al. ( 2 0 0 6 ) ] ) . I n [ M a z ' y a a n d S c h m i d t (2001)] t h e a u t h o r s
present a q u a s i - i n t e r p o l a t i o n scheme o f t h e f o r m
(26 4)
^ > w - f - " E / w ( w ) ' '
w h e r e t h e d a t a sites x u = hv are r e q u i r e d t o lie o n a r e g u l a r s - d i m e n s i o n a l g r i d w i t h
gridsize h. T h e p a r a m e t e r T> scales t h e s u p p o r t o f t h e g e n e r a t i n g f u n c t i o n \I>. A s we
26. Approximate Moving Least Squares Approximation 231
w i l l see b e l o w , t h e p a r a m e t e r T> c a n be chosen t o m a k e a so-called saturation error

so s m a l l t h a t i t does n o t affect n u m e r i c a l c o m p u t a t i o n s . T h e g e n e r a t i n g f u n c t i o n is
r e q u i r e d t o satisfy t h e continuous moment conditions
a
[ y V(y)dy = 6 a 0 , 0 < | a | < d. (26.5)
T h i s is t h e c o n t i n u o u s a n a l o g o f ( 2 6 . 2 ) .
T h e f o l l o w i n g a p p r o x i m a t e a p p r o x i m a t i o n r e s u l t is due t o M a z ' y a a n d S c h m i d t
(see, e.g., [ M a z ' y a a n d S c h m i d t ( 2 0 0 1 ) ] ) .
d + 1 S s s
T h e o r e m 2 6 . 1 . Let f <G C ( R ) , {x u : i / . Z } c R and let V be a continuous
generating function which satisfies the moment conditions (26.5) along with the
decay requirement
s
|*(aj)| < c (l K + ||a;|| )- / , 2 K 2
x e R,
where CK is some constant, K > d + s + 1 and d is the desired degree of polynomial
reproduction. Then
d+1
11/ - - M ^ H o o = 0(h ) + Eo(V, V). (26.6)
U s i n g Poisson's s u m m a t i o n f o r m u l a one c a n b o u n d t h e saturation error EQ b y

(see L e m m a 2.1 i n [ M a z ' y a a n d S c h m i d t (1996)])
E (*,V)<
0 ^2 F^{VVv). (26.7)
3
i^z \{0}
f
For t h i s result t h e F o u r i e r t r a n s f o r m o f \_ is defined v i a
TV(UJ) = [ 2nixoj
y{x)e- dx
JR 3
w i t h x UJ t h e s t a n d a r d E u c l i d e a n i n n e r p r o d u c t o f x a n d UJ i n M . T h e s a t u r a t i o n s
e r r o r can be i n t e r p r e t e d as t h e d i s c r e p a n c y b e t w e e n t h e c o n t i n u o u s a n d discrete m o -
m e n t c o n d i t i o n s , a n d i t s influence c a n be c o n t r o l l e d b y t h e choice o f t h e p a r a m e t e r
V i n (26.4).
I f we use r a d i a l g e n e r a t i n g f u n c t i o n s , t h e n we c a n use t h e f o r m u l a for t h e F o u r i e r
t r a n s f o r m o f a r a d i a l f u n c t i o n (see T h e o r e m B . l i n A p p e n d i x B ) t o c o m p u t e the
l e a d i n g t e r m o f (26.7), a n d t h e r e f o r e o b t a i n a n e s t i m a t e for D for a n y desired s a t u -
r a t i o n e r r o r . I f V is chosen large e n o u g h , t h e n t h e s a t u r a t i o n e r r o r w i l l be s m a l l e r
t h a n t h e m a c h i n e a c c u r a c y for a n y g i v e n c o m p u t e r , a n d therefore n o t noticeable
in numerical computations. T h i s means, t h a t even t h o u g h t h e o r e t i c a l l y t h e
q u a s i - i n t e r p o l a t i o n scheme (26.4) fails t o converge, i t does converge for a l l p r a c t i c a l
n u m e r i c a l purposes. M o r e o v e r , we c a n have a n a r b i t r a r i l y h i g h r a t e o f convergence,
d+1, by picking a generating f u n c t i o n ^ w i t h sufficiently m a n y vanishing moments.
W e p o i n t o u t t h a t t h e e r r o r b o u n d (26.6) is f o r m u l a t e d for g r i d d e d d a t a o n a n
u n b o u n d e d d o m a i n . A n analogous r e s u l t also h o l d s for f i n i t e g r i d s (see [ i v a n o v et al.
(1999)]); a n d s i m i l a r results for s c a t t e r e d d a t a o n b o u n d e d d o m a i n s c a n be f o u n d
i n , e.g., [ L a n z a r a et al. ( 2 0 0 6 ) ; M a z ' y a a n d S c h m i d t (2001)]. H o w e v e r , i n t h i s case
t h e f u n c t i o n / ( d e f i n i n g t h e " d a t a " ) is r e q u i r e d t o be c o m p a c t l y s u p p o r t e d o n t h e
f i n i t e d o m a i n , or a p p r o p r i a t e l y m o l l i f i e d .
26.3 C o n s t r u c t i o n o f G e n e r a t i n g F u n c t i o n s for A p p r o x i m a t e M L S
Approximation
I n o r d e r t o c o n s t r u c t t h e g e n e r a t i n g f u n c t i o n s for t h e a p p r o x i m a t e M L S a p p r o a c h
we assume t h e g e n e r a t i n g f u n c t i o n t o be r a d i a l a n d o f t h e f o r m
2 2
*(__) = M\\x\\ )q(\\x\\ )-
Therefore, t h e c o n t i n u o u s m o m e n t c o n d i t i o n s b e c o m e (c.f. (26.5))
2k 2 x 2
/ \\x\\ q(\\x\\ )' Po(\\ \\ )dx i = S Q, k 0<k<d, (26.8)
a n d we need t o d e t e r m i n e t h e u n i v a r i a t e p o l y n o m i a l q o f degree d a c c o r d i n g l y t o
2d+2
get a p p r o x i m a t i o n order 0(h ).
B y u s i n g s - d i m e n s i o n a l s p h e r i c a l c o o r d i n a t e s we c a n r e w r i t e t h e i n t e g r a l i n (26.8)
as
pOO p2n p7T p7T
/ / / . . . / r 2 f c
g(r )ô(r )r -Uin - 0i...sin^_ o?0i...d0 _2^c.r
2 2 a s 2
2 a
Jo Jo Jo Jo
pTT pTT pOO
s 2 2 k 2 2 1
= 2TT s i n - < M 0 i / sin0 _ d0 _ s 2 s 2 / r q ^ ) ^ ) ^ - dr
Jo Jo Jo
m 2k 2 2 s l
= 27rTT/ sin <j>d<j> / r q(r )^ (r )r - dr. 0 (26.9)
J o J o
m=i
2
The substitution y = r c o n v e r t s t h e last i n t e g r a l t o
/OO 1 poo
J r q(r )M^)r - dr
2k 2 a 1
= - J k
y q(y)ô{y)y - dy, {s 2)/2
(26.10)
a n d one c a n show t h a t
s-2
8 4 ^
/-7T g/ 2
m=l
C o m b i n i n g ( 2 6 . 9 ) , (26.10) a n d (26.11) gives us
r s/2
2 2 2
/ ||a || ^(||x|| )^(|||| )da: = -
J - - / y ^ q ^ M v ^ d y , 7 7
7K r (s/2) j 0
a n d therefore we n o w have o b t a i n e d a set o f o n e - d i m e n s i o n a l o r t h o g o n a l i t y c o n d i -
tions
fS /2
s/2 poo
roo
k 1 s/2
^7-7^ y - g(y)My)y dy = s k 0 , o<k<d (26.12)
T(
t h a t can be used t o d e t e r m i n e t h e g e n e r a t i n g f u n c t i o n
2 2
*(c)=^ (||x|| )5(|| || ) 0 a J
once we have chosen a n i n i t i a l w e i g h t ipo. M o r e o v e r , we k n o w t h a t t h i s c o n s t r u c t i o n

2d+2
ensures ^ t o have a p p r o x i m a t e a p p r o x i m a t i o n o r d e r 0(h ).
T h u s , t h e s t r a t e g y for c o n s t r u c t i n g g e n e r a t i n g f u n c t i o n s for h i g h e r - o r d e r ap-
p r o x i m a t e M L S a p p r o x m a t i o n is as follows:
(1) P i c k a n a r b i t r a r y ( u n i v a r i a t e ) w e i g h t f u n c t i o n ifto.
(2) C o m p u t e t h e coefficients o f q 6 IT^ v i a t h e ( u n i v a r i a t e ) m o m e n t conditions
(26.12).
(3) This leads to the (multivariate) radial generating function vE^cc) =
, 2 2
V o ( | | ^ | | ) 9 ( | | a ; | | ) t o be used i n t h e q u a s i - i n t e r p o l a n t ( 2 6 . 4 ) .
y
E x a m p l e 2 6 . 1 . P r o b a b l y t h e m o s t a e s t h e t i c e x a m p l e is g i v e n b y ifio(y) e~ so
t h a t t h e basic g e n e r a t i n g f u n c t i o n ( c o r r e s p o n d i n g t o d = 0, i.e., w i t h approximate
2
a p p r o x i m a t i o n o r d e r 0(h ) is f o u n d b y a s s u m i n g t h a t t h e p o l y n o m i a l q is a c o n s t a n t ,
i.e., q(y) = an. T h e n (26.12) becomes
aoy(s-z)/^ -y
('-We-ydye dy =
=
JO0 7T S /
- s 2
w h i c h leads t o an = 7 r / , SO t h a t w e h a v e
-llll a
V 7T
a n a p p r o p r i a t e l y scaled G a u s s i a n .
4
I f we t a k e d = 1 ( t o o b t a i n a p p r o x i m a t e a p p r o x i m a t i o n o r d e r 0(h )) a n d assume
q t o be a linear p o l y n o m i a l o f t h e f o r m q(y) = an + a\y, t h e n t h e r e are t w o m o m e n t
conditions, namely
'(ao + - / ) y ( - W e - d ! , = s a
S # ,
I l !
s/2
n
Jo
f
a 2
(ao + a )y / e-vdy iy = 0,
or Jo
a r( /2) +
0 S a i r ( ( s + 2)/2) =
--s/_
a r ( ( s + 2 ) / 2 ) + a _ r ( ( s + 4 ) / 2 ) = 0.
0
W e can solve t h i s s y s t e m o f l i n e a r e q u a t i o n s a n d o b t a i n
s + 2 -1
an 2 ^ F = , ai
I n t h e special case s = 2 t h i s y i e l d s an = ^ a n d a i = ^ , so t h a t
y(x) = - ( 2 - \ \ x \ \ ) e - ^ . 2 2
7T
I n general, t h e p o l y n o m i a l s q t u r n o u t t o be ( u n i v a r i a t e ) g e n e r a l i z e d Laguerre
p o l y n o m i a l s Lj o f degree d (c./. S e c t i o n 4.2) w h i c h are k n o w n t o be o r t h o g o -
s 2 v
n a l o n t h e i n t e r v a l [0, oo) w i t h respect t o t h e w e i g h t f u n c t i o n y l e~ . Therefore
t h e g e n e r a t i n g f u n c t i o n s for t h e a p p r o x i m a t e M L S a p p r o x i m a t i o n m e t h o d are t h e
Laguerre-Gaussians
*(__) = - ^ e - ^ L f ^ x f ) .
I n p a r t i c u l a r , T a b l e 4 . 1 c o n t a i n s specific e x a m p l e s for s = 1,2,3 and d = 1 and 2

except for t h e scale f a c t o r lfy/ft*. F i g u r e 4 . 1 shows p l o t s o f t h e g e n e r a t i n g f u n c t i o n s
i n t h e cases s = 1, d = 2, a n d s = 2, d = 2.
Example 26.2. I f we use t h e f u n c t i o n ipo(y) = ( l y/y)^_ (4^/y + 1) as initial

weight, t h e n we can p e r f o r m calculations analogous t o those above. For d = 0 a n d
s = 2 we get
2 2
*(__) = ~ V o ( N | ) = ~ (1 - \\x\\)l (411x11 + 1 ) , xe K , (26.13)
7T TV ^
2
w i t h a p p r o x i m a t i o n o r d e r 0(h ). E x c e p t for t h e f a c t o r 7/-7T t h i s g e n e r a t i n g func-
2
t i o n corresponds t o Wendland's c o m p a c t l y s u p p o r t e d C r a d i a l basic f u n c t i o n (c.f.
T a b l e 11.1).
U s i n g t h e same f u n c t i o n ipo for d = 1 a n d s = 2 w e o b t a i n
* ( x ) = * , ( | | _ f ) I @ - ^ | | f ) (26.14)
A
w i t h a p p r o x i m a t i o n o r d e r 0(h ). T h i s f u n c t i o n is d i s p l a y e d i n t h e left p l o t o f
F i g u r e 2 6 . 1 . See E x a m p l e s 27.2 a n d 27.4 f o r m o r e s p e c i a l cases b a s e d o n t h i s i n i t i a l
weight function.
E x a m p l e 2 6 . 3 . M a n y o t h e r choices for t h e i n i t i a l w e i g h t f u n c t i o n ipo are p o s s i b l e .

F o r e x a m p l e , w e c a n t a k e tpo(y) = (1 y ) ( l + y ) a / 3
, t h e weight f u n c t i o n for u n i v a r i a t e
J a c o b i p o l y n o m i a l s ( w h i c h are o r t h o g o n a l o n [ 1 , 1 ] ) . Since t h e i n t e g r a l d e f i n i n g
2 f c s 2 2
the orthogonality relations contains an e x t r a factor of y ( + - )/ j the m o m e n t con-
d i t i o n s (26.12) d o n o t y i e l d J a c o b i p o l y n o m i a l s . H o w e v e r , t h e r e s u l t i n g g e n e r a t i n g
f u n c t i o n s for a p p r o x i m a t e M L S a p p r o x i m a t i o n c a n s t i l l be r a t h e r s i m p l e . I n Ta-
b l e 2 6 . 1 w e l i s t several s u c h e x a m p l e s f o r s = 2, (3 = 0 a n d v a r i o u s combinations
of d a n d a. N o t e t h a t these f u n c t i o n s are also c o m p a c t l y s u p p o r t e d , i.e., t h e y are
d e f i n e d t o be zero for ||CE|| > 1.
Table 26.1 Approximate M L S generating functions \& based on ipo(y) = (1 y ) > a
y e [1,1] for various choices of d and a.
d q = 2 a = 5/2
0 -(i-INI ) 2 2
^(1-llxii ) / 2 5 2
7T 2.TT
1 * (2 - 5 I M I ) ( 1 - | | * | | )
2 2 2
i - (4 - 1 1 I M I ) ( 1 - | | x | | ) /
2
2 5 2
2 - ( 1 - 6||*|| + 7 I M I ) ( 1 - I N I )
2 4 2 2
(8 - 5 2 I M I + 6 5 I M I ) ( 1 -
2
4
||^|| ) 2 5 / 2
7T 107T
2 2 2
T h e f u n c t i o n * ( x ) = ^ (2 5 | | i c | | ) ( 1 - | | c c | | ) is d i s p l a y e d i n t h e r i g h t p l o t o f
Figure 26.1.
Fig. 26.1 Compactly supported generating functions for approximate linear reproduction.
2 2 2
= ( ! - W I M P ) (1 - I M D i (4||*|| + 1) (left) and = (2 - 5 | M | ) (1 - M )
2
(right) centered at the origin in R .
Chapter 27
Numerical Experiments for Approximate

M L S Approximation
I n t h i s c h a p t e r we present a series o f e x p e r i m e n t s for a p p r o x i m a t e M L S a p p r o x -

i m a t i o n w i t h b o t h globally supported Laguerre-Gaussian g e n e r a t i n g f u n c t i o n s as
w e l l as w i t h c o m p a c t l y s u p p o r t e d g e n e r a t i n g f u n c t i o n s based o n t h e i n i t i a l w e i g h t
ipo(y) = ( l \/y) + + 1) as i n E x a m p l e 26.2 o f t h e p r e v i o u s c h a p t e r .
27.1 Univariate Experiments
Example 27.1. We begin w i t h univariate globally supported Laguerre-Gaussians.

T h e s e f u n c t i o n s are l i s t e d i n T a b l e 4 . 1 e x c e p t for t h e s c a l i n g f a c t o r required
for t h e I D case. I n t h e left p l o t o f F i g u r e 27.1 w e i l l u s t r a t e t h e effect t h e s c a l i n g
p a r a m e t e r T> has o n t h e convergence b e h a v i o r for G a u s s i a n g e n e r a t i n g f u n c t i o n s .
W e use a m o l l i f i e d u n i v a r i a t e F r a n k e - l i k e f u n c t i o n o f t h e f o r m
,, x , _ T- 1
A3 (9x-2) 32
(9* + l ) 2
1 (9*-7) 2
1 _(-q__ ' 2\ 4 )
a 9x
f(x) = 15e i-(2-D - e 4 + e 4 9 + e 4 _ _ e ( V
\4 4 2 5 /
as t e s t f u n c t i o n . For each choice o f T> e { 0 . 4 , 0 . 8 , 1 . 2 , 1 . 6 , 2 . 0 } w e use a sequence o f
k
g r i d s o f N = 2 + 1 ( w i t h k = 1 , . . . , 14) e q u a l l y spaced p o i n t s i n [ 0 , 1 ] at w h i c h we
sample t h e test f u n c t i o n . T h e a p p r o x i m a n t is c o m p u t e d v i a
N 2
1 - )
{
v (x)
f = -== f M e - - ^ ~ , x e [0,1],
w h e r e h = 1/{N 1 ) . T h i s c o r r e s p o n d s t o o u r u s u a l shape p a r a m e t e r e h a v i n g a
value of
fc
1 N - 1 2
e =
i.e., we are i n t h e r e g i m e o f s t a t i o n a r y a p p r o x i m a t i o n . T h e effect o f T> is c l e a r l y

2
v i s i b l e i n t h e f i g u r e . A v a l u e o f T> > 2 e x h i b i t s a n a p p r o x i m a t i o n o r d e r o f G(h )
t h r o u g h o u t t h e r a n g e o f o u r e x p e r i m e n t s , w h i l e s m a l l e r values a l l o w t h e s a t u r a t i o n
e r r o r t o creep i n at earlier stages.
237
Fig. 27.1 Convergence of I D approximate M L S approximation. T h e left plot shows the effect
of various choices of T> on the convergence behavior of Gaussians. T h e right plot illustrates the
convergence of Laguerre-Gaussians for various values of d.
I n the r i g h t p l o t o f F i g u r e 27.1 we c o m p a r e the a p p r o x i m a t i o n orders achievable

w i t h the Laguerre-Gaussians o f o r d e r s d = 0 , 1 , 2 i n I D . T h e r e s p e c t i v e values o f
T> are T> = 2 , 4 , 6 . T h e steepest sections o f t h e c u r v e s c o r r e s p o n d t o a p p r o x i m a t e
2 0 4 0 5
a p p r o x i m a t i o n o r d e r s o f 0(h - ), 0(h ), a n d 0(h "), r e s p e c t i v e l y a perfect
m a t c h w i t h the rates predicted b y the theory. N o t i c e t h a t for t h e second-order
L a g u e r r e - G a u s s i a n we have convergence a l l t h e w a y t o m a c h i n e accuracy.
T h e M A T L A B p r o g r a m ApproxMLSApproxlD .m (see P r o g r a m 27.1) was used t o
generate the r i g h t p l o t i n F i g u r e 2 7 . 1 . W e define t h e t h r e e d i f f e r e n t Laguerre-
G a u s s i a n g e n e r a t i n g f u n c t i o n s as m e m b e r s o f a M A T L A B cell a r r a y r b f a n d p l a c e
t h e c o r r e s p o n d i n g values o f T> t o be used w i t h each o f t h e f u n c t i o n s i n t h e v e c t o r
D (see lines 1-4). T h e u n i v a r i a t e F r a n k e - l i k e t e s t f u n c t i o n is d e f i n e d i n lines 5 - 1 0 .
T h i s f u n c t i o n is m o l l i f i e d so t h a t i t goes t o zero s m o o t h l y a t t h e b o u n d a r i e s o f t h e
i n t e r v a l . T h e p r o g r a m c o n t a i n s t w o f o r - l o o p s . T h e f i r s t is over t h e t h r e e d i f f e r e n t
generating functions (corresponding t o approximate constant, linear and quadratic
r e p r o d u c t i o n , r e s p e c t i v e l y ) . T h e i n n e r l o o p p e r f o r m s a series o f e x p e r i m e n t s with
an increasing number N o f d a t a . H e r e w e p e r f o r m 14 i t e r a t i o n s w i t h N ranging
f r o m N = 3 to N 16385.
For applications o f a p p r o x i m a t e M L S a p p r o x i m a t i o n we l i m i t ourselves to
uniformly spaced d a t a since there are presently no robust methods for deal-
ing w i t h nonuniform data (see [Lanzara et al. (2006); M a z ' y a and Schmidt
(2001)] for a t h e o r e t i c a l a p p r o a c h t o n o n - u n i f o r m d a t a , a n d [Fasshauer ( 2 0 0 4 ) ;
L a n z a r a et al. ( 2 0 0 6 ) ] for some n u m e r i c a l e x p e r i m e n t s ) . A l l w e need i n o r d e r t o
c o m p u t e t h e a p p r o x i m a n t is t h e e v a l u a t i o n m a t r i x EM c o m p u t e d o n l i n e 23, w h i c h is
s 2
t h e n m u l t i p l i e d b y t h e f u n c t i o n values f a n d scaled b y t h e f a c t o r T>~ l o n l i n e 24.
T h e c o m m a n d s n e e d e d t o p r o d u c e t h e p l o t are i n c l u d e d o n lines 15, 27 a n d 2 9 - 3 1 .
27. Numerical Experiments for Approximate MLS Approximation 239
P r o g r a m 27.1. ApproxMLSApproxlD.m
% ApproxMLSApproxlD
% S c r i p t t h a t performs ID approximate MLS approximation
% Laguerre-Gaussians f o r ID
1 r b f { l } = @(e,r) e x p ( - ( e * r ) . ~ 2 ) / s q r t ( p i ) ;
2 r b f { 2 } = <3(e,r) e x p ( - ( e * r ) . ~ 2 ) / s q r t ( p i ) . * ( 1 . 5 - ( e * r ) . ~ 2 ) ;
3a r b f { 3 } = @(e,r) e x p ( - ( e * r ) . ~ 2 ) / s q r t ( p i ) . * . . .
3b (1.875-2.5*(e*r).~2+0.5*(e*r).~4);
4 D = [ 2 , 4, 6] ; % S c a l e parameters f o r g e n e r a t i n g f u n c t i o n s
% Define F r a n k e - l i k e f u n c t i o n a s t e s t f u n c t i o n
5 f l = @(x) 0.75*exp(-(9*x-2).~2/4);
6 f 2 = @(x) 0 . 7 5 * e x p ( - ( 9 * x + l ) . ~ 2 / 4 9 ) ;
7 f 3 = @(x) 0 . 5 * e x p ( - ( 9 * x - 7 ) . ~ 2 / 4 ) ;
8 f 4 = @(x) 0.2*exp(-(9*x-4) . "D \
9 moll = @(x) 1 5 * e x p ( - l . / ( l - 4 * ( x - 0 . 5 ) . ~ 2 ) ) ;
10 t e s t f u n c t i o n = @(x) m o l l ( x ) . * ( f 1 ( x ) + f 2 ( x ) + f 3 ( x ) - f 4 ( x ) ) ;
11 maxlevel = 14; % number of i t e r a t i o n s
12 M = 200; % to create M evaluation points i n unit i n t e r v a l
13 xe = l i n s p a c e ( 0 , 1 , M ) ; e p o i n t s = x e ( : ) ;
14 exact = t e s t f u n c t i o n ( e p o i n t s ) ;
5
15 f i g u r e ; hold on; cword = c e l l s t r ( [ r - ';'g';'b: ' ] ) ;
16 f o r i = l : l e n g t h ( D )
17 f o r k=l:maxlevel
18 N(k) = ( 2 ~ k + l ) ; ep = ( N ( k ) - l ) / s q r t ( D ( i ) ) ;
19 name = s p r i n t f ( ' D a t a l D . X d u ' , N ( k ) ) ; load(name);
20 ctrs = dsites;
% C r e a t e v e c t o r of f u n c t i o n v a l u e s
21 f = testfunction(dsites);
22 DM = D i s t a n c e M a t r i x ( e p o i n t s , c t r s ) ;
23 EM = r b f { i } ( e p , D M ) ;
'/, Compute approximate MLS approximation
24 Pf = E M * f / s q r t ( D ( i ) ) ;
% Compute RMS e r r o r on e v a l u a t i o n g r i d
25 rms_err(k) = norm(Pf-exact)/sqrt(M);
26 end
27 p l o t (N,rms_err, cword-Ci}) ;
28 end
29 set(gca,'XScale','log','YScale','log','Fontsize',14)
30 legend('d=0, D=2.0','d=l, D=4.0','d=2, D = 6 . 0 \ 3 ) ;
31 xlabeK'N'); y l a b e K 'Error') ; hold o f f
E x a m p l e 2 7 . 2 . I n t h e second set o f e x p e r i m e n t s w e use c o m p a c t l y s u p p o r t e d gen-

e r a t i n g f u n c t i o n s w i t h i n i t i a l w e i g h t ipo(y) = ( l -\/y) + ( 4 ^ / y + l ) . I n the univariate
case these f u n c t i o n s are for d = 0 , 1 , 2
*(x) = l(l-\x\)$4\x\ + l),
2
= | (7 - 35|as| ) ( 1 - \x$% (4\x\ + 1 ) , (27.1)
I AC
*(x) = YY993 ( 3 5 0
- 3 9 6
l^| 2 4
+ 7 2 9 3 | a : | ) ( 1 - \x\)4
+ (4\x\ + 1 ) .
These f u n c t i o n s c a n be c o m p u t e d as i n S e c t i o n 26.3 o f t h e p r e v i o u s c h a p t e r . An
2
a p p r o x i m a t e a p p r o x i m a t i o n o r d e r o f 0(h ) f o r t h e f i r s t f u n c t i o n i n (27.1) is i l l u s -
t r a t e d i n t h e left p l o t o f F i g u r e 27.2. N o t e t h a t a r a t h e r l a r g e v a l u e o f T> ( n a m e l y
T> ?s 500) is r e q u i r e d t o m a k e t h e s a t u r a t i o n e r r o r so s m a l l t h a t i t n o l o n g e r affects
our experiments. Since t h e shape p a r a m e t e r e d e t e r m i n e s t h e s u p p o r t r a d i u s o f
our g e n e r a t i n g f u n c t i o n s , a n d since e = l/y/Dh = (N l ) / \ / ^ , we see t h a t t h e
e v a l u a t i o n m a t r i x w i l l be c o m p l e t e l y dense u n t i l N g r o w s above a p p r o x i m a t e l y 25.
F u r t h e r m o r e , for such a large v a l u e o f D t h e r e is a v i s i b l e s m o o t h i n g effect ( v e r y
slow convergence) d u r i n g t h e first few i t e r a t i o n s w i t h N = 3, 5 , 9 , 1 7 , 3 3 , 65.
I n t h e r i g h t p l o t o f F i g u r e 27.2 w e c o m p a r e t h e a p p r o x i m a t i o n o r d e r s achiev-
able w i t h t h e u n i v a r i a t e c o m p a c t l y s u p p o r t e d g e n e r a t i n g f u n c t i o n s (27.1) o f o r d e r s
d = 0,1,2. T h e r e s p e c t i v e values o f T> r e q u i r e d t o p r e v e n t t h e s a t u r a t i o n e r r o r
f r o m c o r r u p t i n g o u r e x p e r i m e n t s are T> = 5 0 0 , 5 0 0 0 , 20000, r e s p e c t i v e l y . N o t e t h a t
T> = 20000 i m p l i e s t h a t t h e e v a l u a t i o n m a t r i x w i l l be dense ( a n d t h e r e f o r e c o m p u -
t a t i o n a l l y inefficient) u n t i l N reaches a b o u t 150. T h u s , i t is n o t u n t i l t h e v e r y l a s t
i t e r a t i o n s w i t h N = 8193 a n d N = 16385 t h a t we get t o t a k e r e a l a d v a n t a g e o f t h e
c o m p a c t s u p p o r t o f t h e g e n e r a t i n g f u n c t i o n , i.e., h a v e sparse sums. T h e steepest
20
sections o f t h e curves c o r r e s p o n d t o a p p r o x i m a t e a p p r o x i m a t i o n o r d e r s o f 0(h ),
3 m 5A5
0(h ) , a n d 0(h ), respectively.
The M A T L A B code for t h e c o m p a c t l y s u p p o r t e d e x p e r i m e n t s c a n be writ-
ten i n t w o ways. One possibility w o u l d be t o r e w r i t e t h e g e n e r a t i n g func-
t i o n s i n s h i f t e d f o r m as e x p l a i n e d i n C h a p t e r 12, a n d t h e n use t h e sparse code
D i s t a n c e M a t r i x C S R B F . m . H o w e v e r , as j u s t e x p l a i n e d , we c a n n o t t a k e m u c h a d v a n -
t a g e o f t h e s p a r s i t y u s u a l l y associated w i t h c o m p a c t l y s u p p o r t e d f u n c t i o n s . T h e r e -
fore, we use essentially t h e same code as i n P r o g r a m 2 7 . 1 . T h e o n l y changes needed
are t h e s u b s t i t u t i o n o f t h e d e f i n i t i o n o f t h e c o m p a c t l y s u p p o r t e d g e n e r a t i n g f u n c -
t i o n s for t h e L a g u e r r e - G a u s s i a n s a l o n g w i t h a p p r o p r i a t e values for D.
Our e x p e r i m e n t s seem t o suggest t h a t t h e r e is n o p o i n t i n u s i n g c o m p a c t l y sup-

p o r t e d g e n e r a t i n g f u n c t i o n s for u n i v a r i a t e a p p r o x i m a t e M L S a p p r o x i m a t i o n . T h e
results u s i n g L a g u e r r e - G a u s s i a n s are far m o r e a c c u r a t e , a n d d u e t o t h e l a r g e v a l u e
o f V r e q u i r e d for t h e c o m p a c t l y s u p p o r t e d f u n c t i o n s t h e y do n o t offer a n a d v a n t a g e
in terms of c o m p u t a t i o n a l complexity, either.
27. Numerical Experiments for Approximate MLS Approximation 241
Fig. 27.2 Convergence of I D approximate M L S approximation with compactly supported gener-

ating functions. The left plot shows the effect of various choices of T> on the convergence behavior
for the first function in (27.1). T h e right plot illustrates the convergence for the three functions
in (27.1).
27.2 Bivariate Experiments
E x a m p l e 2 7 . 3 . T h i s e x a m p l e is s i m i l a r t o t h e second p a r t o f E x a m p l e 2 7 . 1 . N o w
we use b i v a r i a t e L a g u e r r e - G a u s s i a n s w i t h scale f a c t o r 1/TT. T h e t e s t d a t a are s a m -
p l e d f r o m a b i v a r i a t e m o l l i f i e d F r a n k e f u n c t i o n , i.e., we m u l t i p l y F r a n k e ' s f u n c t i o n
(2.2) by the mollifier
_ i 2
_ i
g(x,y) = l$e i-(2*-i) e i-w -i)*
v .
fc 2
T h e d a t a sites are u n i f o r m g r i d s o f ( 2 + l ) p o i n t s ( w i t h k = 1 , . . . , 5) i n t h e u n i t
square. T h e values f o r t h e scale p a r a m e t e r T> used are V = 1 , 2 , 2 . 5 . T h e steep-
est sections o f t h e e r r o r curves c o r r e s p o n d t o a p p r o x i m a t e a p p r o x i m a t i o n o r d e r s o f
1 8 3 2 8 0 3 00
0(h ) , 0(h ) , a n d 0(h - ), r e s p e c t i v e l y . N o t e t h a t these r a t e s d o n o t m a t c h
t h e theoretically predicted orders. W e w i l l see t h a t we c a n achieve t h e t h e o r e t i -
c a l l y p r e d i c t e d orders b y u s i n g m o r e d a t a sites. T h i s , h o w e v e r , w i l l r e q u i r e s p e c i a l
e v a l u a t i o n t e c h n i q u e s t o d e a l w i t h t h e l a r g e s u m s efficiently (see t h e n e x t c h a p t e r ) .
E x a m p l e 27.4. T h e bivariate c o m p a c t l y supported generating functions w i t h ini-

t i a l w e i g h t tpo(y) = ( l y/y) + ( 4 ^ / y + 1) p r o v i d i n g a p p r o x i m a t e r e p r o d u c t i o n o f
c o n s t a n t s , l i n e a r a n d q u a d r a t i c p o l y n o m i a l s , r e s p e c t i v e l y , are (c.f. E x a m p l e 26.2)
4
*(*) = - (i-N|) + ( 4 H | + i),
252
= ( U 5 5 N | 2 ( 1 { { X { 1 ) ( 4 | W I + 1 }
229^ " ) " + '
=
llluW
( 4 1 ? 9
" 3705
H ll a; 2
+ 59605||tf|| ) ( 1 - H I ) ! (411*11 + 1 ) .4
T h e values we use for t h e scale p a r a m e t e r are T> = 2 0 , 4 0 , 80, r e s p e c t i v e l y . The

steepest sections o f t h e e r r o r c u r v e s c o r r e s p o n d t o a p p r o x i m a t e a p p r o x i m a t i o n o r -
Fig. 27.3 Convergence of 2D approximate M L S approximation with Laguerre-Gaussians (left)

and compactly supported (right) generating functions for various values of d.
196 3 0 3 3 26
ders o f 0(h ), 0(h ) , a n d 0(h - ), respectively. A g a i n , we do not o b t a i n a
m a t c h w i t h t h e t h e o r e t i c a l l y p r e d i c t e d o r d e r s , even t h o u g h w e used u p t o N = 66049
data points.
T h e b i v a r i a t e case p r o v i d e s a m o r e level p l a y i n g f i e l d for t h e c o m p a c t l y sup-

p o r t e d generating functions. W e can take advantage of the compact support and
o b t a i n m o r e a c c u r a t e r e s u l t s a t a r e a s o n a b l e cost. However, another alternative
w a y o f o b t a i n i n g h i g h l y a c c u r a t e m u l t i v a r i a t e a p p r o x i m a t e M L S a p p r o x i m a t i o n s is
presented i n the next chapter.
i . 1 :
Chapter 28
Fast Fourier Transforms
28.1 NFFT
I n t h e recent p a p e r s [ K u n i s et al. ( 2 0 0 2 ) ; N i e s l o n y et al. ( 2 0 0 4 ) ; P o t t s a n d S t e i d l

(2003)] use o f t h e fast Fourier transform for non-uniformly spaced points was sug-
gested as a n efficient w a y t o solve a n d e v a l u a t e r a d i a l basis f u n c t i o n p r o b l e m s . T h e
C++ software package N F F T b y t h e a u t h o r s is a v a i l a b l e for free d o w n l o a d [ K u n i s
a n d P o t t s (2002)]. A discussion o f t h e a c t u a l N F F T s o f t w a r e w o u l d go b e y o n d t h e
scope o f t h i s b o o k . I n s t e a d , we b r i e f l y describe h o w t o use N F F T s a n d F F T s t o
s i m u l t a n e o u s l y e v a l u a t e expansions o f t h e f o r m
fc=l
a t m a n y e v a l u a t i o n p o i n t s yj, j = 1,..., M. N o t e t h a t t h i s covers n o t o n l y a p p r o x -

i m a t e M L S a p p r o x i m a t i o n s , b u t also t h e e v a l u a t i o n o f o t h e r q u a s i - i n t e r p o l a n t s as
w e l l as t h e e v a l u a t i o n o f R B F i n t e r p o l a n t s .
D i r e c t s u m m a t i o n o f (28.1) r e q u i r e s O(MN) o p e r a t i o n s , w h i l e i t c a n be s h o w n
t h a t use o f t h e N F F T reduces t h e cost t o 0(M + N) o p e r a t i o n s . T h e r e f o r e , as is
a l w a y s t h e case w i t h fast F o u r i e r t r a n s f o r m s , use o f t h e a l g o r i t h m w i l l p a y o f f for
sufficiently m a n y e v a l u a t i o n s .
I n t h e i r p a p e r s N i e s l o n y , P o t t s a n d S t e i d l d i s t i n g u i s h b e t w e e n basic f u n c t i o n s <3>
t h a t are s i n g u l a r a n d t h o s e t h a t are n o n - s i n g u l a r . S i n g u l a r basic f u n c t i o n s are C
e v e r y w h e r e except a t t h e o r i g i n a n d i n c l u d e e x a m p l e s s u c h as
2
-, \ , logr, r l o g r ,
w h e r e r = || ||. N o n - s i n g u l a r basic f u n c t i o n s are s m o o t h e v e r y w h e r e such as Gaus-

sians a n d (inverse) m u l t i q u a d r i c s . W e w i l l r e s t r i c t o u r discussion t o t h i s l a t t e r
class.
T h e basic idea for t h e f o l l o w i n g a l g o r i t h m is r e m a r k a b l y s i m p l e . I t relies o n t h e
a y Xk _ Q J / a a : f c
fact t h a t t h e e x p o n e n t i a l e~ ^ ^~ ^ c a n be w r i t t e n as e Je . Moreover, the
m e t h o d applies t o a r b i t r a r y basic f u n c t i o n s ( w h i c h is i n s t r o n g c o n t r a s t t o t h e fast
243
m u l t i p o l e t y p e m e t h o d s discussed i n C h a p t e r 3 5 . O n e s t a r t s o u t b y a p p r o x i m a t i n g
t h e ( a r b i t r a r y , b u t s m o o t h ) basic f u n c t i o n <fr u s i n g s t a n d a r d F o u r i e r series, i.e.,
$(x) J2 bee 2 7 r i x
(28.2)
S
w i t h a m u l t i - i n d e x i n t h e i n d e x set I n = [f, f ) - T h e coefficients bi a r e f o u n d
b y t h e d i s c r e t e inverse F o u r i e r t r a n s f o r m
<4S*() e- ' 2 i f e i /
" (28.3)
fc -/"n
N u m e r i c a l l y , t h i s t a s k is a c c o m p l i s h e d w i t h s o f t w a r e f o r t h e s t a n d a r d (inverse) F F T
{e.g., [FFTW]).
Remark 2 8 . 1 . N o t e t h a t t h i s d e f i n i t i o n o f t h e F o u r i e r t r a n s f o r m (as w e l l as t h e
one b e l o w ) is different f r o m t h e o n e used t h r o u g h o u t t h e rest o f t h i s b o o k . H o w e v e r ,
i n o r d e r t o s t a y closer t o t h e s o f t w a r e packages, w e a d o p t t h e n o t a t i o n used t h e r e .
U s i n g t h e r e p r e s e n t a t i o n ( 2 8 . 2 ) o f t h e basic f u n c t i o n <> w e c a n r e w r i t e ( 2 8 . 1 ) as
fc=i eein
N
<=I n fc=l
Now, t h e e x p o n e n t i a l is s p l i t u s i n g t h e a b o v e m e n t i o n e d p r o p e r t y , i.e.,
eei n k=i
T h i s , h o w e v e r , c a n be v i e w e d as a fast F o u r i e r t r a n s f o r m a t t h e n o n - u n i f o r m l y
spaced p o i n t s yj, i.e.,
v { )^j2 ^ ' -
f yj
d 2nie Vi
w h e r e t h e coefficients di = biag with
2
a e = f 2 c k e - ^ ,
k=l
w h i c h i n t u r n is n o t h i n g b u t a n inverse d i s c r e t e F o u r i e r t r a n s f o r m a t t h e n o n -
u n i f o r m l y spaced p o i n t s x . T h e s e l a t t e r t w o t r a n s f o r m s are d e a l t w i t h n u m e r i c a l l y
k
using t h e N F F T software.
T o g e t h e r , for t h e case o f n o n - s i n g u l a r basic f u n c t i o n s <&, w e h a v e t h e f o l l o w i n g
algorithm.
28. Fast Fourier Transforms 245
A l g o r i t h m 28.1. Fast Fourier t r a n s f o r m evaluation
F o r e l n
C o m p u t e t h e coefficients
b y inverse F F T .
C o m p u t e t h e coefficients
fc=i
by inverse N F F T .
C o m p u t e t h e coefficients di = agbt-
end
For 1 < j < M
C o m p u t e t h e values
Vf(y )
3 * ] T die *"*
2
by N F F T .
end
I n t h e p a p e r s [ K u n i s et al. ( 2 0 0 2 ) ; N i e s l o n y et al. ( 2 0 0 4 ) ; P o t t s a n d S t e i d l (2003)]

t h e a u t h o r s suggest a s p e c i a l b o u n d a r y r e g u l a r i z a t i o n i n case t h e basic f u n c t i o n
does n o t decay fast e n o u g h , i.e., t h e basic f u n c t i o n is large n e a r t h e b o u n d a r y o f
t h e d o m a i n . H o w e v e r , for o u r e x p e r i m e n t s w i t h L a g u e r r e - G a u s s i a n s r e p o r t e d i n t h e
n e x t s e c t i o n t h i s is n o t a n issue.
O f course, t h i s m e t h o d w i l l o n l y p r o v i d e a n a p p r o x i m a t i o n o f t h e expansion
(28.1) a n d e r r o r e s t i m a t e s are p r o v i d e d i n t h e l i t e r a t u r e (see, e.g., [ N i e s l o n y et al.
(2004)]).
W h i l e w e o n l y i l l u s t r a t e t h e use o f ( N ) F F T s for t h e e v a l u a t i o n o f r a d i a l s u m s i t
s h o u l d be clear t h a t t h i s m e t h o d c a n also be c o u p l e d w i t h a n y o t h e r a l g o r i t h m t h a t
is based o n e v a l u a t i o n o f fast s u m m a t i o n a t n o n - u n i f o r m p o i n t s (such as t h e p r e c o n -
d i t i o n e d G M R E S a l g o r i t h m o f S e c t i o n 34.3, t h e "greedy" a l g o r i t h m o f S e c t i o n 3 3 . 1 ,
or t h e F a u l - P o w e l l a l g o r i t h m o f S e c t i o n 3 3 . 2 ) .
28.2 Approximate M L S Approximation via Non-uniform Fast

Fourier Transforms
A few e x a m p l e s t h a t i l l u s t r a t e t h e use o f fast F o u r i e r t r a n s f o r m s for t h e evalua-

t i o n o f a p p r o x i m a t e m o v i n g least squares a p p r o x i m a t i o n s ( q u a s i - i n t e r p o l a n t s ) are
t a k e n f r o m t h e p a p e r [Fasshauer a n d Z h a n g ( 2 0 0 4 ) ] . W e deviate from our usual

p o l i c y o f p r o v i d i n g o n l y r e s u l t s o f e x p e r i m e n t s based o n M A T L A B code a n d i n c l u d e
F i g u r e s 2 8 . 1 - 2 8 . 3 , w h i c h w e r e o b t a i n e d w i t h C++ s o f t w a r e i n c o r p o r a t i n g t h e N F F T
library.
s
W e use t h e f o l l o w i n g m o l l i f i e d F r a n k e - t y p e f u n c t i o n i n t h e u n i t c u b e [0, l ] in
space d i m e n s i o n s s = 1, 2 , 3 :
3 (9xi - 2 ) 2
(9x - 2)
2
2
(9x - 2) 3
5
/(xi,X ,X )2 3 exp(- )
4 4 4
2 2
(9xi + l ) (9x + l )
2 (9x 3 + If
+ exp(-
49 10 29
(9xx - 7 ) s
s
(9x 3 - 5) 1
+ 2 e x
P(- (9x 2 - 3)
1 , 2 2
6 X P (
(9xi - 4 ) - (9x - 7 ) - 2 (9x 3
2
5) ),
- 5 -
# ( x i , x , x ) = 1 5 / ( x i , x , x ) JJexp(:j-
2 3 2 3
2
(2xi-l)
w h e r e x i , X2, X3 are used a c c o r d i n g t o t h e space d i m e n s i o n s. T h e d a t a sites are

k s
N = (2 + l ) e q u a l l y spaced p o i n t s i n t h e u n i t c u b e , w h i l e t h e e r r o r s are c o m p u t e d
s
a t M e v a l u a t i o n p o i n t s r a n d o m l y d i s t r i b u t e d i n [0, l ] w i t h M = 32768 for s = 1,
M = 262144 for s = 2, a n d M = 2146689 for s = 3.
1
I n o u r e x p e r i m e n t s we use n = AN ^ i n (28.3) for a l l c o m p u t a t i o n s e x c e p t for
t h e v e r y last e x p e r i m e n t s i n 2 D a n d 3 D . W e d o n o t have a n a u t o m a t e d s t r a t e g y for
c h o o s i n g n. H o w e v e r , t h e values j u s t m e n t i o n e d y i e l d s a t i s f a c t o r y r e s u l t s a n d go
a l o n g w i t h t h e values suggested b y T h e o r e m s 3.1 a n d 3.4 o f [ N i e s l o n y et al. ( 2 0 0 4 ) ] .
I n a l l e x p e r i m e n t s d i s p l a y e d i n F i g u r e s 2 8 . 1 - 2 8 . 3 , t h e scale p a r a m e t e r T> is t a k e n t o
be 3.0.
T h e left p l o t s i n F i g u r e s 2 8 . 1 - 2 8 . 3 s h o w t h e m a x i m u m e r r o r versus t h e n u m b e r o f
centers A ^ o n a l o g a r i t h m i c scale for t h e t h r e e t y p e s o f L a g u e r r e - G a u s s i a n g e n e r a t i n g
f u n c t i o n s o f T a b l e 4 . 1 . T h i s i l l u s t r a t e s t h a t t h e a p p r o x i m a t i o n does converge w e l l
( a l m o s t r e a c h i n g t h e rates p r e d i c t e d b y t h e t h e o r y ) as w e increase t h e n u m b e r o f
d a t a sites.
T h e presence o f t h e s a t u r a t i o n e r r o r is c l e a r l y v i s i b l e i n F i g u r e 2 8 . 1 . T h e p l o t s o n
t h e r i g h t c o m p a r e t h e cost o f d i r e c t s u m m a t i o n versus t h a t for N F F T s u m m a t i o n ,
a n d s h o w t h a t t h e efficiency is g r e a t l y i m p r o v e d b y t h e use o f t h e N F F T . D u e t o
t h e i r l o n g d u r a t i o n some o f t h e c o m p u t a t i o n a l t i m e s f o r t h e d i r e c t s u m m a t i o n w e r e
omitted.
T h e e x p e r i m e n t s p r e s e n t e d i n t h i s s e c t i o n s h o w t h a t i t is n o t u n r e a s o n a b l e to
a p p r o x i m a t e large d a t a sets w i t h g l o b a l l y s u p p o r t e d g e n e r a t i n g f u n c t i o n s . I n f a c t ,
t h e a c c u r a c y a c h i e v e d w i t h t h e g l o b a l f u n c t i o n s is far s u p e r i o r t o t h a t w h i c h w e o b -
t a i n e d w i t h t h e c o m p a c t l y s u p p o r t e d f u n c t i o n s for s i m i l a r p r o b l e m s i n t h e p r e v i o u s
chapter.
28. Fast Fourier Transforms 247
10 u d=0
d=1
d=2
g 10
LU
10
3 9 33 257 4097 65537 1048577 3 9 33 257 4097 65537 1048577
Fig. 28.1 Convergence and execution times for I D example (Gaussian, linear and quadratic
Laguerre-Gaussian generating functions).
10'
10
10'
y
/ . d=0, direct
d=1, direct '
d=2, direct
d=0, NFFT
d=1, NFFT
d=2, NFFT
10'
263169 81 1089 16641 263169
N
Fig. 28.2 Convergence and execution times for 2D example (Gaussian, linear and quadratic
Laguerre-Gaussian generating functions).
10 10
10
10'
g 10" 2 01
LU i-
d=0, direct
10 d=1, direct '
10 d=2, direct
d=0, NFFT
d=1, NFFT
d=2, NFFT
10 10
27 125 729 4913 35937 274625 27 125 729 4913 35937 274625
N N
Fig. 28.3 Convergence and (predicted) execution times for 3D example (Gaussian, linear and
quadratic Laguerre-Gaussian generating functions).
Chapter 29
Partition of Unity Methods
A n o t h e r p o s s i b i l i t y for fast c o m p u t a t i o n w i t h meshfree a p p r o x i m a t i o n m e t h o d s is

t h e partition of unity m e t h o d . T h i s a p p r o a c h offers a s i m p l e w a y t o d e c o m p o s e a
large p r o b l e m i n t o m a n y s m a l l problems w h i l e at the same t i m e ensuring t h a t t h e
a c c u r a c y o b t a i n e d for t h e l o c a l f i t s is c a r r i e d over t o t h e g l o b a l f i t .
29.1 Theory
T h e p a r t i t i o n o f u n i t y m e t h o d was suggested i n [ B a b u s k a a n d M e l e n k ( 1 9 9 7 ) ;
M e l e n k a n d B a b u s k a (1996)] i n t h e m i d 1990s i n t h e c o n t e x t o f meshfree G a l e r k i n
m e t h o d s for t h e s o l u t i o n o f p a r t i a l d i f f e r e n t i a l e q u a t i o n s (see C h a p t e r s 4 4 a n d 45
for a discussion o f a n R B F - b a s e d G a l e r k i n a p p r o a c h ) . I n t h e s c a t t e r e d d a t a f i t t i n g
c o n t e x t t h e p a p e r [ F r a n k e (1977)] a l r e a d y c o n t a i n s a s i m i l a r a l g o r i t h m . W e base
the presentation i n this section o n the paper [ W e n d l a n d (2002a)].
T h e basic i d e a for t h e p a r t i t i o n o f u n i t y m e t h o d is t o s t a r t w i t h a p a r t i t i o n o f t h e
s
open and bounded domain f l C R i n t o M s u b d o m a i n s flj such t h a t ( J j l i 2 fl
w i t h some m i l d o v e r l a p a m o n g t h e s u b d o m a i n s . A s s o c i a t e d w i t h these s u b d o m a i n s
we choose a partition of unity, i.e., a f a m i l y o f c o m p a c t l y s u p p o r t e d , n o n - n e g a t i v e ,
c o n t i n u o u s f u n c t i o n s Wj s u p p o r t e d o n t h e closure o f flj s u c h t h a t a t e v e r y p o i n t x
i n fl we have
M
X>,-(aO = l . (29.1)
3= 1
N o w , for every s u b d o m a i n flj we c o n s t r u c t a l o c a l a p p r o x i m a t i o n Uj {e.g., a radial

basis f u n c t i o n i n t e r p o l a n t ) , a n d t h e n f o r m t h e g l o b a l a p p r o x i m a n t t o t h e d a t a o n
t h e e n t i r e d o m a i n fl v i a
M
V {x)
f = ^2u (x)w (x),
j j xefl. (29.2)
3= 1
N o t e t h a t i f t h e l o c a l fits i n t e r p o l a t e a t a g i v e n d a t a p o i n t xe, i.e., Uj{xe) = f(xi),
249
t h e n t h e g l o b a l a p p r o x i m a n t also i n t e r p o l a t e s a t t h i s p o i n t :
M
Pf(xe) = Yû {xi)w {x )
j j e
T h e last e q u a l i t y h o l d s d u e t o t h e p a r t i t i o n o f u n i t y p r o p e r t y ( 2 9 . 1 ) . T e c h n i c a l l y ,
t h e second s u m w i l l m o s t l i k e l y n o t r u n over t h e f u l l set o f indices, 1 , . . . , M , since
X w i l l lie o n l y i n t h e s u p p o r t o f some o f t h e Wj. O f course, t h i s does n o t change
the result.
I n o r d e r t o be able t o f o r m u l a t e e r r o r b o u n d s w e n e e d some t e c h n i c a l c o n d i t i o n s .
W e r e q u i r e t h e p a r t i t i o n o f u n i t y f u n c t i o n s t o be k-stable, i.e., we r e q u i r e t h a t each
k s
Wj G C (M. ) satisfies for e v e r y m u l t i - i n d e x ct w i t h | a | < k t h e i n e q u a l i t y
where C a is some p o s i t i v e c o n s t a n t , a n d Sj = d i a m ( O ^ ) .
I n order t o understand the following a p p r o x i m a t i o n theorem f r o m [ W e n d l a n d
s k
(2002a)] we need t o define t h e space C g ( R ) o f a l l C functions / whose derivatives
a
o f o r d e r | a | = A; satisfy D f(x) = 0(\\x\\%) for | | * | | 2 - 0.
s
T h e o r e m 2 9 . 1 . Suppose fl C R is open and bounded, and let X = {x\,... ,x^} C
s
f2. Let <3> G Cp(R ) be strictly conditionally positive definite of order m. Let
{flj} be a regular covering for (fl, X) and let {wj} be k-stable for {flj}- Then
the error between f G Af$(fi) and its partition of unity interpolant (29.2) with
Uj G span{<E>(-, cc) : x G X D flj} + II^ _ l 1 can be bounded by
a a
\D f{x) - D V {x)\ f < C h % - ^ \ f W , m ,
for all x G fl and all \a\ < k/2.
T h e r e g u l a r i t y a s s u m p t i o n s o n t h e s u b d o m a i n s flj are:
For every x G fl t h e n u m b e r o f s u b d o m a i n s flj w i t h x G flj is b o u n d e d b y a

global constant K.
E v e r y s u b d o m a i n flj satisfies a n i n t e r i o r cone c o n d i t i o n (c.f. D e f i n i t i o n 14.2).
T h e l o c a l f i l l distances hx^cij are u n i f o r m l y b o u n d e d b y t h e g l o b a l f i l l d i s t a n c e
hx,o.i w h e r e Xj = X f l flj.
I f we c o m p a r e t h i s w i t h t h e g l o b a l e r r o r e s t i m a t e s f r o m C h a p t e r 15 w e see t h a t
t h e p a r t i t i o n o f u n i t y preserves t h e l o c a l a p p r o x i m a t i o n o r d e r for t h e g l o b a l f i t .
T h u s , we c a n efficiently c o m p u t e l a r g e R B F i n t e r p o l a n t s b y s o l v i n g m a n y s m a l l
RBF i n t e r p o l a t i o n p r o b l e m s ( i n p a r a l l e l i f w e w i s h ) a n d t h e n glue t h e m together
w i t h the global p a r t i t i o n of u n i t y {wj}.
A s i m p l e w a y t o o b t a i n a p a r t i t i o n o f u n i t y is v i a a S h e p a r d a p p r o x i m a n t (c.f.
C h a p t e r 2 3 ) . T h e r e f o r e , w e c a n t h i n k o f t h e p a r t i t i o n o f u n i t y m e t h o d as a S h e p a r d
m e t h o d w i t h higher-order data. Namely, the " d a t a " are n o w g i v e n b y t h e l o c a l

a p p r o x i m a t i o n s uj i n s t e a d o f j u s t t h e values f(xj). T h e benefits o f t h i s k i n d o f
a p p r o a c h seem t o have first b e e n r e a l i z e d i n [ F r a n k e ( 1 9 7 7 ) ] .
29.2 P a r t i t i o n of U n i t y A p p r o x i m a t i o n w i t h MATLAB
T h e M A T L A B p r o g r a m for t h e p a r t i t i o n o f u n i t y a p p r o x i m a t i o n based o n l o c a l R B F
i n t e r p o l a n t s is a g a i n r a t h e r s i m i l a r t o o u r earlier p r o g r a m s . T h e m a i n difference is
t h a t we n o w also have t o create t h e s u b d o m a i n s flj ( w h i c h we w i l l d o as o v e r l a p p i n g
circles) a n d t h e associated p a r t i t i o n o f u n i t y {vjj} (for w h i c h w e use a S h e p a r d
m e t h o d based o n W e n d l a n d ' s c o m p a c t l y s u p p o r t e d R B F s ) .
T h e c o m p a c t l y s u p p o r t e d r a d i a l w e i g h t f u n c t i o n s o n l i n e 1 o f P r o g r a m 29.1 a n d
t h e R B F o n l i n e 2 are used i n t h e s h i f t e d f o r m > fc = (p ,k(lSi s ) (cf. T a b l e 1 1 . 1 ) .
N o t e t h a t we use t h e kd-tree r o u t i n e s t o b u i l d t w o trees: a d a t a tree, and an
evaluation tree. I n s i d e t h e l o o p over a l l p a r t i t i o n o f u n i t y cells (lines 2 7 - 3 8 ) we
first use k d r a n g e q u e r y t o find a l l d a t a sites i n cell j , b u i l d t h e l o c a l i n t e r p o l a t i o n
m a t r i x based o n these p o i n t s , a n d t h e n r e p e a t t h e process for t h e e v a l u a t i o n p o i n t s .
N o t e t h a t t h e c o n t r i b u t i o n s t o t h e final g l o b a l fit are a c c u m u l a t e d cell b y cell (see
line 3 6 ) .
P r o g r a m 2 9 . 1 . PU2D_CS.m
/ PU2D_CS
% Script that performs p a r t i t i o n of u n i t y approximation using
/ s p a r s e
0 matrices
% Calls on: DistanceMatrixCSRBF
/ U s e s :
0 k - D t r e e p a c k a g e b y Guy S h e c h t e r
% f r o m MATLAB C e n t r a l File Exchange
/, W e i g h t f u n c t i o n f o r g l o b a l S h e p a r d p a r t i t i o n o f u n i t y w e i g h t i n g
1 wf = @ ( e , r ) r."4.*(5*spones(r)-4*r);
% RBF b a s i s f u n c t i o n f o r l o c a l RBF i n t e r p o l a t i o n
2 r b f = @(e,r) r."4.*(5*spones(r)-4*r);
3 ep = 0 . 1 ; % P a r a m e t e r f o r local basis functions
% D e f i n e F r a n k e ' s f u n c t i o n as testfunction
4 f l = @(x,y) 0.75*exp(-((9*x-2),"2+(9*y-2)."2)/4);
5 f 2 = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x + l ) . " 2 / 4 9 + ( 9 * y + l ) . " 2 / 1 0 ) ) ;
6 f3 = @(x,y) 0.5*exp(-((9*x-7)."2+(9*y-3)."2)/4);
7 f4 = @(x,y) 0.2*exp(-((9*x-4),"2+(9*y-7).~2));
8 testfunction = @(x,y) fl(x,y)+f2(x,y)+f3(x,y)-f4(x,y);
9 N = 1089; gridtype = 'h';
/ P a r a m e t e r f o r n p u - b y - n p u g r i d
0 o f PU c e l l s in unit square
10 npu = 16;
% Parameter f o r n e v a l - b y - n e v a l evaluation grid in unit square
11 n e v a l = 40;
5
12 name = s p r i n t f (Data2D_7 d/,s' ,N,gridtype) ; load(name)
0
13 ctrs = dsites;
15 wep = npu; % Parameter f o r weight f u n c t i o n
% C r e a t e n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
% i n t h e u n i t square
17 epoints = [xe(:) y e ( : ) ] ;
% Create npu-by-npu e q u a l l y spaced c e n t e r s of PU c e l l s i n t h e
'/, u n i t square
18 p u g r i d = l i n s p a c e ( 0 , 1 , n p u ) ; [xpu,ypu] = m e s h g r i d ( p u g r i d ) ;
19 c e l l c t r s = [xpu(:) y p u ( : ) ] ;
20 c e l l r a d i u s = 1/wep;
% Compute Shepard e v a l u a t i o n m a t r i x
21 DM_eval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , c e l l c t r s , w e p ) ;
22 SEM = wf(ep,DM_eval);
23 SEM = s p d i a g s d ./(SEM*ones(npu~2,1)) ,0,neval~2,neval~2)*SEM;
% B u i l d k-D t r e e s f o r d a t a s i t e s and e v a l u a t i o n p o i n t s
24 [tmp,tmp,datatree] = k d t r e e ( d s i t e s , [ ] ) ;
25 [tmp,tmp,evaltree] = k d t r e e ( e p o i n t s , [ ] ) ;
26 Pf = z e r o s ( n e v a l ~ 2 , 1 ) ; % i n i t i a l i z e
27 f o r j=l:npu"2
% Find data s i t e s i n c e l l j
28a [ p t s , d i s t , i d x ] = kdrangequery(datatree,...
28b cellctrs(j,:),cellradius);
29 i f ( l e n g t h ( i d x ) > 0)
/ B u i l d l o c a l i n t e r p o l a t i o n m a t r i x f o r c e l l j
0
30a DM_data = D i s t a n c e M a t r i x C S R B F ( d s i t e s ( i d x , : ) , . . .
30b ctrs(idx,:),ep);
% Find evaluation points i n c e l l j
32a [epts,edist,eidx] = kdrangequery(evaltree,...
% Compute l o c a l e v a l u a t i o n m a t r i x
33a DM_eval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s ( e i d x , : ) , . . .
% Compute l o c a l RBF i n t e r p o l a n t
35 l o c a l f i t = EM * ( I M \ r h s ( i d x ) ) ;
Accumulate g l o b a l f i t
36 Pf(eidx) = Pf(eidx) + localfit.*SEM(eidx,j);

37 end
38 end
/ Compute e x a c t
0 solution
39 exact = testfunction(epoints(:,1),epoints(:,2));
42 f p r i n t f ('RMS error: 7.e\n , r m s . e r r )
3
43 f p r i n t f ('Maximum e r r o r : /e\n', maxerr) 0
7, P l o t interpolant
44 fview = [160,20];
7o P l o t maximum e r r o r
I n Tables 29.1 a n d 29.2 we i l l u s t r a t e h o w t h e l o c a l convergence o r d e r is m a i n -

2
t a i n e d g l o b a l l y for a p a r t i t i o n o f u n i t y based o n W e n d l a n d ' s C compactly supported
R B F s . I n b o t h tables t h e l o c a l a p p r o x i m a t i o n s are c o m p u t e d v i a e i t h e r t h e c o m -
2
pactly supported C f u n c t i o n s o f W e n d l a n d , or v i a Gaussians ( t h a t are g l o b a l l y
supported on the local subdomains). T h e d a t a were s a m p l e d f r o m F r a n k e ' s func-
t i o n a t v a r i o u s sets o f u n i f o r m l y spaced p o i n t s ( i n T a b l e 29.1) a n d H a l t o n p o i n t s
( i n T a b l e 29.2) i n t h e u n i t square. T h e M l o c a l s u b d o m a i n s were g i v e n b y circles
centered a t e q u a l l y spaced p o i n t s i n t h e u n i t square. W e c a n see t h a t (especially
o n t h e u n i f o r m l y spaced d a t a sites) t h e p a r t i t i o n o f u n i t y m e t h o d reflects t h e ap-
p r o x i m a t i o n behavior of the local methods. For the c o m p a c t l y supported W e n d l a n d
2
f u n c t i o n s we o b t a i n 0{h ) t h r o u g h o u t o u r series o f e x p e r i m e n t s ( w i t h a t h e o r e t i c a l l y
3 2
p r e d i c t e d l o c a l o r d e r o f 0(h / )), w h e r e a s for t h e Gaussians w e o b t a i n a n a p p r o x i -
m a t i o n b e h a v i o r i n places v a g u e l y suggestive o f e x p o n e n t i a l convergence. F o r these
e x p e r i m e n t s t h e f i l l distances for t h e sets o f H a l t o n p o i n t s were n o t e s t i m a t e d v i a
(2.4). I n s t e a d , we assumed t h a t t h e fill d i s t a n c e decreases b y a f a c t o r o f t w o f r o m
one i t e r a t i o n t o t h e n e x t , as i t does in" t h e case o f u n i f o r m l y d i s t r i b u t e d p o i n t s .
A r e l a t i v e l y large u n i f o r m v a l u e o f t h e shape p a r a m e t e r e was used for t h e Gaus-
sians o n t h e u n i f o r m d a t a sets i n T a b l e 29.1 t o o b t a i n t h e e x p o n e n t i a l convergence
results. Use o f t h e same v a l u e o f e o n t h e H a l t o n d a t a sets results i n R M S - e r r o r s for
t h e Gaussians t h a t are worse t h a n those for l o c a l i n t e r p o l a n t s based o n c o m p a c t l y
s u p p o r t e d R B F s (see T a b l e 29.2). F o r t h e l o c a l i n t e r p o l a n t s based o n t h e W e n d -
l a n d f u n c t i o n s we used a large s u p p o r t r a d i u s o f p = 1/e = 10 i n accordance t o t h e
observations m a d e i n C h a p t e r 17. T h e m a i n reason for t h e r e l a t i v e l y p o o r q u a l -
i t y o f t h e l o c a l i n t e r p o l a n t s based o n Gaussians i n t h e H a l t o n s e t t i n g is t h a t a l l o f
t h e c o m p u t a t i o n s w i t h t h e W e n d l a n d f u n c t i o n s are p e r f o r m e d w i t h w e l l - c o n d i t i o n e d
i n t e r p o l a t i o n m a t r i c e s ( i n s p i t e o f t h e large s u p p o r t r a d i u s , i.e., flat basis f u n c t i o n s ) .
N o t e t h a t t h e R M S - e r r o r for l o c a l G a u s s i a n i n t e r p o l a t i o n i n t h e last r o w o f
Table 29.1 2D partition of unity approximation on uniform points with Wend-

2
land's C compactly supported functions for partition of unity and local Wend-
land and Gaussian interpolants.
Wendland Gaussian
N M RMS-error rate RMS-error rate
9 1 3.475816e-001 0.1 3.703960e-001 6

25 4 1.301854e-001 1.4168 0.1 1.229046e-001 1.5915 6
81 16 8.089165e-003 4.0084 0.1 2.413852e-002 2.3481 6
289 64 1.183369e-003 2.7731 0.1 4.551325e-003 2.4070 6
1089 256 2.716542e-004 2.1231 0.1 5.393771e-004 3.0769 6
4225 1024 6.795949e-005 1.9990 0.1 1.112507e-005 5.5994 6
16641 4096 1.697195e-005 2.0015 0.1 2.031757e-006 2.4530 6
66049 16384 4.241184e-006 2.0006 0.1 1.798274e-007 3.4980 6
263169 65536 9.778231e-007 2.1168 0.1 2.657751e-008 2.7583 6
1050625 262144 2.557991e-007 1.9346 0.1 1.844820e-009 3.8487 6
Table 29.2 2D partition of unity approximation on Halton points with Wend-

2
land's C compactly supported functions for partition of unity and local Wend-
land and Gaussian interpolants.
Wendland Gaussian
N M RMS-error rate RMS-error rate
9 1 3.325975e-001 0.1 3.384117e-001 6

25 4 1.355396e-001 1.2951 0.1 1.383321e-001 1.2906 6
81 16 1.035963e-002 3.7097 0.1 3.640953e-002 1.9257 6
289 64 2.569458e-003 2.0114 0.1 1.030984e-002 1.8203 6
1089 256 5.860966e-004 2.1323 0.1 3.543463e-003 1.5408 6
4225 1024 2.703318e-004 1.1164 0.1 1.045103e-003 1.7615 6
16641 4096 7.701234e-005 1.8116 0.1 3.896345e-004 1.4235 6
66049 16384 4.492321e-005 0.7776 0.1 5.012220e-005 2.9586 6
263169 65536 1.589134e-005 1.4992 0.1 1.631609e-005 1.6192 6
1050625 262144 1.032629e-006 3.9438 0.1 2.000238e-006 3.0281 6
T a b l e 29.1 presents the best a p p r o x i m a t i o n o f F r a n k e ' s test f u n c t i o n r e p o r t e d i n t h i s

b o o k . H o w e v e r , i n T a b l e 17.5 w e n e e d e d o n l y N = 4225 u n i f o r m l y s p a c e d p o i n t s for
a g l o b a l G a u s s i a n i n t e r p o l a n t w i t h e = 6.3 t o a c h i e v e a n R M S - e r r o r o f 7.371879e-
009. O f c o u r s e , i t m a y be p o s s i b l e t o o b t a i n e v e n b e t t e r a p p r o x i m a t i o n s w i t h o t h e r
RBFs. W e d o n o t c l a i m t h a t G a u s s i a n s are the "best" R B F s . However, we do
r e c o m m e n d t h e p a r t i t i o n o f u n i t y a p p r o a c h for t h e s o l u t i o n o f large i n t e r p o l a t i o n or
a p p r o x i m a t i o n p r o b l e m s since i t is r e l a t i v e l y s i m p l e t o i m p l e m e n t a n d i t s e x e c u t i o n
is q u i t e efficient.
Chapter 30
Approximation of Point Cloud Data in 3D
30.1 A General A p p r o a c h v i a Implicit Surfaces
A c o m m o n p r o b l e m i n c o m p u t e r g r a p h i c s a n d c o m p u t e r a i d e d design ( C A D ) is t h e
r e c o n s t r u c t i o n o f a t h r e e - d i m e n s i o n a l surface defined i n t e r m s o f point cloud data,
i.e., as a set o f u n o r g a n i z e d , i r r e g u l a r p o i n t s i n 3 D . F o r e x a m p l e , t h i s c o u l d be
laser range d a t a o b t a i n e d for t h e p u r p o s e o f c o m p u t e r m o d e l i n g o f a c o m p l i c a t e d
3D object. Such a p p l i c a t i o n s also arise, e.g., i n c o m p u t e r g r a p h i c s o r i n m e d i c a l
i m a g i n g . A n a p p r o a c h t o o b t a i n i n g a surface t h a t fits t h e g i v e n 3 D p o i n t c l o u d d a t a
t h a t has r e c e n t l y become r a t h e r p o p u l a r (see, e.g., [ C a r r et al. (1997); C a r r et al.
(2001); M o r s e et al. (2001); O h t a k e et al. (2003a); O h t a k e et al. ( 2 0 0 3 b ) ; T u r k a n d
O ' B r i e n (2002); W e n d l a n d ( 2 0 0 2 b ) ] ) is based o n t h e use o f implicit surfaces defined
i n t e r m s o f some meshfree a p p r o x i m a t i o n m e t h o d such as a n R B F i n t e r p o l a n t o r
an M L S approximant.
M o r e precisely, g i v e n d a t a o f t h e f o r m {xi = (xi,yi,Zi) 3
G M , i = 1,...,N}
3
assumed t o come f r o m some t w o - d i m e n s i o n a l m a n i f o l d Ai (i.e., a surface i n M ) , w e
seek a n o t h e r surface Ai* t h a t is a reasonable a p p r o x i m a t i o n t o Ai. For the i m p l i c i t
surface a p p r o a c h we t h i n k o f Ai as t h e surface o f a l l p o i n t s (x, y, z) t h a t satisfy t h e
implicit equation
f(x,y,z) = 0
for some f u n c t i o n / . Thus, the function / i m p l i c i t l y defines t h e surface Ai. In

o t h e r w o r d s , t h e e q u a t i o n f(x, y, z) = 0 defines t h e zero iso-surface o f t h e t r i v a r i a t e
f u n c t i o n / a n d therefore t h i s iso-surface coincides w i t h Ai.
A s so often before, we w i l l c o n s t r u c t t h e surface Ai * v i a i n t e r p o l a t i o n . O b v i -
ously, i f we o n l y specify t h e i n t e r p o l a n t t o be zero at t h e d a t a p o i n t s , t h e n we w i l l
o b t a i n a zero i n t e r p o l a n t , a n d w i l l n o t b e able t o e x t r a c t a m e a n i n g f u l iso-surface.
Therefore, the key t o finding an approximation to the trivariate function / from
t h e g i v e n d a t a p o i n t s x^, i = 1 , . . . , N, is t o a d d a n e x t r a set o f off-surface points to
t h e d a t a so t h a t we c a n t h e n c o m p u t e a ( s o l i d ) t h r e e - d i m e n s i o n a l i n t e r p o l a n t Vf to
t h e t o t a l set o f p o i n t s , i.e., t h e surface p o i n t s p l u s t h e a u x i l i a r y off-surface p o i n t s .
T h i s w i l l result i n a n o n t r i v i a l i n t e r p o l a n t , a n d we w i l l t h e n be able t o e x t r a c t i t s
255
zero iso-surface. T o i l l u s t r a t e t h i s t e c h n i q u e w e discuss h o w t h i s i d e a w o r k s i n t h e

2 D setting i n the next section.
T h e a d d i t i o n o f off-surface p o i n t s r e s u l t s i n a p r o b l e m o f t h e same t y p e as t h e
s c a t t e r e d d a t a a p p r o x i m a t i o n p r o b l e m s discussed i n e a r l i e r c h a p t e r s . I n p a r t i c u l a r ,
i f t h e d a t a sets are large, i t is a d v i s a b l e t o use e i t h e r a l o c a l r a d i a l basis i n t e r p o l a n t ,
a m o v i n g - l e a s t squares a p p r o a c h , o r a p a r t i t i o n o f u n i t y i n t e r p o l a n t . H o w e v e r , t h e r e
are also i m p l i c i t p o i n t c l o u d i n t e r p o l a n t s based o n fast e v a l u a t i o n a l g o r i t h m s w i t h
g l o b a l R B F s (see, e.g., [ C a r r et al. ( 2 0 0 1 ) ] ) .
T h e surface r e c o n s t r u c t i o n p r o b l e m consists o f t h r e e sub-problems:
(1) C o n s t r u c t t h e e x t r a off-surface p o i n t s .
(2) F i n d t h e t r i v a r i a t e meshfree a p p r o x i m a n t t o t h e a u g m e n t e d d a t a set.
(3) R e n d e r t h e iso-surface ( z e r o - c o n t o u r ) o f t h e f i t c o m p u t e d i n step ( 2 ) .
I n o r d e r t o keep t h e d i s c u s s i o n as s i m p l e as possible w e assume t h a t , i n a d d i t i o n

t o t h e p o i n t c l o u d d a t a , we are also g i v e n a set o f surface n o r m a l s rn = ( n f , rv(, n f )
t o t h e surface Ai a t t h e p o i n t s ccj = (xi,yi,Zi). I f these n o r m a l s are n o t e x p l i c i t l y
g i v e n , t h e r e are t e c h n i q u e s a v a i l a b l e t h a t c a n be used t o e s t i m a t e t h e n o r m a l s (see,
e.g., t h e discussion i n [ W e n d l a n d ( 2 0 0 2 b ) ] ) . Once we have the (oriented) surface
n o r m a l s , we c o n s t r u c t t h e e x t r a off-surface p o i n t s b y m a r c h i n g a s m a l l d i s t a n c e
a l o n g t h e surface n o r m a l , i.e., w e o b t a i n for e a c h d a t a p o i n t ( x j , yi, Zi) t w o a d d i t i o n a l
off-surface p o i n t s . O n e p o i n t lies " o u t s i d e " t h e m a n i f o l d Ai a n d is g i v e n b y
(x N + i , y N + i , z N +i) = Xi + drii = (x^ + 5 n f , y + Su?, t y i + bn\),
a n d t h e o t h e r p o i n t lies " i n s i d e " Ai a n d is g i v e n b y
(x N+i,U2N+i,
2 z N+i)
2 =Xi Srii = (xi - Sn^,yi - 5n\,yi - 5n*).
H e r e 5 is a s m a l l s t e p size (whose specific m a g n i t u d e c a n be r a t h e r c r i t i c a l for a

g o o d surface f i t , see [ C a r r et al. ( 2 0 0 1 ) ] ) . I n p a r t i c u l a r , i f S is chosen t o o l a r g e ,
t h e n t h i s c a n easily r e s u l t i n s e l f - i n t e r s e c t i n g i n n e r or o u t e r a u x i l i a r y surfaces. In
our simple M A T L A B i m p l e m e n t a t i o n i n the next section we u n i f o r m l y t a k e S t o be
1 % o f t h e m a x i m u m d i m e n s i o n o f t h e b o u n d i n g b o x o f t h e d a t a as suggested i n
[Wendland (2002b)].
O n c e w e have c r e a t e d t h e a u x i l i a r y d a t a , t h e i n t e r p o l a n t is c o m p u t e d b y d e t e r -
m i n i n g a f u n c t i o n Vf w h o s e zero c o n t o u r i n t e r p o l a t e s t h e g i v e n p o i n t c l o u d d a t a ,
a n d w h o s e " i n n e r " a n d " o u t e r " offset c o n t o u r s i n t e r p o l a t e t h e a u g m e n t e d d a t a , i.e.,
V (xi)
f = 0, i = l,...,N,
V (xi)
f = l,. i = N-rl,...,2N,
V (xi)
f = -1, i = 2N+l,...,3N.
T h e values o f 1 for t h e a u x i l i a r y d a t a are a r b i t r a r y . T h e i r precise v a l u e is n o t as

c r i t i c a l as t h e choice o f 5.
For t h e t h i r d s t e p we also use a v e r y s i m p l e s o l u t i o n , n a m e l y w e j u s t r e n d e r t h e
r e s u l t i n g a p p r o x i m a t i n g surface Ai* as t h e zero c o n t o u r o f t h e 3 D i n t e r p o l a n t . In
30. Approximation of Point Cloud Data in 3D 257
M A T L A B t h i s c a n be a c c o m p l i s h e d w i t h t h e c o m m a n d i s o s u r f a c e (or c o n t o u r for
2 D p r o b l e m s ) . T h i s p r o v i d e s a r o u g h p i c t u r e o f t h e i m p l i c i t surface i n t e r p o l a n t , b u t
m a y also lead t o some r e n d e r i n g a r t i f a c t s t h a t c a n be a v o i d e d w i t h m o r e sophis-
ticated rendering procedures. F o r m o r e serious a p p l i c a t i o n s one u s u a l l y e m p l o y s
some v a r i a t i o n o f a ray tracing or marching cube a l g o r i t h m (see t h e discussion i n
t h e references l i s t e d above).
T h e i m p l i c i t surface r e p r e s e n t a t i o n has t h e a d v a n t a g e t h a t t h e surface n o r m a l s
of t h e a p p r o x i m a t i n g surface Ai* c a n be e x p l i c i t l y a n d a n a l y t i c a l l y c a l c u l a t e d as
t h e gradients o f t h e t r i v a r i a t e f u n c t i o n Vf, i.e., n(x) = Wf(x).
Since i n p r a c t i c e t h e d a t a (e.g., o b t a i n e d b y laser r a n g e scanners) is s u b j e c t
t o m e a s u r e m e n t errors i t is o f t e n beneficial i f a n a d d i t i o n a l s m o o t h i n g p r o c e d u r e is
e m p l o y e d . O n e c a n e i t h e r use t h e r i d g e regression a p p r o a c h suggested i n C h a p t e r 19,
or use a m o v i n g least squares a p p r o x i m a t i o n i n s t e a d o f a n R B F i n t e r p o l a n t . A n o t h e r
i m p l i c i t s m o o t h i n g t e c h n i q u e was suggested i n [ B e a t s o n a n d B u i (2003)]. N o i s y d a t a
c a n also be d e a l t w i t h b y u s i n g a m u l t i l e v e l t e c h n i q u e such as suggested i n [ O h t a k e
et al. ( 2 0 0 3 b ) ] . W e discuss m u l t i l e v e l i n t e r p o l a t i o n a n d a p p r o x i m a t i o n a l g o r i t h m s
i n C h a p t e r 32.
30.2 A n Illustration in 2 D
Since t h e 3 D p o i n t c l o u d i n t e r p o l a t i o n p r o b l e m requires a n i n t e r p o l a n t t o p o i n t s
v i e w e d as samples o f a t r i v a r i a t e f u n c t i o n w h o s e g r a p h is a 4 D h y p e r s u r f a c e , the
v i s u a l i z a t i o n o f t h e i n d i v i d u a l steps o f t h e c o n s t r u c t i o n o f t h e f i n a l iso-surface is
p r o b l e m a t i c . W e therefore i l l u s t r a t e t h e process w i t h a n a n a l o g o u s t w o - d i m e n s i o n a l
p r o b l e m , i.e., we assume we are g i v e n p o i n t s ( t a k e n f r o m a closed c u r v e C) i n t h e
plane, a n d i t is o u r g o a l t o f i n d a n i n t e r p o l a t i n g c u r v e C*. B e l o w we present b o t h
M A T L A B code a n d several figures.
P r o g r a m 30.1. PointCloud2D.m
% PointCloud2D
% Script that fits a curve t o 2D p o i n t cloud
% C a l l s on: DistanceMatrix
% Uses: haltonseq ( w r i t t e n by D a n i e l Dougherty
f r o m MATLAB C e n t r a l F i l e Exchange)
% G a u s s i a n RBF
1 r b f = @(e,r) exp(-(e*r).~2); ep = 3.5;
2 N = 81; % number of data points
3 neval = 40; % to create neval-by-neval evaluation grid
4 t = 2*pi*haltonseq(N 1); } dsites = [cos(t) sin(t)];
5 x = (2+sin(t)).*cos(t); y = (2+cos(t)).*sin(t);
6 nx = (2+cos(t)).*cos(t)-sin(t)."2;
7 ny = ( 2 + s i n ( t ) ) . * s i n ( t ) - c o s ( t ) . " 2 ;
8 d s i t e s = [x y ] ; normals = [nx n y ] ;
/ Produce a u x i l i a r y p o i n t s along normals " i n s i d e " and " o u t s i d e "
0
9 bmin = m i n ( d s i t e s , [] , 1) ; bmax = m a x ( d s i t e s , [] , 1) ;
10 bdim = max(bmax-bmin);
% D i s t a n c e along normal a t which t o p l a c e new p o i n t s
11 d e l t a = bdim/100;
% Create new p o i n t s
12 dsites(N+l:2*N, :) = d s i t e s d :N, : ) + d e l t a * n o r m a l s ;
13 dsites(2*N+1:3*N,:) = d s i t e s ( 1 : N ) - d e l t a * n o r m a l s ;
% " o r i g i n a l " p o i n t s have rhs=0,
% " i n s i d e " p o i n t s have r h s = - l , " o u t s i d e " p o i n t s have r h s = l
14 rhs = [zeros(N,l); ones(N,l); -ones(N,l)];
% L e t centers coincide with data s i t e s
15 ctrs = dsites;
% Compute new bounding box
/ Create n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
0
/ i n bounding box
0
17 xgrid = linspace(bmin(l),bmax(l),neval);
18 ygrid = linspace(bmin(2),bmax(2),neval);
19 [xe,ye] = m e s h g r i d ( x g r i d , y g r i d ) ;
20 epoints = [xe(:) y e ( : ) ] ;
25 Pf = EM * ( I M \ r h s ) ;
7, P l o t extended data w i t h 2 D - f i t Pf
26 f i g u r e ; hold on; v i e w ( [ - 3 0 , 3 0 ] )
27 plot3(dsites(:,1),dsites(:,2),rhs,'r.','markersize',20);
28 mesh(xe,ye,reshape(Pf,neval,neval));
29 a x i s t i g h t ; hold o f f
7o P l o t data s i t e s w i t h i n t e r p o l a n t (zero contour of 2 D - f i t P f )
30 f i g u r e ; hold on
31 p l o t ( d s i t e s d :N, 1) . d s i t e s (1: N, 2) , 'bo') ;
32 c o n t o u r ( x e , y e , r e s h a p e ( P f , n e v a l , n e v a l ) , [0 0 ] , ' r ' ) ;
33 hold o f f
In the M A T L A B program PointCloud2D .m (see Program 30.1) w e create test

data on lines 4-8 by sampling a parametric curve (in polar coordinates) at irregular
parameter values t. These points are displayed in the left plot of Figure 30.1.
Since we use a k n o w n r e p r e s e n t a t i o n o f t h e c u r v e t o g e n e r a t e t h e p o i n t c l o u d i t is
also possible t o o b t a i n a n e x a c t n o r m a l v e c t o r associated w i t h each d a t a p o i n t (see
lines 6 - 8 ) .
T h e s t r a t e g y for c r e a t i o n o f t h e a u x i l i a r y p o i n t s is t h e same as above, i.e., we
a d d d a t a p o i n t s "inside" a n d " o u t s i d e " t h e g i v e n p o i n t set. T h i s is d o n e b y p l a c i n g
these p o i n t s a l o n g t h e n o r m a l v e c t o r at each o r i g i n a l d a t a p o i n t (see lines 12 a n d
13). T h e d i s t a n c e a l o n g t h e n o r m a l at w h i c h t h e a u x i l i a r y p o i n t s are p l a c e d is t a k e n
t o be 1 % o f t h e size o f t h e m a x i m u m d i m e n s i o n o f t h e b o u n d i n g b o x o f t h e o r i g i n a l
d a t a (see lines 9 - 1 1 ) .
N e x t , t h e p r o b l e m is t u r n e d i n t o a f u l l 2 D i n t e r p o l a t i o n p r o b l e m (whose s o l u t i o n
has a 3 D g r a p h ) b y a d d i n g f u n c t i o n values ( o f t h e u n k n o w n b i v a r i a t e f u n c t i o n /
whose zero-level iso-curve w i l l be t h e desired i n t e r p o l a t i n g c u r v e ) at t h e extended
d a t a set. W e assign a v a l u e o f 0 t o each o r i g i n a l d a t a p o i n t , a n d a v a l u e o f 1 o r
1 t o " o u t s i d e " or "inside" p o i n t s , respectively. T h i s is done o n l i n e 14 o f t h e code
a n d t h e r e s u l t i n g d a t a is d i s p l a y e d i n t h e r i g h t p l o t o f F i g u r e 3 0 . 1 (c.f. also l i n e 27
of t h e c o d e ) .
Fig. 30.1 Point cloud data (left) and extended "inner" and "outer" data (right) for implicit curve
with 81 non-uniform data points.
Now we c a n solve t h e p r o b l e m j u s t l i k e a n y o f o u r 2 D i n t e r p o l a t i o n p r o b l e m s
discussed earlier. I n fact, i n P r o g r a m 3 0 . 1 w e use s t r a i g h t f o r w a r d R B F i n t e r p o l a t i o n
w i t h G a u s s i a n R B F s (see lines 1 a n d 1 9 - 2 5 ) .
F i n a l l y , t h e zero c o n t o u r o f t h e r e s u l t i n g surface is e x t r a c t e d o n l i n e 32 u s i n g
the c o n t o u r command. I n t h e left p l o t o f F i g u r e 30.2 we d i s p l a y a surface p l o t
of t h e b i v a r i a t e R B F i n t e r p o l a n t t o t h e e x t e n d e d d a t a set ( o b t a i n e d v i a t h e mesh
c o m m a n d o n l i n e 2 8 ) , a n d i n t h e r i g h t p l o t w e s h o w t h e final i n t e r p o l a t i n g c u r v e
along w i t h the original data.
Fig. 30.2 Surface fit (left) and zero contour for 81 non-uniform data points.
30.3 A Simplistic Implementation in 3 D v i a Partition of U n i t y

Approximation in M A T L A B
I n t h e M A T L A B p r o g r a m P o i n t C l o u d 3 D _ P U C S (see P r o g r a m 30.2) w e p r e s e n t a f a i r l y
simple i m p l e m e n t a t i o n o f t h e p a r t i t i o n o f u n i t y approach t o i n t e r p o l a t i o n o f p o i n t
c l o u d d a t a i n 3 D . T h e p a r t i t i o n o f u n i t y is c r e a t e d w i t h a S h e p a r d approximant
2
as i n t h e p r e v i o u s c h a p t e r . A s i n P r o g r a m 2 9 . 1 w e use W e n d l a n d ' s C compactly
s u p p o r t e d f u n c t i o n as b o t h t h e S h e p a r d w e i g h t s a n d f o r t h e l o c a l R B F i n t e r p o l a n t s .
T h e d a t a sets used i n o u r e x a m p l e s c o r r e s p o n d t o v a r i o u s r e s o l u t i o n s o f t h e S t a n -
f o r d b u n n y available o n t h e w o r l d - w i d e w e b a t h t t p : / / g r a p h i c s . S t a n f o r d . e d u / -
data/3Dscanrep/. Data sets consisting o f 35947, 8 1 7 1 , 1889, and 453
points are i n c l u d e d i n t h e file bunny . t a r .gz. T h e normals for t h i s kind
of PLY d a t a c a n be c o m p u t e d with the utility normalsply from the pack-
age p l y . t a r . g z p r o v i d e d b y G r e g T u r k , a n d available o n t h e w o r l d - w i d e web
at h t t p : / / w w w . c c . g a t e c h . e d u / p r o j e c t s / l a r g e _ m o d e l s / p l y . h t m l . Results ob-
t a i n e d w i t h t h e P o i n t C l o u d 3 D _ P U C S for t h e 453 a n d 8 1 7 1 p o i n t c l o u d sets are dis-
p l a y e d i n F i g u r e 30.3. Processed d a t a files are i n c l u d e d o n t h e enclosed C D .
M a n y p a r t s o f t h e M A T L A B code for P o i n t C l o u d 3 D _ P U C S are s i m i l a r t o P r o -
g r a m 2 9 . 1 . T h e b u n n y d a t a set i n c l u d i n g p o i n t n o r m a l s is l o a d e d o n l i n e 6, a n d
t h e b o u n d i n g b o x for t h e p o i n t c l o u d a n d i t s m a x i m u m d i m e n s i o n are c o m p u t e d
on lines 7 - 8 . T h e off-surface p o i n t s are a d d e d i n lines 1 0 - 1 4 . T h e n t h e r i g h t - h a n d
side for t h e a u g m e n t e d ( 3 D ) i n t e r p o l a t i o n p r o b l e m is d e f i n e d o n l i n e 15 ( a s s i g n i n g
a value o f 0 for t h e on-surface d a t a p o i n t s , a n d a v a l u e o f 1 for t h e " o u t s i d e " a n d
"inside" off-surface p o i n t s ) , a n d w e r e c o m p u t e t h e b o u n d i n g b o x for t h e a u g m e n t e d
d a t a o n l i n e 16.
We have f o u n d t h a t a r e a s o n a b l e v a l u e for t h e r a d i u s o f t h e p a r t i t i o n o f u n i t y
subdomains seems t o be g i v e n b y t h e m a x i m a l d i m e n s i o n o f t h e b o u n d i n g b o x
d i v i d e d b y t h e cube r o o t o f t h e n u m b e r , M, o f s u b d o m a i n s , i.e., d i a m ( - l j ) = l/w e
with w = \/~M/bdim (see lines 9 a n d 2 8 ) .

The main part of the program (lines 29-46) is almost identical to lines 21-38
of Program 29.1. On lines 47-55 we add the code that creates and displays the
zero-contour iso-surface for the 3D (solid) interpolant Pf along with the point cloud
data.
P r o g r a m 30.2. PointCloud3D_PUCS .m
/. PointCloud3D_PUCS
% S c r i p t t h a t f i t s a s u r f a c e t o 3D p o i n t cloud u s i n g p a r t i t i o n of
/ u n i t y approximation w i t h s p a r s e m a t r i c e s
0
'/. C a l l s on: CSEvalMatrix

7 Uses: k-D t r e e package by Guy Shechter
7. from MATLAB C e n t r a l F i l e Exchange
% Weight f u n c t i o n f o r g l o b a l Shepard p a r t i t i o n of u n i t y weighting
1 wf = @(e,r) r . ""4. * (5*spones ( r ) - 4 * r ) ;
7, The RBF b a s i s f u n c t i o n f o r l o c a l RBF i n t e r p o l a t i o n
2 r b f = @(e,r) r . ~ 4 . * ( 5 * s p o n e s ( r ) - 4 * r ) ;
3 ep = 1; % Parameter f o r b a s i s f u n c t i o n
7, Parameter f o r npu-by-npu-by-npu g r i d of PU c e l l s
4 npu = 8 ;
7o Parameter f o r npu-by-npu-by-npu g r i d of PU c e l l s
5 n e v a l = 25;
7, Load data p o i n t s and compute bounding box
5
6 load('Data3D_Bunny3 ); N = s i z e ( d s i t e s , 1) ;
7 bmin = m i n ( d s i t e s , [] , 1) ; bmax = max ( d s i t e s , [ ] , 1) ;
8 bdim = max(bmax-bmin);
9 wep = npu/bdim;
7o Add a u x i l i a r y p o i n t s along normals " i n s i d e " and " o u t s i d e "
7o F i n d p o i n t s w i t h nonzero normal v e c t o r s and count them
10 withnormals = f i n d ( n o r m a l s ( : , 1 ) I n o r m a l s ( : , 2 ) I n o r m a l s ( : , 3 ) ) ;
11 addpoints = l e n g t h ( w i t h n o r m a l s ) ;
% D i s t a n c e along normal a t which t o p l a c e new p o i n t s
12 d e l t a = bdim/100;
7, Create new p o i n t s
13a dsites(N+l:N+addpoints,:) = ...
13b dsites(withnormals,:) + delta*normals(withnormals,:);
14a dsites(N+addpoints+1:N+2*addpoints,:) = ...
14b dsites(withnormals,:) - delta*normals(withnormals,:);
7o I n t e r p o l a n t i s i m p l i c i t s u r f a c e , i . e . ,
7o " o r i g i n a l " p o i n t s have rhs=0, " i n s i d e " r h s = - l , " o u t s i d e " r h s = l
15 r h s = [ z e r o s ( N , l ) ; o n e s ( a d d p o i n t s , 1 ) ; - o n e s ( a d d p o i n t s , 1 ) ] ;
7o Compute new bounding box
17 ctrs = dsites;
% Create neval-by-neval-by-neval e q u a l l y spaced e v a l u a t i o n
% l o c a t i o n s i n bounding box
18 xgrid = linspace(bmin(1),bmax(l),neval);
19 ygrid = linspace(bmin(2),bmax(2),neval);
20 zgrid = linspace(bmin(3),bmax(3),neval);
21 [xe,ye,ze] = m e s h g r i d ( x g r i d , y g r i d , z g r i d ) ;
22 epoints = [xe(:) y e ( : ) z e ( : ) ] ;
/ Create npu-by-npu-by-npu e q u a l l y spaced c e n t e r s of PU c e l l s
0
% i n bounding box
23 puxgrid = l i n s p a c e ( b m i n ( l ) , b m a x ( l ) , n p u ) ;
24 puygrid = linspace(bmin(2),bmax(2),npu);
25 p u z g r i d = linspace(bmin(3),bmax(3),npu);
26 [xpu,ypu,zpu] = meshgrid(puxgrid,puygrid,puzgrid);
27 c e l l c t r s = [xpu(:) ypu(:) z p u ( : ) ] ;
28 c e l l r a d i u s = 1/wep;
/ Compute Shepard e v a l u a t i o n m a t r i x
0
29 DM_eval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , c e l l c t r s , w e p ) ;
30 SEM = wf(wep,DM_eval);
31 SEM = spdiags(l./(SEM*ones(npu"3,1)),0,neval"3,neval"3)*SEM;
/ B u i l d k-D t r e e s f o r d a t a s i t e s and e v a l u a t i o n p o i n t s
0
32 [tmp,tmp,datatree] = k d t r e e ( d s i t e s , [ ] ) ;
33 [tmp,tmp,evaltree] = k d t r e e ( e p o i n t s , [ ] ) ;
34 Pf = z e r o s ( n e v a l " 3 , 1 ) ; % i n i t i a l i z e
35 f o r j=l:npu"3
% Find data s i t e s i n c e l l j
36a [ p t s , d i s t , i d x ] = kdrangequery(datatree,...
37 i f ( l e n g t h ( i d x ) > 0)
0/ B u i l d l o c a l i n t e r p o l a t i o n m a t r i x f o r c e l l j
38a DM_data = D i s t a n c e M a t r i x C S R B F ( d s i t e s ( i d x , : ) , . . .
0/ F i n d e v a l u a t i o n p o i n t s i n c e l l j
40a [epts,edist,eidx] = kdrangequery(evaltree,...
0/ Compute l o c a l e v a l u a t i o n m a t r i x
41a DM_eval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s ( e i d x , : ) , . . .
0/ Compute l o c a l RBF i n t e r p o l a n t
43 l o c a l f i t = EM * ( I M \ r h s ( i d x ) ) ;
r
/ Accumulate g l o b a l f i t
0
44 Pf(eidx) = Pf(eidx) + localfit.*SEM(eidx,j);

45 end
46 end
% P l o t d a t a s i t e s w i t h i n t e r p o l a n t (zero contour of 3 D - f i t P f )
47 f i g u r e ; h o l d on
48 p l o t 3 ( d s i t e s ( l :N,1) , d s i t e s d :N,2) , d s i t e s ( l :N,3) , 'bo') ;
49a p f i t = p a t c h d s o s u r f ace (xe ,ye ,ze, . . .
49b reshape(Pf,neval,neval,neval),0));
50 isonormals(xe,ye,ze.reshape(Pf,neval,neval,neval),pfit)
51a set(pfit,'FaceLighting','gouraud','FaceColor',...
51b 'red','EdgeColor','none');
52 l i g h t ( ' P o s i t i o n ' , [ 0 0 1] , ' S t y l e ' , ' i n f i n i t e ' ) ;
53 d a s p e c t ( [ l 1 1 ] ) ; view( [0,90] ) ;
54 a x i s ( [ b m i n ( l ) bmax(l) bmin(2) bmax(2) bmin(3) b m a x ( 3 ) ] ) ;
55 a x i s o f f ; hold off
I n F i g u r e 30.3 we d i s p l a y p a r t i t i o n o f u n i t y fits based o n l o c a l R B F i n t e r p o l a n t s

2
built w i t h compactly supported Wendland's C basis f u n c t i o n s . The point cloud
d a t a sets consist o f 453 p o i n t s ( t o p p l o t s i n F i g u r e 30.3) a n d 8171 p o i n t s ( b o t t o m
p l o t s i n F i g u r e 30.3). T h e a u g m e n t e d d a t a sets for t h e 3D i n t e r p o l a n t s are a l m o s t
t h r e e t i m e s as large (since n o t e v e r y d a t a p o i n t has a n o r m a l v e c t o r a s s o c i a t e d w i t h
it). O n t h e left we s h o w t h e f i t t e d surface a l o n g w i t h t h e d a t a p o i n t s , a n d o n t h e
r i g h t t h e f i t is d i s p l a y e d b y itself. F o r t h e p l o t s i n t h e b o t t o m p a r t o f F i g u r e 30.3 t h e
s i m p l e iso-surface p l o t i n M A T L A B does a s u r p r i s i n g l y g o o d j o b . I n o t h e r s i t u a t i o n s ,
however, i t causes some a r t i f a c t s s u c h as t h e e x t r a surface f r a g m e n t n e a r t h e b u n n y ' s
ear i n t h e t o p p a r t o f F i g u r e 30.3.
F o r t h e t o p p l o t s i n F i g u r e 30.3 w e u s e d 8 = 256 s u b d o m a i n s a n d 2 5 = 15625
3 3
e v a l u a t i o n p o i n t s i n t h e b o u n d i n g b o x o f t h e 3D d a t a , a n d for those o n t h e b o t t o m o f
F i g u r e 30.3 w e used 3 2 = 32768 s u b d o m a i n s a n d 5 0 = 125000 e v a l u a t i o n p o i n t s .
3 3
.4.:
Chapter 31
Fixed Level Residual Iteration
I n t h e n e x t few c h a p t e r s we w i l l l o o k a t v a r i o u s v e r s i o n s o f r e s i d u a l i t e r a t i o n . T h e
basic idea o f u s i n g a n i t e r a t i v e a l g o r i t h m i n w h i c h one takes a d v a n t a g e o f t h e r e s i d u a l
o f a n i n i t i a l a p p r o x i m a t i o n t o o b t a i n a m o r e a c c u r a t e s o l u t i o n is w e l l - k n o w n i n m a n y
branches o f m a t h e m a t i c s .
31.1 Iterative Refinement
F o r e x a m p l e , i n n u m e r i c a l l i n e a r a l g e b r a t h i s process is k n o w n as iterative refinement

(see, e.g., [ K i n c a i d a n d C h e n e y ( 2 0 0 2 ) ] ) . W e m i g h t be i n t e r e s t e d i n s o l v i n g a s y s t e m
o f l i n e a r e q u a t i o n s Ax = b, a n d o b t a i n a ( n u m e r i c a l ) s o l u t i o n xo by applying
a n a l g o r i t h m such as G a u s s i a n e l i m i n a t i o n . W e can then compute the residual
r = b AXQ, a n d realize t h a t i t is r e l a t e d t o t h e e r r o r , e = x XQ, v i a t h e r e l a t i o n
Ae = Ax AXQ - r. (31.1)
T h u s , b y a d d i n g t h e ( n u m e r i c a l ) s o l u t i o n en o f e q u a t i o n (31.1) t o t h e i n i t i a l s o l u t i o n
XQ one expects t o i m p r o v e t h e i n i t i a l a p p r o x i m a t i o n t o x\ = XQ + eo (since t h e t r u e
s o l u t i o n x xo + e). O f course, t h i s p r o c e d u r e c a n be r e p e a t e d i t e r a t i v e l y . T h i s
leads t o t h e a l g o r i t h m
Algorithm 31.1. Iterative refinement
(1) C o m p u t e a n a p p r o x i m a t e s o l u t i o n ccn o f Ax = b.
(2) F o r k 1 , 2 , . . . do
(a) C o m p u t e t h e r e s i d u a l r k = b Ax -i-
k
( b ) Solve Ae k = r .
k
(c) U p d a t e x k = x - i + e .
k k
W e c a n r e w r i t e t h e last s t a t e m e n t i n t h e a l g o r i t h m as
x k - Xk-i + B(b - Ax -i)

k
(31.2)
w h e r e B is a n approximate (or numerical) inverse o f A characterized by the p r o p e r t y

t h a t || J BA\\ < 1 for some m a t r i x n o r m . T h i s c o n d i t i o n allows us t o express t h e
265
e x a c t inverse o f A b y a N e u m a n n series, i.e.,
oo
E( - )
1 1
A ' = (BA)- !! = J B A J B.
3=0
T h e r e f o r e t h e e x a c t s o l u t i o n o f Ax = b c a n be w r i t t e n i n t h e f o r m
E( -
1
x = A~ b = B A Bb. (31.3)
7
y
3=0
For t h e i t e r a t i v e r e f i n e m e n t a l g o r i t h m , o n t h e o t h e r h a n d , we have f r o m ( 3 1 . 2 )
a n d t h e fact t h a t XQ = Bb, that
x k = x - i + B(bk - Ax -i)k
= (I - BA)x - k 1 + Bb
= ( I - BA)x -! k +x . 0 (31.4)
W e c a n r e c u r s i v e l y s u b s t i t u t e t h i s r e l a t i o n b a c k i n for x k - i , x -2,
k etc., a n d o b t a i n
XQ
E( J
- B A
y Bb. (31.5)
3=0 3=0
I t is n o w easy t o see t h a t t h e i t e r a t e s x k o f t h e r e f i n e m e n t a l g o r i t h m converge

t o t h e e x a c t s o l u t i o n x. W e s i m p l y l o o k a t t h e difference x x k a t l e v e l k, i.e.,
f r o m (31.3) a n d (31.5) we o b t a i n
Bb,
x x k E ( - J
B A
y
j=k+l
whose n o r m goes t o zero for k oo since \ \ I BA\\ < 1 by the assumption made
on t h e a p p r o x i m a t e inverse B.
W e n o w a p p l y these ideas t o R B F i n t e r p o l a t i o n a n d M L S a p p r o x i m a t i o n . In
t h i s a n d t h e f o l l o w i n g c h a p t e r s w e w i l l consider t h r e e d i f f e r e n t scenarios:
F i x e d level i t e r a t i o n , i.e., t h e i t e r a t i v e r e f i n e m e n t a l g o r i t h m is p e r f o r m e d o n a
fixed set o f d a t a p o i n t s X .
M u l t i l e v e l i t e r a t i o n , i.e., w e w o r k w i t h a n e s t e d sequence o f d a t a sets XQ C
X\ c C X.
A d a p t i v e i t e r a t i o n , i.e., r e s i d u a l i t e r a t i o n is p e r f o r m e d o n a d a p t i v e l y chosen
subsets o f X , e.g., b y s t a r t i n g w i t h some s m a l l subset o f X a n d t h e n a d d i n g one
p o i n t a t a t i m e f r o m t h e r e m a i n d e r o f X t h a t is d e t e r m i n e d t o be " o p t i m a l " .
N o t e t h a t t h e t h i r d a p p r o a c h is s i m i l a r t o t h e a d a p t i v e k n o t i n s e r t i o n a l g o r i t h m
of C h a p t e r 2 1 .
31.2 Fixed Level Iteration
T h e s i m p l e s t s e t t i n g for a meshfree r e s i d u a l i t e r a t i o n a l g o r i t h m arises w h e n w e

f i x t h e d a t a sites X = {xi,..., CCJV} t h r o u g h o u t t h e i t e r a t i v e p r o c e d u r e . F o r t h i s
setting the iterative refinement a l g o r i t h m f r o m l i n e a r a l g e b r a c a n be a d a p t e d i n
a s t r a i g h t f o r w a r d way. We first discuss r e s i d u a l i t e r a t i o n for quasi-interpolants
(or a p p r o x i m a t e M L S a p p r o x i m a n t s ) based o n ( r a d i a l ) g e n e r a t i n g f u n c t i o n s Wj,
j = l,...,iV.
I f w e keep t h e same set o f g e n e r a t i n g f u n c t i o n s for a l l steps o f t h e i t e r a t i o n t h e n
we are p e r f o r m i n g non-stationary approximation a n d we o b t a i n
A l g o r i t h m 31.2. F i x e d l e v e l r e s i d u a l i t e r a t i o n based o n q u a s i - i n t e r p o l a t i o n
0
(1) C o m p u t e a n i n i t i a l a p p r o x i m a t i o n V ^ f t o t h e d a t a {(xj, f(xj)), j = 1,..., N}
N
o f t h e f o r m vf\x) = ^ f{xj)^j{x).
3= 1
(2) F o r k = 1 , 2 , . . . d o
k X
(a) C o m p u t e t h e residuals ri-(xj) = f(xj) Vj ~ \xj) for a l l j 1 , . . . , N.
N
( b ) C o m p u t e t h e c o r r e c t i o n u(x) rkjxj)^j(x).
3= 1
X
(c) U p d a t e ^ ( c c ) = vf~ \x) + u{x).
A s for t h e i t e r a t i v e r e f i n e m e n t a l g o r i t h m , w e c a n r e w r i t e t h e last l i n e o f t h e
a l g o r i t h m as
x X
Vf\x) = Vf~ \x) + j r [f {Xj) - Vf- \x )] 3 * {x).
s
3= 1
N o w w e r e s t r i c t t h e e v a l u a t i o n o f t h e a p p r o x i m a t i o n t o t h e d a t a sites o n l y . T h u s ,
we have
JV
v f \ X i ) = Vf- \ X
x i ) + E [/(;) - V f
k x
- \ X j )] Vjixi), i = 1,..., N.
3= 1
(31.6)
N e x t w e collect a l l o f these N e q u a t i o n s i n t o one single m a t r i x - v e c t o r e q u a t i o n b y
i n t r o d u c i n g t h e vectors / = [ / ( a ? i ) , f(x ),.. 2 , /(JCJV)] and ^ = [ ^ i , , ^ N ]
T h i s allows us t o r e w r i t e t h e i n i t i a l a p p r o x i m a n t i n m a t r i x - v e c t o r f o r m
T
v f \ x ) = ^ (x)f. (31.7)
M o r e o v e r , e v a l u a t i o n o f t h e v e c t o r \I> o f g e n e r a t i n g f u n c t i o n s at t h e d a t a sites Xi,

T
i = 1 , . . . , N gives rise t o a m a t r i x A w i t h r o w s V (xi), i = 1,...,N. Therefore,
(31.6) n o w becomes
k ) { k X ) { k l )
T/ = V f - + A(f - V f ~ ),
w h e r e we i n t e r p r e t "PfW as a v e c t o r o f values o f t h e a p p r o x i m a n t at t h e d a t a sites,

T
i.e.,V ^
f = [vf\x ),...,vf\x )\ .
l N
N e x t we f o l l o w analogous steps as i n o u r d i s c u s s i o n o f i t e r a t i v e r e f i n e m e n t above.

Thus
{ k ) { k x )
V f = V f ~ + A(f - Vf^-V)
k 1)
= {I-A)V ( - +Af f (31.8)
k 1 ) i 0 )
= ( I - A ) V / - + V f ,
5
since (31.7) i m p l i e s t h a t o n t h e d a t a sites w e h a v e T / ^ = Af. N o w we can again
recursively substitute back i n a n d o b t a i n
k k
<p W (31.9)
f =
Af
J=0 J=0
N o t e t h a t here w e have t o deal o n l y w i t h t h e m a t r i x A since t h e c o m p u t a t i o n

of t h e c o r r e c t i o n i n t h e a l g o r i t h m does n o t r e q u i r e t h e s o l u t i o n o f a l i n e a r s y s t e m .
7 A c a n s e e n 3 5 a
A s before, t h e s u m X ^ = o ( V be t r u n c a t e d N e u m a n n series
e x p a n s i o n for t h e inverse o f t h e m a t r i x A. I f w e d e m a n d t h a t \ \ I A\\ < 1, t h e n
t h e m a t r i x (^2j (I =0 A)^ is a n a p p r o x i m a t e inverse o f A w h i c h converges t o A - 1
k
since || J A\\ 0 for A; * oo. M o r e d e t a i l s (such as sufficient c o n d i t i o n s u n d e r
w h i c h \ \ I A\\ < 1) are g i v e n i n [Fasshauer a n d Z h a n g ( 2 0 0 6 ) ] .
I n order to establish a connection between iterated (approximate) M L S approx-
i m a t i o n a n d R B F i n t e r p o l a t i o n w e assume t h e m a t r i x A t o be p o s i t i v e d e f i n i t e a n d
g e n e r a t e d b y r a d i a l basis f u n c t i o n s _>j = ip(\\ Xj\\) as i n o u r discussions i n ear-
lier c h a p t e r s . T h e n A c o r r e s p o n d s t o a n R B F i n t e r p o l a t i o n m a t r i x , a n d w e see
t h a t t h e i t e r a t e d ( a p p r o x i m a t e ) M L S a p p r o x i m a t i o n converges t o t h e R B F i n t e r -
p o l a n t p r o v i d e d t h e same f u n c t i o n spaces are used, i.e., span-fSE'j, j = 1 , . . . , N} =
spanf^-, j = 1,...,N}.
I n p a r t i c u l a r , w e have e s t a b l i s h e d
T h e o r e m 31.1. Assume strictly positive definite (radial) generat-

ing functions for approximate MLS approximation as discussed in Chapter 26. Then
the residual iteration fit of Algorithm 31.2 based on approximate MLS approxima-
tion with these generating functions converges to the RBF interpolant based on the
same basis functions ^ i , . . . , \I/jv provided the matrix A with entries Aij = tyj(xi)
satisfies \ \ I - A\\ < 1.
A sufficient c o n d i t i o n for A t o satisfy | | J A\\ < 1 w a s g i v e n i n [Fasshauer a n d

Z h a n g (2006)]. A s l o n g as t h e m a x i m u m r o w s u m o f A is s m a l l e n o u g h , i.e.,
m ax 2
i=1 2 ,^|Eiî|< '
w e have convergence o f t h e r e s i d u a l i t e r a t i o n a l g o r i t h m . T h i s c o n d i t i o n is c l o s e l y
r e l a t e d t o t h e Lebesgue f u n c t i o n o f t h e R B F i n t e r p o l a n t . F o r e x a m p l e , i t is n o t
h a r d t o see t h a t S h e p a r d g e n e r a t i n g f u n c t i o n s satisfy t h i s c o n d i t i o n since each r o w
s u m is e q u a l t o one due t o t h e p a r t i t i o n o f u n i t y p r o p e r t y o f t h e S h e p a r d f u n c t i o n s .
For o t h e r t y p e s o f f u n c t i o n s t h e c o n d i t i o n c a n be satisfied b y a n a p p r o p r i a t e s c a l i n g
of t h e basic f u n c t i o n w i t h a s u f f i c i e n t l y s m a l l shape p a r a m e t e r . H o w e v e r , i f e is
t a k e n t o o s m a l l , t h e n t h e a l g o r i t h m converges v e r y s l o w l y . A series o f e x p e r i m e n t s
a n a l y z i n g t h e b e h a v i o r o f t h e a l g o r i t h m are p r e s e n t e d i n [Fasshauer and Zhang
(2006)] a n d also i n S e c t i o n 31.4 b e l o w .
T h e q u e s t i o n o f w h e t h e r t h e a p p r o x i m a t e M L S g e n e r a t i n g f u n c t i o n s are s t r i c t l y
p o s i t i v e d e f i n i t e has been i r r e l e v a n t u p t o t h i s p o i n t . H o w e v e r , i n o r d e r t o m a k e
t h e c o n n e c t i o n b e t w e e n A M L S a p p r o x i m a t i o n a n d R B F i n t e r p o l a t i o n as s t a t e d i n
T h e o r e m 31.1 i t is i m p o r t a n t t o find A M L S g e n e r a t i n g f u n c t i o n s t h a t satisfy t h i s
additional condition. O f course, a n y ( a p p r o p r i a t e l y n o r m a l i z e d ) s t r i c t l y p o s i t i v e
definite f u n c t i o n c a n serve as a second-order a c c u r a t e A M L S g e n e r a t i n g f u n c t i o n .
H o w e v e r , i t is a n o p e n q u e s t i o n for w h i c h o f these f u n c t i o n s t h e i r h i g h e r - o r d e r
g e n e r a t i n g f u n c t i o n s c o m p u t e d a c c o r d i n g t o o u r d i s c u s s i o n i n C h a p t e r 26 are also
s t r i c t l y positive definite.
T h e f a m i l y o f L a g u e r r e - G a u s s i a n s ( 4 . 2 ) p r o v i d e s one e x a m p l e o f g e n e r a t i n g / b a s i s
f u n c t i o n s t h a t c a n be used t o i l l u s t r a t e T h e o r e m 3 1 . 1 (see t h e n u m e r i c a l e x p e r i m e n t s
b e l o w ) since t h e i r F o u r i e r t r a n s f o r m s are p o s i t i v e (see ( 4 . 3 ) ) .
31.3 Modifications o f the Basic Fixed Level I t e r a t i o n A l g o r i t h m
I f we s t a r t f r o m t h e i n t e r p o l a t i o n e n d , t h e n t h e i n t e r p o l a t i o n c o n d i t i o n s V/(xi) =
f(xi) t e l l us t h a t we need t o solve t h e l i n e a r s y s t e m Ac = f i n o r d e r t o find t h e
coefficients o f t h e R B F e x p a n s i o n
N
F o l l o w i n g t h e same i t e r a t i v e p r o c e d u r e as a b o v e (c.f. ( 3 1 . 4 ) ) t h i s leads t o

c k = c - i +B(f-Ac - )
f c k 1 (31.10)
k
= ^2(J-BAyBf, (31.11)
3=0
w h e r e B is a n a p p r o x i m a t e inverse o f A as i n S e c t i o n 3 1 . 1 a n d w e let en = Bf.
Here c k is t h e k-th step a p p r o x i m a t i o n t o t h e coefficient v e c t o r c T
[c\,...,c/v] -
E q u a t i o n (31.10) c a n also be r e w r i t t e n as
c fc = (I - BA)ck-i + Bf,
a n d t h e r e f o r e c o r r e s p o n d s t o a s t a n d a r d s t a t i o n a r y i t e r a t i o n for t h e s o l u t i o n o f
l i n e a r systems (see, e.g., p . 620 o f [ M e y e r ( 2 0 0 0 ) ] ) . T h e s p l i t t i n g m a t r i c e s s u c h t h a t
x l X
A = M N are M = B~ , N = B~ - A, a n d H = M~ N = (/ - BA).
On t h e o t h e r h a n d , (31.11) gives us a n i n t e r p r e t a t i o n o f t h e r e s i d u a l i t e r a t i o n
as a K r y l o v subspace m e t h o d w i t h t h e K r y l o v subspaces g e n e r a t e d b y t h e m a t r i x
I BA and the vector Bf.
I n t h e q u a s i - i n t e r p o l a t i o n f o r m u l a t i o n t h e c o r r e s p o n d i n g f o r m u l a s are g i v e n b y
(31.9), i.e.,
k
(31.12)
3=0
a n d c a n also be i n t e r p r e t e d as a K r y l o v subspace i t e r a t i o n w i t h t h e K r y l o v subspaces

g e n e r a t e d b y t h e m a t r i x I A a n d t h e v e c t o r Af. N o t e , however, t h a t i n (31.11)
we are c o m p u t i n g t h e coefficients o f t h e R B F i n t e r p o l a n t , w h i l e i n (31.12) w e are
directly computing an approximation to the interpolant.
A n a t u r a l p r o b l e m associated w i t h K r y l o v subspace m e t h o d s is t h e determina-
a
t i o n o f coefficients (search d i r e c t i o n s ) dj s u c h t h a t Y^=o j(^ ~ A)Âf converges
faster t h a n t h e generic m e t h o d w i t h a.j = 1 discussed above., Some r e l a t e d w o r k is
discussed i n t h e c o n t e x t o f t h e F a u l - P o w e l l a l g o r i t h m i n S e c t i o n 33.2.
W e c o n c l u d e o u r discussion o f m o d i f i c a t i o n s o f t h e basic f i x e d l e v e l r e s i d u a l i t e r -
ation algorithm by noting t h a t the usual stationary approximation m e t h o d cannot
be a p p l i e d w i t h i n t h e fixed l e v e l i t e r a t i o n p a r a d i g m since w e do n o t have a c h a n g e
i n d a t a d e n s i t y t h a t c a n be used as a g u i d e t o re-scale t h e basis f u n c t i o n s . H o w e v e r ,
i t is possible t o generalize t h e n o n - s t a t i o n a r y a l g o r i t h m t o a m o r e g e n e r a l s e t t i n g
i n w h i c h we change t h e a p p r o x i m a t i o n space f r o m one s t e p t o t h e n e x t . A s i n t h e
n o n - s t a t i o n a r y s e t t i n g we c a n o n l y a p p l y t h i s s t r a t e g y w i t h a p p r o x i m a t i o n m e t h o d s
since a n i n t e r p o l a t i o n m e t h o d w i l l i m m e d i a t e l y l e a d t o a zero i n i t i a l r e s i d u a l . For
e x a m p l e , one c o u l d devise a n a l g o r i t h m i n w h i c h w e use c r o s s - v a l i d a t i o n a t each i t -
e r a t i o n step t o d e t e r m i n e t h e o p t i m a l shape p a r a m e t e r (or s u p p o r t size) for t h e n e x t
r e s i d u a l c o r r e c t i o n . S u c h a n a l g o r i t h m w o u l d also fit i n t o t h e c a t e g o r y o f a d a p t i v e
i t e r a t i o n s as discussed b e l o w .
31.4 Iterated Approximate M L S Approximation in MATLAB
W e n o w i l l u s t r a t e t h e fixed l e v e l r e s i d u a l i t e r a t i o n a l g o r i t h m w i t h some M A T L A B
e x p e r i m e n t s based o n t h e i t e r a t i o n o f a p p r o x i m a t e M L S a p p r o x i m a n t s w i t h G a u s -
s i a n g e n e r a t i n g f u n c t i o n s . T o o b t a i n some t e s t d a t a w e use F r a n k e ' s f u n c t i o n (2.2)
o n 289 H a l t o n p o i n t s i n t h e u n i t s q u a r e .
I n o u r earlier discussion o f a p p r o x i m a t e M L S a p p r o x i m a t i o n w e l i m i t e d ourselves
m o s t l y t o t h e case o f u n i f o r m l y spaced d a t a . T h i s was d u e t o t h e fact, t h a t for
n o n - u n i f o r m l y spaced d a t a one needs t o scale t h e g e n e r a t i n g f u n c t i o n s i n d i v i d u a l l y
according to the local variation i n the data density i n order t o m a i n t a i n the approx-
i m a t e a p p r o x i m a t i o n o r d e r s s t a t e d i n T h e o r e m 2 6 . 1 . N o w t h e convergence r e s u l t
o f T h e o r e m 3 1 . 1 shows t h a t w e n o l o n g e r need t o feel b o u n d b y t h o s e l i m i t a t i o n s .
I t e r a t i o n w i l l a u t o m a t i c a l l y i m p r o v e t h e a p p r o x i m a t e M L S f i t also o n n o n - u n i f o r m
d a t a . O n t h e o t h e r h a n d , t h i s o b s e r v a t i o n suggests t h a t t h e use o f a u n i f o r m shape
p a r a m e t e r for R B F i n t e r p o l a t i o n is m o s t l i k e l y n o t t h e i d e a l s t r a t e g y t o o b t a i n
h i g h l y a c c u r a t e R B F fits. W h i l e a few e x p e r i m e n t s o f R B F i n t e r p o l a t i o n w i t h v a r y -
i n g shape p a r a m e t e r s exist i n t h e l i t e r a t u r e (see, e.g., [Kansa a n d Carlson (1992);
B o z z i n i et al. (2002); F o r n b e r g a n d Z u e v ( 2 0 0 6 ) ] ) , t h e t h e o r y for t h i s case is o n l y
r u d i m e n t a r y [ B o z z i n i et al. ( 2 0 0 2 ) ] .
T h e M A T L A B code for o u r e x a m p l e s is p r o v i d e d i n P r o g r a m 3 1 . 1 . Since w e are
i t e r a t i n g t h e a p p r o x i m a t e M L S a p p r o x i m a t i o n w e define t h e scale o f t h e g e n e r a t -
ing functions i n terms of the parameter V (see l i n e 2 ) . H o w e v e r , since t h e RBF
( G a u s s i a n ) is defined w i t h t h e p a r a m e t e r e w e c o n v e r t T> t o e based o n t h e for-
m u l a e l/(\/T)h). W e a p p r o x i m a t e h (even for n o n - u n i f o r m H a l t o n p o i n t s ) b y
h = 1/(VN 1), w h e r e N is t h e n u m b e r o f d a t a p o i n t s ( i n 2 D ) .
I n c o n t r a s t t o p r e v i o u s p r o g r a m s w e n o w r e q u i r e t w o sets o f e v a l u a t i o n p o i n t s .
T h e u s u a l e p o i n t s t h a t we e m p l o y for e r r o r c o m p u t a t i o n a n d p l o t t i n g a l o n g w i t h
a n o t h e r set r e s p o i n t s , t h e p o i n t s a t w h i c h w e e v a l u a t e t h e residuals d u r i n g t h e
iterative procedure. These p o i n t s c o i n c i d e w i t h t h e d a t a p o i n t s (see l i n e 13). The
i t e r a t i o n o n lines 2 3 - 2 8 is e q u i v a l e n t t o t h e f o r m u l a t i o n i n A l g o r i t h m 31.2 a b o v e .
P r o g r a m 3 1 . 1 . I t e r a t e d _ M L S A p p r o x A p p r o x 2 D .m
/ I t e r a t e d _ M L S A p p r o x A p p r o x 2 D
0
% Script that performs iterated approximate MLS a p p r o x i m a t i o n

1 r b f = @(e,r) exp(-(e*r).~2);
2 D = 64/9; % Parameter f o r b a s i s function
7, D e f i n e F r a n k e ' s f u n c t i o n as t e s t f unct ion
3 f l = <-(x,y) 0.75*exp(-((9*x-2). ~2+(9*y-2).~2)/4);
4 f 2 = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x + l ) . ~ 2 / 4 9 + ( 9 * y + l ) . " 2 / 1 0 ) ) ;
5 f3 = @(x,y) 0.5*exp(-((9*x-7).~2+(9*y-3).~2)/4);
6 f 4 = @(x,y) 0.2*exp(-((9*x-4).~2+(9*y-7).~2));
7 testfunction = @(x,y) f1(x,y)+f2(x,y)+f3(x,y)-f4(x,y);
8 neval =40;
9 N = 289; gridtype = 'h';
% Convert D t o epsilon f o r use with basis function definition
10 h = l/(sqrt(N)-l); ep = 1/(sqrt(D)*h);
% Number o f l e v e l s for multilevel iteration
11 maxlevel = 10000;
% Load d a t a points
0
12 name = s p r i n t f ( ' D a t a 2 D _ y d / s ' , N , g r i d t y p e ) ;
o 0 load(name)
13 respoints = dsites; ctrs = dsites;
% Create neval-by-neval equally spaced e v a l u a t i o n locations
/, i n t h e unit square
15 epoints = [xe(:) y e ( : ) ] ;
7. Compute e x a c t s o l u t i o n
% Compute e v a l u a t i o n m a t r i x d i r e c t l y based on t h e d i s t a n c e s
7, between t h e e v a l u a t i o n p o i n t s and c e n t e r s
17 DM = D i s t a n c e M a t r i x ( e p o i n t s , c t r s ) ;
18 EM = rbf(ep,DM)/(pi*D);
Compute - f o r a l l l e v e l s - e v a l u a t i o n m a t r i c e s f o r
% r e s i d u a l s d i r e c t l y based on t h e d i s t a n c e s between t h e
% next f i n e r p o i n t s ( r e s p o i n t s ) and c e n t e r s
19 DM = D i s t a n c e M a t r i x ( r e s p o i n t s , c t r s ) ;
20 RM = rbf(ep,DM)/(pi*D);
21 Pf = z e r o s ( n e v a l ~ 2 , 1 ) ; % initialize
% C r e a t e v e c t o r of f u n c t i o n v a l u e s ( i n i t i a l r e s i d u a l ) ,
23 f o r level=l:maxlevel
% E v a l u a t e on e v a l u a t i o n p o i n t s
/ ( f o r e r r o r computation and p l o t t i n g )
0
24 Pf = Pf + EM*rhs;
% Compute new r e s i d u a l
25 r h s = r h s - RM*rhs;
26 maxerr(level) = norm(Pf-exact,inf);
27 r m s _ e r r ( l e v e l ) = norm(Pf-exact)/neval;
28 end
29 figure; semilogy(1:maxlevel,maxerr,'b',1:maxlevel,rms_errr');
A c c o r d i n g t o t h e e x p e r i m e n t s s h o w n i n F i g u r e 2.5 t h e o p t i m a l shape p a r a m e t e r
for Gaussian i n t e r p o l a t i o n t o F r a n k e ' s f u n c t i o n o n 289 H a l t o n p o i n t s is close t o
e 6. T h e c o r r e s p o n d i n g v a l u e o f T> for t h e G a u s s i a n as a n a p p r o x i m a t e M L S
g e n e r a t i n g f u n c t i o n is T> 6 4 / 9 (since e = l/(\/T)h), a n d we a p p r o x i m a t e h for
t h e n o n - u n i f o r m H a l t o n p o i n t s b y t h e v a l u e w e w o u l d h a v e for u n i f o r m p o i n t s , i.e.,
h = l/(s/N 1 ) . F o r t h i s v a l u e o f t h e s h a p e p a r a m e t e r w e see t h e convergence
b e h a v i o r o f t h e a p p r o x i m a t e M L S r e s i d u a l i t e r a t i o n i n t h e left p l o t o f F i g u r e 3 1 . 1 .
T h e f i n a l m a x i m u m e r r o r after 10000 i t e r a t i o n s is 9.921772e-002 ( t o p / s o l i d c u r v e ) ,
a n d t h e R M S e r r o r is 3.939342e-003 ( l o w e r / d a s h e d c u r v e ) . For comparison, the
e r r o r s for t h e c o r r e s p o n d i n g R B F i n t e r p o l a n t ( w h i c h is t h e t h e o r e t i c a l l i m i t o f t h e
r e s i d u a l i t e r a t i o n ) are 3.238735e-002 for t h e m a x i m u m e r r o r a n d 1.074443e-003 for
t h e R M S e r r o r . T h e s e e r r o r s are i n c l u d e d as h o r i z o n t a l s t r a i g h t lines i n t h e left p l o t
of F i g u r e 3 1 . 1 .
r
# Iterations V
Fig. 31.1 Convergence for iterated M L S approximation based on Gaussian generating functions
with T> = 64/9 (e = 6) to data sampled from Franke's function at 289 Halton points (left), and fit
for an R B F interpolant based on Gaussians with e = 1 (right).
A n a d v a n t a g e o f t h e r e s i d u a l i t e r a t i o n is t h a t i t a l l o w s us t o c o m p u t e r a d i a l basis
a p p r o x i m a t i o n s also for values o f t h e shape p a r a m e t e r for w h i c h t h e i n t e r p o l a t i o n
m a t r i x is v e r y i l l - c o n d i t i o n e d . F o r e x a m p l e , t h e r i g h t p l o t o f F i g u r e 3 1 . 1 shows a n
R B F i n t e r p o l a n t t o t h e 289 H a l t o n samples o f F r a n k e ' s f u n c t i o n based o n G a u s s i a n s
w i t h shape p a r a m e t e r e = 1. T h e r e c i p r o c a l c o n d i t i o n n u m b e r e s t i m a t e p r o v i d e d b y
M A T L A B for t h e i n t e r p o l a t i o n m a t r i x for t h i s p r o b l e m is RCOND = 2.132739e-020.
I n these i l l - c o n d i t i o n e d cases convergence o f t h e r e s i d u a l i t e r a t i o n t o t h e l i m i t
is r a t h e r slow, b u t t h e a p p r o x i m a t i o n s c a n be c o m p u t e d v e r y s t a b l y . I n the left
p l o t o f F i g u r e 31.2 w e s h o w t h e convergence b e h a v i o r for t h e r e s i d u a l i t e r a t i o n
w i t h a p p r o x i m a t e M L S a p p r o x i m a n t s based o n G a u s s i a n g e n e r a t i n g f u n c t i o n s w i t h
V 256 ( c o r r e s p o n d i n g t o e 1). N o t e t h a t t h e a p p r o x i m a t i o n e r r o r s for t h e
i t e r a t i v e scheme q u i c k l y d r o p b e l o w t h e m a x i m u m e r r o r o f 2 . 5 0 7 0 1 7 e + 0 0 0 a n d R M S
e r r o r o f 2.186992e-001 o f t h e m o s t l y m e a n i n g l e s s i n t e r p o l a n t . T h e c o r r e s p o n d i n g f i t
for t h e i t e r a t i v e m e t h o d is d i s p l a y e d i n t h e r i g h t p l o t o f t h e f i g u r e . W h i l e t h i s f i t
is n o t v e r y a c c u r a t e , i t is s t i l l m u c h m o r e r e l i a b l e t h a n t h e f i t c o n s i s t i n g o f m o s t l y
n u m e r i c a l noise s h o w n i n t h e r i g h t p l o t o f F i g u r e 3 1 . 1 . T h e f i n a l e r r o r s after 10000
i t e r a t i o n s are 2.933448e-001 for t h e m a x i m u m e r r o r a n d 8.470775e-002 for t h e R M S
error.
I t is o f i n t e r e s t t o n o t e t h a t t h e r e s i d u a l i t e r a t i o n shows t h e m o s t dramatic
error improvement d u r i n g the first few i t e r a t i o n s . T h u s , o n l y a few i t e r a t i o n s o f
a p p r o x i m a t e M L S a p p r o x i m a t i o n are r e q u i r e d t o o b t a i n a reasonable ( a n d s t a b l y
computable) a p p r o x i m a t i o n t o the R B F i n t e r p o l a n t . Moreover, we emphasize again
t h a t w h i l e o u r discussion o f a p p r o x i m a t e M L S a p p r o x i m a t i o n was m o s t l y l i m i t e d
t o t h e case o f u n i f o r m d a t a ( a t least for m o s t p r a c t i c a l p u r p o s e s ) , t h i s l i m i t a t i o n n o
l o n g e r exists for t h e r e s i d u a l i t e r a t i o n a l g o r i t h m .
maxerr AMLS
RMSerr AMLS
RMSerr RBF
uj
0 2000 4000 6000 8000 10000

# Iterations
Fig. 31.2 Convergence and final fit for iterated approximate M L S approximation based on 289
Halton points and Gaussians with T> = 256 (e = 1).
31.5 Iterated Shepard Approximation
T h e use o f o t h e r a p p r o x i m a t i o n m e t h o d s w i t h i n t h e f i x e d l e v e l r e s i d u a l i t e r a t i o n
a l g o r i t h m s u c h as r e g u l a r M L S a p p r o x i m a t i o n o r R B F least squares a p p r o x i m a t i o n
is also possible. W e p o i n t o u t , h o w e v e r , t h a t t h e use o f R B F i n t e r p o l a t i o n does n o t
m a k e a n y sense i n t h e c o n t e x t o f f i x e d l e v e l r e s i d u a l i t e r a t i o n since t h e r e s i d u a l s o n
t h e d a t a are a u t o m a t i c a l l y zero i f w e p e r f o r m i n t e r p o l a t i o n o f t h e d a t a .
I f w e w a n t t o use r e g u l a r M L S a p p r o x i m a t i o n i n s t e a d of approximate MLS
a p p r o x i m a t i o n i n t h e residual i t e r a t i o n , t h e n , for Shepard's m e t h o d , t h i s means
r e p l a c i n g l i n e 18 i n P r o g r a m 3 1 . 1 b y
18a EM = rbf(ep,DM);
18b EM = EM./repmat(EM*ones(N,1),1,N); % Shepard n o r m a l i z a t i o n
a n d l i n e 20 b y
20a RM = rbf(ep.DM);
20b RM = RM./repmat(RM*ones(N,1),1,N); % Shepard n o r m a l i z a t i o n
N o t e t h a t we can n o w no longer c l a i m t h a t t h e l i m i t of t h e i t e r a t e d Shepard

a p p r o x i m a n t is g i v e n b y t h e R B F i n t e r p o l a n t b a s e d o n t h e S h e p a r d w e i g h t s as basis
f u n c t i o n s . I n fact, t h e i t e r a t e d S h e p a r d a p p r o x i m a n t b e c o m e s m o r e a c c u r a t e t h a n
the R B F i n t e r p o l a n t . For example, i f we take Gaussian weight functions w i t h e = 6
or e = 1 as above, t h e n t h e c o r r e s p o n d i n g c o n v e r g e n c e b e h a v i o r for t h e iterated
S h e p a r d a p p r o x i m a n t is d i s p l a y e d i n F i g u r e 3 1 . 3 . M o r e o v e r , f o r t h e e x a m p l e with
s = 6, 45 i t e r a t i o n s o f t h e S h e p a r d a p p r o x i m a n t r e s u l t i n a s m a l l e r m a x i m u m e r r o r
t h a n t h e R B F i n t e r p o l a n t , w h i l e 3515 i t e r a t i o n s are r e q u i r e d t o p u s h t h e RMS
e r r o r for t h e S h e p a r d i t e r a t i o n b e l o w 1.074443e-003. A f t e r 10000 i t e r a t i o n s w i t h
S h e p a r d a p p r o x i m a n t s t h e m a x i m u m e r r o r is 8.760946e-003 a n d t h e R M S e r r o r is
7.889609e-004. F o r t h e e = 1 e x a m p l e t h e final e r r o r s (after 10000 i t e r a t i o n s ) are
2.664303e-001 for t h e m a x i m u m e r r o r a n d 8.169080e-002 for t h e R M S e r r o r . T h i s
e x a m p l e is v e r y s i m i l a r t o t h e A M L S e x a m p l e d i s p l a y e d i n F i g u r e 31.2. Again,
the most significant p a r t of the error i m p r o v e m e n t occurs d u r i n g the first 10-20
iterations.
10 10'
maxerr IterShep - maxerr IterShep
RMSerr IterShep - RMSerr IterShep
maxerr RBF
10 RMSerr RBF RMSerr RBF
10
2i
LU
^
10"
10
10 10
2000 4000 6000 8000 10000 2000 4000 6000 8000 10000
# Iterations # Iterations
Fig. 31.3 Convergence for iterated Shepard approximation based on 289 Halton points and Gaus-
sian weights with e = 6 (left), and = 1 (right).
F o r o t h e r d a t a sets a n d o t h e r values o f e r e s i d u a l i t e r a t i o n m a y converge faster.

F o r e x a m p l e , i n F i g u r e 31.4 we used 1089 d a t a p o i n t s ( a g a i n t a k e n f r o m F r a n k e ' s
f u n c t i o n ) a n d a value o f e = 16. O n t h e left w e s h o w t h e convergence b e h a v i o r for
i t e r a t e d a p p r o x i m a t e M L S w i t h G a u s s i a n g e n e r a t i n g f u n c t i o n s , a n d o n t h e r i g h t for
iterated Shepard a p p r o x i m a t i o n w i t h Gaussian weights. B o t h graphs contain t h e
e r r o r s for R B F i n t e r p o l a t i o n w i t h G a u s s i a n basis f u n c t i o n s for c o m p a r i s o n . W e n o t e
t h a t t h e a p p r o x i m a t e M L S i t e r a t i o n a p p r o a c h e s t h e R B F i n t e r p o l a n t faster t h a n i n
the previous examples. M o r e o v e r , b o t h e r r o r s for i t e r a t e d S h e p a r d a p p r o x i m a t i o n
are s m a l l e r t h a n those for t h e i n t e r p o l a n t after o n l y t h r e e i t e r a t i o n s . I n fact, after 10
i t e r a t i o n s t h e m a x i m u m e r r o r for t h e i t e r a t e d S h e p a r d a p p r o x i m a t i o n is one o r d e r
o f m a g n i t u d e s m a l l e r t h a n t h a t for t h e R B F i n t e r p o l a n t .
A d e t a i l e d s t u d y o f t h e d e p e n d e n c e o f t h e convergence o f t h e fixed-level residual
i t e r a t i o n a l g o r i t h m o n t h e shape p a r a m e t e r e a n d m o r e e x a m p l e s are p r o v i d e d i n
[Fasshauer a n d Z h a n g ( 2 0 0 6 ) ] .
10" 10
maxerr IterShep
RMSerr IterShep
maxerr RBF
RMSerr RBF
10 10"
maxerr IterShep
RMSerr IterShep
maxerr RBF
RMSerr RBF
10" 10
10" 10
2000 4000 6000 8000 10000 4 6 10
# Iterations # Iterations
Fig. 31.4 Convergence for iterated approximate M L S approximation (left) and Shepard approx-
imation (right) based on 1089 Halton points and Gaussian weights with e = 16.
Chapter 32
Multilevel Iteration
A s we saw i n C h a p t e r s 12 a n d 16 t h a t t h e r e is a t r a d e - o f f p r i n c i p l e for i n t e r p o -
lation w i t h compactly supported radial functions. Namely, using a non-stationary
a p p r o a c h we c a n o b t a i n g o o d a p p r o x i m a t i o n results a t t h e cost o f i n c r e a s i n g c o m p u -
t a t i o n a l c o m p l e x i t y , w h i l e w i t h a s t a t i o n a r y a p p r o a c h one has a n efficient a p p r o x -
imation method. However, stationary a p p r o x i m a t i o n w i t h c o m p a c t l y supported
r a d i a l f u n c t i o n s is s a t u r a t e d , i.e., i t p r o v i d e s o n l y l i m i t e d convergence.
32.1 Stationary Multilevel Interpolation
I n order t o c o m b i n e t h e advantages o f b o t h approaches for i n t e r p o l a t i o n w i t h c o m -

p a c t l y s u p p o r t e d r a d i a l f u n c t i o n s d e s c r i b e d above, Schaback suggested t h e use o f a
multilevel s t a t i o n a r y scheme. T h i s scheme was i m p l e m e n t e d first i n [ F l o a t e r a n d Iske
(1996b)] a n d l a t e r s t u d i e d b y a n u m b e r o f o t h e r researchers (see, e.g., [ C h e n et al.
(2002); Fasshauer a n d J e r o m e ( 1 9 9 9 ) ; Hales a n d Levesley (2002); H a r t m a n n ( 1 9 9 8 ) ;
Iske (2001); N a r c o w i c h et al. (1999); W e n d l a n d ( 1 9 9 9 a ) ] ) .
I n c o n t r a s t t o t h e fixed level i t e r a t i o n o f t h e p r e v i o u s c h a p t e r w e w i l l n o w use a
nested sequence X \ C C XK = X C M s
o f p o i n t sets w i t h i n c r e a s i n g l y g r e a t e r
d a t a density, i.e., s m a l l e r fill d i s t a n c e hx,ci- T h e basic idea o f t h e s t a t i o n a r y m u l t i -
level i n t e r p o l a t i o n a l g o r i t h m is t o scale t h e size o f t h e s u p p o r t o f t h e basis f u n c t i o n s
w i t h t h e fill distance, b u t t o i n t e r p o l a t e t o residuals o n p r o g r e s s i v e l y refined sets o f
centers. T h i s m e t h o d has a l l o f t h e c o m b i n e d benefits o f t h e i n t e r p o l a t i o n m e t h o d s
for c o m p a c t l y s u p p o r t e d R B F s r e f e r r e d t o earlier: i t is c o m p u t a t i o n a l l y efficient
(can be p e r f o r m e d i n O(N) o p e r a t i o n s ) , w e l l - c o n d i t i o n e d , a n d a p p e a r s t o be c o n -
vergent.
A n a l g o r i t h m for m u l t i l e v e l i n t e r p o l a t i o n is as follows:
A l g o r i t h m 32.1. Stationary multilevel interpolation

(1) C r e a t e nested p o i n t sets X \ C C XK X C M , a n d i n i t i a l i z e P/(x)s
0.
(2) For k = 1,2,..., K do
(a) Solve u(x) = f(x) Vf(x) on X . K
277
( b ) U p d a t e V {x) f <- V {x)f + u(x).
T h e r e p r e s e n t a t i o n o f t h e u p d a t e u a t s t e p A; is a r a d i a l basis f u n c t i o n e x p a n s i o n
of the form
() = E c f v ( ^ ^ )
w i t h ip a ( c o m p a c t l y s u p p o r t e d ) basic f u n c t i o n a n d i t s s u p p o r t scale p k hx ,n-

k
T h i s requires t h e s o l u t i o n o f a l i n e a r s y s t e m w h o s e size is d e t e r m i n e d b y t h e n u m b e r
of points i n Xk.
U n f o r t u n a t e l y , so far t h e r e are o n l y l i m i t e d t h e o r e t i c a l r e s u l t s c o n c e r n i n g t h e
convergence o f t h i s m u l t i l e v e l a l g o r i t h m . I n [ N a r c o w i c h et al. (1999)] t h e a u t h o r s
s h o w t h a t a r e l a t e d a l g o r i t h m ( i n w h i c h a d d i t i o n a l b o u n d a r y c o n d i t i o n s are i m -
p o s e d ) converges a t least l i n e a r l y . H a r t m a n n a n a l y z e s t h e m u l t i l e v e l a l g o r i t h m i n
his P h . D . thesis [ H a r t m a n n ( 1 9 9 8 ) ] . H e s h o w s a t least l i n e a r c o n v e r g e n c e for m u l -
t i l e v e l i n t e r p o l a t i o n o n a r e g u l a r l a t t i c e f o r v a r i o u s r a d i a l basis f u n c t i o n s . S i m i l a r
r e s u l t s are o b t a i n e d i n [Hales a n d L e v e s l e y ( 2 0 0 2 ) ] for ( g l o b a l l y s u p p o r t e d ) p o l y -
h a r m o n i c splines, i.e., t h i n p l a t e splines a n d r a d i a l p o w e r s . I n this context linear
convergence is t o be i n t e r p r e t e d as a n i m p r o v e m e n t o f t h e f o r m
f c ) f c 1 )
||/-7>} ||<C||/-P} - ||, k = l,...,K,
where d e n o t e s t h e i n t e r p o l a n t a t l e v e l k, and C < 1 is a p o s i t i v e ( l e v e l -

independent) constant.
T h e m a i n d i f f i c u l t y i n p r o v i n g t h e c o n v e r g e n c e o f t h e m u l t i l e v e l a l g o r i t h m is t h e
fact t h a t t h e a p p r o x i m a t i o n space changes f r o m one l e v e l t o t h e n e x t . T h e a p p r o x -
i m a t i o n spaces are n o t n e s t e d (as t h e y u s u a l l y are for w a v e l e t s ) . T h i s m e a n s t h a t
t h e n a t i v e space n o r m changes f r o m one l e v e l t o t h e n e x t . Hales a n d L e v e s l e y a v o i d
t h i s p r o b l e m b y s c a l i n g t h e ( u n i f o r m l y s p a c e d ) d a t a i n s t e a d o f t h e basis f u n c t i o n s .
T h e n t h e fact t h a t p o l y h a r m o n i c splines are i n a c e r t a i n sense h o m o g e n e o u s (see
S e c t i o n 34.4) s i m p l i f i e s t h e a n a l y s i s . T h i s fact w a s also used i n [ W e n d l a n d ( 2 0 0 5 a ) ]
t o p r o v e l i n e a r convergence for m u l t i l e v e l ( s c a t t e r e d d a t a ) i n t e r p o l a t i o n b a s e d o n
t h i n p l a t e splines.
A n o t h e r a p p r o a c h t o m u l t i l e v e l i n t e r p o l a t i o n was r e c e n t l y suggested b y O p f e r
(see [Opfer ( 2 0 0 4 ) ; O p f e r ( 2 0 0 6 ) ] ) . H e c o n s t r u c t s so-called multiscale kernels that
h a v e t h e i n f o r m a t i o n f r o m d i f f e r e n t r e s o l u t i o n levels b u i l t i n t o a s i n g l e f u n c t i o n .
T h e s e kernels are b u i l t as t e n s o r p r o d u c t s o f s c a l i n g f u n c t i o n s t h a t f o r m a w a v e l e t -
like m u l t i r e s o l u t i o n analysis. Opfer provides error b o u n d s for i n t e r p o l a t i o n w i t h
these kernels a n a l o g o u s t o t h o s e i n T h e o r e m 15.3. H e also d e m o n s t r a t e s h o w m u l -
tiscale kernels c a n be u s e d for s c a t t e r e d d a t a i n t e r p o l a t i o n , a n d for i m a g e d e c o m -
position and compression.
32.2 A M A T L A B I m p l e m e n t a t i o n of S t a t i o n a r y M u l t i l e v e l
Interpolation
T h e M A T L A B p r o g r a m ML_CSRBF3D. m for s t a t i o n a r y m u l t i l e v e l i n t e r p o l a t i o n w i t h
c o m p a c t l y s u p p o r t e d R B F s is d i s p l a y e d as P r o g r a m 3 2 . 1 . I n t h i s p r o g r a m w e use
2
t h e c o m p a c t l y s u p p o r t e d R B F <p(r) = ( 1 - r ) ^ . ( 3 5 r + 1 8 r + 3) o f W e n d l a n d . T h i s
4 3
f u n c t i o n is C a n d s t r i c t l y p o s i t i v e d e f i n i t e a n d r a d i a l o n 1R . T h e p r o g r a m shows
a n u m b e r o f differences c o m p a r e d t o o u r p r e v i o u s codes. O n l i n e 2 w e define t h e
m a x i m u m n u m b e r o f levels K w e w i s h t o use, a n d t h e n define t h e corresponding
scale p a r a m e t e r s e for t h e basic f u n c t i o n . Since we l o a d d a t a sets c o n s i s t i n g o f
fc 3
(2 + l ) u n i f o r m l y spaced d a t a p o i n t s i n t h e u n i t c u b e ( w h e r e k r u n s f r o m 1 t o K,
see lines 1 3 - 1 7 ) , t h e fill d i s t a n c e hx ,n k changes b y a f a c t o r o f t w o f r o m one l e v e l t o
the next. T h e r e f o r e , i n o r d e r t o g u a r a n t e e a s t a t i o n a r y i n t e r p o l a t i o n scheme, t h e
scale p a r a m e t e r e needs t o c h a n g e b y a f a c t o r o f t w o f r o m one level t o t h e n e x t ,
also. T h i s is achieved i n l i n e 2 o f t h e code w h e r e we also use a n a d d i t i o n a l f a c t o r
of 0.7 t o u n i f o r m l y scale a l l values o f e. N o t e t h a t here e c o r r e s p o n d s t o a s u p p o r t
radius p = 1/e.
I n each i t e r a t i o n we solve one i n t e r p o l a t i o n p r o b l e m (see l i n e 1 8 - 2 2 ) . T h e e x p a n -
s i o n coefficients c o e f are s t o r e d i n one c o m p o n e n t o f a M A T L A B c e l l array. S i m i l a r l y ,
w e need t o keep t h e centers for a l l levels i n m e m o r y ( a g a i n u s i n g a cell a r r a y c t r s ,
see l i n e 1 7 ) . T h e r i g h t - h a n d side for t h e i n t e r p o l a t i o n p r o b l e m is g i v e n b y t h e d a t a
(values o f t h e test f u n c t i o n o n t h e i n i t i a l d a t a set) i n t h e f i r s t i t e r a t i o n , a n d i n l a t e r
i t e r a t i o n s b y t h e r e s i d u a l , i.e., t h e difference b e t w e e n t h e d a t a (values o f t h e t e s t
f u n c t i o n o n t h e present g r i d ) a n d t h e values o f t h e fit o n t h e present g r i d . These
values are c o m p u t e d at t h e e n d o f t h e p r e v i o u s i t e r a t i o n a n d s t o r e d i n t h e v e c t o r R f
(see lines 2 8 - 3 2 ) . T h i s e x t r a e v a l u a t i o n ( i n a d d i t i o n t o t h e e v a l u a t i o n o n a s e p a r a t e
e v a l u a t i o n g r i d for e r r o r m o n i t o r i n g a n d p l o t t i n g p u r p o s e s , see lines 3 4 - 3 9 ) a d d s t o
t h e c o m p l e x i t y ( a n d i n e f f i c i e n c y ) o f t h e p r e s e n t code. Moreover, the evaluation of
t h e r e s i d u a l o n t h e present g r i d r e q u i r e s us t o keep e v a l u a t i o n m a t r i c e s for a l l levels
i n m e m o r y ( i n t h e c e l l a r r a y RM, see lines 2 4 - 2 7 ) . N o t e t h a t t h e e v a l u a t i o n p o i n t s
( r e s p o i n t s ) for t h e residuals change w i t h e v e r y i t e r a t i o n a n d need n o t b e k e p t i n
storage. A n a l t e r n a t i v e a p p r o a c h w o u l d be t o e v a l u a t e a l l residuals o n a c o m m o n
(fine) g r i d , e.g., t h e e v a l u a t i o n g r i d . T h i s , h o w e v e r , w o u l d m a k e ( t h e e v a l u a t i o n o f )
the i n i t i a l iterates rather expensive.
F i n a l l y , o n lines 4 2 - 4 8 w e keep t r a c k o f t h e R M S e r r o r , a n d c o m p u t e t h e r a t e o f
convergence f r o m one level t o t h e n e x t . T h e c o m m a n d s t h a t g e n e r a t e p l o t s o f t h e
d a t a sites, i n t e r p o l a n t a n d e r r o r are g i v e n o n lines 4 9 - 5 4 .
P r o g r a m 3 2 . 1 . ML_CSRBF3D .m
% ML_CSRBF3D
/ S c r i p t t h a t performs multilevel RBF I n t e r p o l a t i o n u s i n g
'/, s p a r s e matrices
7, C a l l s on: DistancMatrixCSRBF
7, Wendland C4
1 r b f = @(e,r) r."6.*(35*r.~2-88*r+56*spones(r));
7o Number of l e v e l s w i t h e p s i l o n s f o r s t a t i o n a r y i n t e r p o l a t i o n
2 K = 4; ep = 0.7*2.~[0:K-1];
3 t e s t f u n c t i o n = @(x,y,z) 6 4 * x . * ( 1 - x ) . * y . * ( 1 - y ) . * z . * ( 1 - z ) ;
4 g r i d t y p e = 'u' ; 7o Type of d a t a p o i n t s : u'=uniform 5
5 n e v a l = 10; M = n e v a l " 3 ;
6 grid=linspace(0,1,neval); [xe,ye,ze]=meshgrid(grid);
7 epoints= [ x e ( : ) y e ( : ) z e ( : ) ] ;
8 exact = t e s t f m i c t i o n ( e p o i n t s ( : , 1 ) , e p o i n t s ( : , 2 ) , e p o i n t s ( : , 3 ) ) ;
9 isomin = 0.1; isomax = 1; i s o s t e p = .1;
10 x s l i c e = .25:.25:1; y s l i c e = 1; z s l i c e = [ 0 , 0 . 5 ] ;
11 R f _ o l d = z e r o s ( 2 7 , l ) ; 7 i n i t i a l i z e
12 f o r k=l:K
13 Nl = ( 2 ~ k + l ) ~ 3 ; N2 = ( 2 * ( k + 1 ) + 1 ) " 3 ;
5
14 namel = s p r i n t f ( 'Data3D_7,d7.s , Nl . g r i d t y p e ) ;
15 name2 = s p r i n t f ( D a t a 3 D _ 7 . d 7 . s N 2 . g r i d t y p e ) ;
,
16 load(name2); respoints = dsites;

17 load(namel); ctrs{k} = dsites;
7o Compute r i g h t - h a n d s i d e (= r e s i d u a l )
18 Tf = t e s t f u n c t i o n ( d s i t e s ( : , 1 ) , d s i t e s ( : , 2 ) , d s i t e s ( : , 3 ) ) ;
19 r h s = Tf - R f _ o l d ;
20 DM.data = D i s t a n c e M a t r i x C S R B F ( d s i t e s , c t r s { k } , e p ( k ) ) ;
21 IM = r b f ( e p ( k ) , D M _ d a t a ) ;
7o Compute c o e f f i c i e n t s f o r RBF i n t e r p o l a n t t o d e t a i l l e v e l
22 c o e f { k } = IM\rhs;
23 i f (k < K)
7o Compute - f o r a l l l e v e l s - e v a l u a t i o n m a t r i c e s f o r
7o r e s i d u a l s d i r e c t l y
24 for j=l:k
25 DM_res = D i s t a n c e M a t r i x C S R B F ( r e s p o i n t s , c t r s { j } - , e p ( j ) ) ;
26 RM-Cj} = r b f ( e p ( j ) ,DM_res) ;
27 end
7 E v a l u a t e RBF i n t e r p o l a n t (sum of a l l p r e v i o u s f i t s
7o e v a l u a t e d on c u r r e n t g r i d )
28 Rf = z e r o s ( N 2 , l ) ;
29 for j=l:k
30 Rf = Rf + RM-Cj } * c o e f { j } ;
31 end
32 R f _ o l d = Rf;
33 end
34 DM_eval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , c t r s f k } , e p ( k ) ) ;
35 EM = r b f ( e p ( k ) , D M _ e v a l ) ;
36 Pf = EM*coef{k};
37 i f (k > 1)
38 Pf = P f _ o l d + P f ;
39 end
40 Pf_old = Pf;
42 rms_err = n o r m ( P f - e x a c t ) / s q r t ( M ) ;
43 f p r i n t f ('RMS e r r o r : %e\n', r m s . e r r )
44 i f (k > 1)
45 rms_rate = l o g ( r m s _ e r r _ o l d / r m s _ e r r ) / l o g ( 2 ) ;
46 f p r i n t f ('RMS r a t e : /,f \n' , r m s _ r a t e )
47 end
48 r m s _ e r r _ o l d = rms.err;
49 figure
50 plot3(dsites(:,1),dsites(:,2),dsites(:,3),'bo');
51a Plotlsosurf(xe,ye,ze,Pf,neval,exact,maxerr,isomin,...
51b isostep,isomax);
52 PlotSlices(xe,ye,ze,Pf,neval,xslice,yslice,zslice);
53a PlotErrorSlices(xe,ye,ze,Pf,exact,neval,...
53b xslice,yslice,zslice);
54 end
I n F i g u r e 3 2 . 1 we d i s p l a y f o u r d a t a sets used i n t h e 3 D m u l t i l e v e l e x p e r i m e n t .
T h e c o r r e s p o n d i n g 3 D m u l t i l e v e l i n t e r p o l a n t s are s h o w n as iso-surfaces in Fig-
u r e 32.2. N o t e t h a t t h e t e s t f u n c t i o n f(x,y,z) = 64x(l x)y(l y)z(l z) is
a three-dimensional " b u m p " function, a n d therefore o n l y the outermost iso-surface
( c o r r e s p o n d i n g t o t h e f u n c t i o n v a l u e 0.1) is v i s i b l e . T h e r e f o r e , we also d i s p l a y t h r e e -
d i m e n s i o n a l slice p l o t s o f t h e a b s o l u t e e r r o r i n F i g u r e 32.3. B o t h t h e iso-surfaces
a n d t h e slice p l o t s are c o l o r c o d e d a c c o r d i n g t o t h e a b s o l u t e e r r o r .
I n T a b l e 3 2 . 1 w e l i s t t h e c o r r e s p o n d i n g R M S e r r o r s a n d observed convergence
r a t e s for t h e 3 D m u l t i l e v e l e x p e r i m e n t .
Table 32.1 3D stationary multilevel interpolation with

2
<p(7-) = (1 - r)^_(35r + 18r + 3).
mesh RMS-error rate % nonzero time
3x 3x3 1.005315e-001 92.32 0.16

5x 5x5 2.764907e-002 1.8623 36.99 0.56
9x 9x9 2.626864e-003 3.3958 8.88 13.45
17 x 17 x 17 5.70606 le-004 2.2028 1.57 73.50
Fig. 32.1 Uniform point sets in the unit cube with 27, 125, 729, and 4913 points (top left to
bottom right).
N e x t we present a t w o - d i m e n s i o n a l m u l t i l e v e l i n t e r p o l a t i o n e x p e r i m e n t with
scattered data. The d a t a were obtained i n the ASCII VRML format from
http://amba.charite.de/~ksch/spsm/beetls.wrl.gz. T h e d a t a set provides
a d i g i t i z e d surface m o d e l o f a b u s t o f t h e f a m o u s G e r m a n c o m p o s e r L u d w i g v a n
Beethoven. T h e o r i g i n a l d a t a set consists o f 2663 p o i n t s i n t h e u n i t s q u a r e . In
o r d e r t o p r o v i d e a n e s t e d set o f d a t a sites for t h e m u l t i l e v e l a l g o r i t h m w e use the
p r o g r a m T h i n . m p r o v i d e d i n A p p e n d i x C . T h e processed d a t a files are i n c l u d e d o n
t h e enclosed C D . T h e r e s u l t i n g p o i n t sets are d i s p l a y e d i n F i g u r e 32.4. A much
m o r e d e t a i l e d d i s c u s s i o n o f t h i n n i n g a l g o r i t h m s for s c a t t e r e d d a t a is p r e s e n t e d i n
[Iske ( 2 0 0 4 ) ] . T h e M A T L A B p r o g r a m used t o g e n e r a t e t h e m u l t i l e v e l interpolants
i n F i g u r e 32.5 is e s s e n t i a l l y t h e same as P r o g r a m 3 2 . 1 . T h e m a i n difference is t h a t
t h e d a t a values are also r e a d f r o m t h e d a t a file i n s t e a d o f b e i n g g e n e r a t e d b y a t e s t
f u n c t i o n . F o r r e n d e r i n g p u r p o s e s t h e i n t e r p o l a n t s are e v a l u a t e d o n a n 80 x 80 g r i d
of e q u a l l y spaced p o i n t s i n t h e u n i t s q u a r e .
Fig. 32.2 Iso-surface plots for multilevel interpolants at levels 1, 2, 3 and 4 (top left to bottom
right) false -colored by absolute error .
32.3 Stationary Multilevel Approximation
T h e same basic m u l t i l e v e l a l g o r i t h m c a n also be used for o t h e r a p p r o x i m a t i o n m e t h -

ods. I n [Fasshauer (2002c)] t h e i d e a was a p p l i e d t o m o v i n g least squares m e t h o d s
a n d a p p r o x i m a t e m o v i n g least squares m e t h o d s . E x p e r i m e n t s s i m i l a r t o those o f
[Fasshauer (2002c)] are n o w r e p e a t e d here. I n o r d e r t o be able t o p r o v i d e a c o m p a r -
ison b e t w e e n t h e m u l t i l e v e l i n t e r p o l a t i o n a n d a p p r o x i m a t i o n a l g o r i t h m s w e b e g i n
w i t h one m o r e e x a m p l e for m u l t i l e v e l i n t e r p o l a t i o n .
W e o b t a i n t h e d a t a for t h e f o l l o w i n g n u m e r i c a l examples b y s a m p l i n g a m o l l i f i e d
2
F r a n k e f u n c t i o n / a t u n i f o r m l y spaced p o i n t s i n t h e u n i t square [0, l ] , i.e.,
f(x,y) = 15exp ( L ^ 1
. 1 ) 2 ) exp ( _ ~ _
1 { y
1
1 ) 2 ) F(x,y),
w h e r e F denotes F r a n k e ' s f u n c t i o n ( 2 . 2 ) .
I n T a b l e 32.2 we list t h e b e n c h m a r k results for m u l t i l e v e l R B F i n t e r p o l a t i o n
w i t h t h e c o m p a c t l y s u p p o r t e d f u n c t i o n <p3,i(r) = ( 1 r)^_ ( 4 r + 1 ) . W e a g a i n use
a n i n i t i a l scale f a c t o r o f 0.7 for t h e shape p a r a m e t e r e. Since t h e shape p a r a m e t e r e
is e q u a l t o t h e r e c i p r o c a l o f t h e s u p p o r t scale p t h i s means t h a t t h e i n i t i a l s u p p o r t
scale p i is chosen so t h a t t h e ( u n i v a r i a t e ) basic f u n c t i o n is f a i r l y w i d e . Subsequent
. i
0 0 x y 0 o
Fig. 32.3 Slice plots of absolute errors for multilevel interpolants at levels 1, 2, 3 and 4 (top left
to bottom right).
s u p p o r t scales are successively d i v i d e d b y t w o (i.e., e is m u l t i p l i e d b y t w o ) just

as t h e fill d i s t a n c e is h a l v e d o n successive c o m p u t a t i o n a l grids Xk-
Table 32.2 2D stationary multilevel interpolation with

2
<p(r) = (1 r)^_(4r + 1) at equally spaced points in [0, l ] .
mesh RMS-error rate % nonzero time
3x3 2.498505e-001 100 0.13

5x5 7.695304e-002 1.6990 57.76 0.16
9x9 2.092849e-002 1.8785 23.18 0.20
17 X 17 1.145664e-003 4.1912 7.47 0.42
33 x 33 1.376035e-004 3.0576 2.13 1.64
65 x 65 3.303559e-005 2.0584 0.57 7.98
129 x 129 2.149123e-006 3.9422 0.15 5.98
I n T a b l e 32.2 w e l i s t t h e R M S e r r o r s c o m p u t e d o n a 40 x 40 u n i f o r m e v a l u a -
t i o n g r i d along w i t h the percentage of non-zero entries i n the R B F interpolation
m a t r i x a n d t h e c o m p u t e r t i m e r e q u i r e d f o r e a c h i t e r a t i o n . T h e first f o u r m u l t i l e v e l
i n t e r p o l a n t s are s h o w n i n F i g u r e 32.6.
Fig. 32.4 Thinned data sets for Beethoven's head. For top left to bottom right: 163, 663, 1163,
1663, 2163, and 2663 points.
Fig. 32.5 Multilevel interpolants to Beethoven data. For top left to bottom right: 163, 663, 1163,
1663, 2163, and 2663 points.
N e x t we replace R B F i n t e r p o l a t i o n a t each step o f t h e m u l t i l e v e l r e s i d u a l i t -

e r a t i o n a l g o r i t h m b y s t a n d a r d m o v i n g least squares a p p r o x i m a t i o n . Table 32.3
i l l u s t r a t e s t h e p e r f o r m a n c e o f t h e m u l t i l e v e l a l g o r i t h m for S h e p a r d ' s m e t h o d a n d a
m o v i n g least squares a p p r o x i m a t i o n w i t h l i n e a r p r e c i s i o n , b o t h based o n t h e c o m -
p a c t l y s u p p o r t e d w e i g h t f u n c t i o n 923,1 ( r ) = ( 1 r ) ^ _ ( 4 r + 1 ) . T h e s u p p o r t scaling
is t h e same as i n t h e p r e v i o u s m u l t i l e v e l i n t e r p o l a t i o n e x a m p l e . T h e M A T L A B code
for these e x a m p l e s is o m i t t e d as i t is v e r y s i m i l a r t o t h a t o f P r o g r a m 3 2 . 1 . We
n o t e , however, t h a t o u r i m p l e m e n t a t i o n o f t h e l i n e a r p r e c i s i o n v a r i a n t b a s e d o n
P r o g r a m 24.3 is r a t h e r inefficient w h e n c o m p a r e d t o t h e o t h e r m u l t i l e v e l e x a m p l e s
p r e s e n t e d here.
y x y
Fig. 32.6 T h e first four interpolants from Table 32.2.
T h e r e seems t o b e n o t h e o r e t i c a l i n v e s t i g a t i o n o f the convergence properties o f

t h e m u l t i l e v e l a l g o r i t h m for m o v i n g least s q u a r e s a p p r o x i m a t i o n i n t h e literature.
Table 32.3 2D multilevel M L S approximation with <p(r) = (1 - r ) t ( 4 r + 1).
Shepard linear precision
mesh RMS-error rate time RMS-error rate time
3x3 2.776569e-001 0.16 2.812184e-001 0.91

5x5 1.615753e-001 0.7811 0.14 1.481365e-001 0.9248 0.97
9x9 7.519432e-002 1.1035 0.19 7.015497e-002 1.0783 1.36
17 x 17 1.858696e-002 2.0163 0.39 1.932368e-002 1.8602 3.09
33 x 33 3.581720e-003 2.3756 1.56 2.639418e-003 2.8721 11.45
65 x 65 5.458943e-004 2.7140 7.67 3.426412e-004 2.9454 89.64
129 x 129 1.047351e-004 2.3819 0.36 3.936786e-005 3.1216 1.17
A s a t h i r d p a r t o f t h i s e x a m p l e w e use approximate M L S approximation at

each level of t h e residual iteration algorithm. We use the generating functions
^(r) = ^(1 r)+(4r + 1 ) ( g i v i n g rise t o a n a p p r o x i m a t e p a r t i t i o n o f u n i t y ) and
^( )
R =
MUtT(1
T*)+(4r + 1 ) ( 1 4 5 5 r ) ( g i v i n g r i s e t o a n a p p r o x i m a t e p a r t i t i o n o f
2
u n i t y w i t h one v a n i s h i n g m o m e n t ) . R e c a l l t h a t w e c o n s t r u c t e d these f u n c t i o n s i n
E x a m p l e 26.2 s t a r t i n g w i t h t h e i n i t i a l w e i g h t f u n c t i o n ifto(y) = ( l ^/y) + (4-y/y+l).
A s scale p a r a m e t e r w e t a k e T> = 4 0 0 / 4 9 . T h i s c o r r e s p o n d s t o t h e same s c a l i n g as
i n t h e o t h e r e x a m p l e s since e l/(y/T)h) a n d t h e i n i t i a l fill d i s t a n c e for t h e 3 x 3
g r i d is h = 1/2.
Table 32.4 Multilevel approximate M L S approximation with basic function

ifo(y) = (1 - y/y)%(4^y + 1) and V = 400/49.
basic method 1 vanishing moment
mesh RMS-error rate time RMS-error rate time
3x3 2.803247e-001 0.13 2.630763e-001 0.13

5x5 1.578062e-001 0.8289 0.16 9.133669e-002 1.5262 0.16
9x9 7.483597e-002 1.0764 0.20 2.783120e-002 1.7145 0.22
17 x 17 1.784522e-002 2.0682 0.39 3.399671e-003 3.0332 0.45
33 x 33 2.468958e-003 2.8536 1.52 4.359882e-004 2.9630 1.81
65 x 65 3.637815e-004 2.7628 7.30 7.856778e-005 2.4723 9.05
129 x 129 5.636161e-005 2.6903 0.39 2.460906e-005 1.6747 0.38
I f we c o m p a r e t h e n u m b e r s f r o m t h e t h r e e d i f f e r e n t a p p r o a c h e s l i s t e d i n T a -
bles 3 2 . 2 - 3 2 . 4 w e see t h a t n o n e o f t h e a p p r o x i m a t i o n m e t h o d s y i e l d m o r e a c c u r a t e
results t h a n t h e i n t e r p o l a t i o n m e t h o d . I t is s u r p r i s i n g , however, t h a t t h e a p p r o x i -
m a t e M L S m e t h o d s p e r f o r m b e t t e r t h a n t h e r e g u l a r M L S m e t h o d s . For n o i s y d a t a
t h e a p p r o x i m a t e M L S m e t h o d w o u l d be p r e f e r a b l e .
32.4 Multilevel Interpolation with Globally Supported RBFs
So far w e have c o n c e n t r a t e d o n t h e use o f c o m p a c t l y s u p p o r t e d f u n c t i o n s w i t h i n t h e

m u l t i l e v e l r e s i d u a l i t e r a t i o n a l g o r i t h m . F o r g l o b a l l y s u p p o r t e d f u n c t i o n s we l e a r n e d
i n earlier c h a p t e r s t h a t w e c a n o b t a i n g o o d a p p r o x i m a t i o n o r d e r e s t i m a t e s i n t h e
n o n - s t a t i o n a r y s e t t i n g . O n e reason for o u r focus o n c o m p a c t l y s u p p o r t e d f u n c t i o n s
i n t h i s c h a p t e r is t h a t i f w e consider t h e use o f g l o b a l l y s u p p o r t e d f u n c t i o n s i n
a n o n - s t a t i o n a r y m u l t i l e v e l i n t e r p o l a t i o n f r a m e w o r k , t h e n we see t h a t n o t h i n g is
g a i n e d b y t h e m u l t i l e v e l a p p r o a c h ( p r o v i d e d w e l i m i t ourselves t o t h e s o l u t i o n o f
l i n e a r p r o b l e m s such as t h e i n t e r p o l a t i o n p r o b l e m s discussed a b o v e ) .
M o r e precisely, i f t h e meshes Xk a t t h e d i f f e r e n t levels are nested a n d t h e shape
p a r a m e t e r e i n t h e g l o b a l l y s u p p o r t e d f u n c t i o n s is k e p t fixed t h r o u g h a l l levels k,
t h e n t h e f u n c t i o n spaces Sk = span{<^(|| x ^ U ) : x ^ G Xk} are also nested.
Consequently, t h e richest space S = (JfcLi $k, w h i c h is used w h e n a l l u p d a t e s have
been p e r f o r m e d a t t h e finest level, is e q u a l t o t h e space S K o n t h e finest m e s h XK-
T h u s , a d i r e c t fit a t t h e finest level K uses t h e same a p p r o x i m a t i o n space, a n d
w i l l t h e r e f o r e y i e l d t h e same q u a l i t y o f fit, as t h e m u l t i l e v e l a l g o r i t h m u s i n g a l l o f
t h e meshes Xk, k 1,...,K. H o w e v e r , t h e m u l t i l e v e l a l g o r i t h m requires a l l t h e
a d d i t i o n a l (unnecessary) i n t e r m e d i a t e w o r k o n t h e coarser meshes.

I t therefore follows
T h e o r e m 3 2 . 1 ( R u l e 1 ) . Consider a linear problem of the form Lu = f on

S
Cl C I R , let X\,..., XK be a nested sequence of point sets in Q., and let <p be a
globally supported RBF with fixed shape parameter e for all k = 1 , . . . , K. Then the
approximate solution UK obtained by the multilevel interpolation algorithm is the
same as the solution of the problem Lu = / on the fixed level XK using the space
S K , i-&-, the use of globally supported RBFs with fixed value of e within a multilevel
residual iteration algorithm for linear problems is pointless.
H e r e L c o u l d even be a l i n e a r d i f f e r e n t i a l o p e r a t o r (as used i n l a t e r c h a p t e r s ) .

H o w e v e r , for t h e present discussion o f i n t e r p o l a t i o n p r o b l e m s we are o n l y i n t e r e s t e d
in L = I .
I f one varies t h e p a r a m e t e r e w i t h t h e levels k (i.e., one d e p a r t s f r o m t h e n o n -
s t a t i o n a r y r e g i m e ) t h e n t h e f u n c t i o n space S (JA-LI $k used for t h e f i n a l f i t w i t h
a m u l t i l e v e l a l g o r i t h m w i l l be r i c h e r t h a n t h e space S K used d i r e c t l y for t h e finest
level XK alone. T h i s is clear since t h e spaces Sk, k = 1 , . . . , K, are n o l o n g e r nested.
T h i s i m p l i e s t h a t , for a " g o o d " sequence o f - v a l u e s , one c a n e x p e c t t o o b t a i n m o r e
a c c u r a t e fits u s i n g t h e m u l t i l e v e l f r a m e w o r k .
T h i s is s u m m a r i z e d i n
C o r o l l a r y 32.1 ( R u l e 2 ) . The multilevel residual iteration algorithm for linear

problems has the potential of being more accurate than a direct fit if the parameter
e is varied with the levels.
2
W e now illustrate Rules 1 and 2 w i t h a scattered d a t a f i t t i n g p r o b l e m i n K .
W e use F r a n k e ' s f u n c t i o n (2.2) o n t h e u n i t s q u a r e as o u r t e s t f u n c t i o n . W e t a k e t h e
2
( r a d i a l ) basic f u n c t i o n t o be a m u l t i q u a d r i c <p(r) = y/l + (er) . T h e p o i n t sets Xk
fc 2
are g i v e n b y ( 2 -4- l ) e q u a l l y spaced p o i n t s i n t h e u n i t s q u a r e a n d are therefore
f e
nested w i t h f i l l - d i s t a n c e hx ,a. k = l/2 .
I n a l l o f o u r n u m e r i c a l e x a m p l e s w e l i s t R M S - e r r o r s c a l c u l a t e d o n a fine e v a l u -
a t i o n m e s h o f 40 x 40 u n i f o r m l y spaced p o i n t s i n t h e u n i t s q u a r e .
F o r t h e first e x a m p l e we f i x t h e m u l t i q u a d r i c s h a p e p a r a m e t e r at e 10/3
throughout. T h e e r r o r s a n d r a t e s i n T a b l e 32.5 i n d i c a t e t h e w e l l - k n o w n s p e c t r a l
convergence b e h a v i o r o f m u l t i q u a d r i c s . T h e last r o w i n t h e t a b l e also shows t h a t t h e
p a r a m e t e r e is t o o s m a l l for t h i s p o i n t set a n d t h e m a t r i x is so i l l - c o n d i t i o n e d t h a t
t h e a p p r o x i m a t i o n is s t a r t i n g t o be c o n t a m i n a t e d b y r o u n d o f f e r r o r s . A c c o r d i n g t o
R u l e 1 t h e r e is no difference b e t w e e n u s i n g t h e m u l t i l e v e l a l g o r i t h m a n d a d i r e c t f i t
o n TV p o i n t s . T h i s c a n also be o b s e r v e d n u m e r i c a l l y .
I n order t o illustrate Rule 2 we repeat t h e above example, b u t n o w t a k e e =
V~N/2 (essentially a s t a t i o n a r y a p p r o a c h ) . T h i s t i m e we list w h a t happens w i t h
the multilevel a l g o r i t h m a n d compare this t o the results obtained b y c o m p u t i n g the
Table 32.5 M Q fit to Franke's func-

tion with fixed value of e = 10/3.
mesh RMS-error rate
3 x3 1.802052e-001
5 x5 2.807009e-002 2.6825
9 x9 4.009608e-003 2.8075
17 x 17 3.885488e-005 6.6892
33 x 33 2.294910e-008 10.7254
65 x 65 1.511150e-008 0.6028
a p p r o x i m a t i o n d i r e c t l y i n one s t e p o n t h e sets X , k k = 1 , . . . , 6 (c.f. T a b l e 3 2 . 6 ) .
Table 32.6 Multilevel M Q and direct fits to Franke's function with

variable value of e = y/N/2.
multilevel direct
mesh RMS-error rate RMS-error rate
3x3 .667 1.847122e-001 1.847122e-001

5x5 .400 3.128037e-002 2.5619 3.105411e-002 2.5724
9x9 .222 4.288046e-003 2.8669 4.036275e-003 2.9437
17 x 17 .118 1.598555e-004 4.7455 1.004206e-004 5.3289
33 x 33 .061 1.187541e-005 3.7507 1.919679e-005 2.3871
65 x 65 .031 1.310485e-006 3.1798 4.700632e-006 2.0299
N o t e t h a t , w i t h t h e v a r y i n g p a r a m e t e r e, u p t o 289 p o i n t s t h e d i r e c t a p p r o a c h is
m o r e accurate, b u t t h e n t h e m u l t i l e v e l a p p r o a c h does a b e t t e r j o b . T h i s is d u e t o t h e
fact t h a t t h e m a t r i c e s for t h e denser p o i n t sets b e c o m e i n c r e a s i n g l y i l l - c o n d i t i o n e d
(even w i t h t h e a d j u s t e d e-value) a n d t h e r e f o r e t h e d i r e c t fits w i t h 1089 o r 4225
p o i n t s are l i k e l y t o be i n a c c u r a t e . W i t h t h e m u l t i l e v e l a l g o r i t h m t h e fits o n t h e
finer g r i d s act o n l y as " c o r r e c t i o n s " t o t h e coarse g r i d fits c o m p u t e d e a r l i e r . T h i s
agrees w i t h t h e p h i l o s o p h y o f t h e m u l t i l e v e l a l g o r i t h m , a n d here is w h e r e t h e r i c h e r
f u n c t i o n space p a y s off.
F i n a l l y , i t is also possible t o c o m b i n e t h e fixed level i t e r a t i o n o f t h e p r e v i o u s
c h a p t e r w i t h t h e m u l t i l e v e l i t e r a t i o n o f t h i s c h a p t e r , i.e., we c a n replace t h e i n t e r -
p o l a t i o n steps i n t h e m u l t i l e v e l scheme b y f i x e d level i t e r a t i o n o f a n a p p r o p r i a t e
a p p r o x i m a t i o n m e t h o d . W e d o n o t r e p o r t a n y s u c h e x p e r i m e n t s here.
Chapter 33
Adaptive Iteration
T h e t w o a d a p t i v e a l g o r i t h m s discussed i n t h i s c h a p t e r were b o t h c o n c e i v e d t o y i e l d
an approximate solution to the R B F interpolation problem. H o w e v e r , t h e y have
some s i m i l a r i t y w i t h t h e least squares k n o t i n s e r t i o n a l g o r i t h m o f C h a p t e r 21 as
w e l l as w i t h t h e i t e r a t i v e a l g o r i t h m s o f t h e p r e v i o u s t w o c h a p t e r s . T h e contents
of t h i s c h a p t e r are based m o s t l y o n t h e p a p e r s [ F a u l a n d P o w e l l ( 1 9 9 9 ) ; F a u l a n d
P o w e l l (2000); Schaback a n d W e n d l a n d (2000a); Schaback a n d W e n d l a n d ( 2 0 0 0 b ) ]
a n d t h e b o o k [ W e n d l a n d (2005a)].
33.1 A Greedy Adaptive Algorithm
W e c o n c e n t r a t e o n systems for s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s ( v a r i a t i o n s for

s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s also e x i s t ) . O n e o f t h e c e n t r a l i n -
gredients ( a n d m a i n differences t o t h e p r e v i o u s i t e r a t i v e a l g o r i t h m s ) is t h e use o f
t h e n a t i v e space i n n e r p r o d u c t discussed i n C h a p t e r 13. A s a l w a y s , we assume t h a t
o u r d a t a sites are X {xi,... ,xj\r}, b u t n o w we also consider a second set y C X.
I f we let Vf be t h e i n t e r p o l a n t t o / o n y C X, t h e n ( / - Vf, Vf)^(n) = 0 by
L e m m a 18.1 ( w i t h u = f) a n d we o b t a i n t h e energy s p l i t (see C o r o l l a r y 18.1)
II/HA^CO) = 11/ _
^ / + WPf HAT*^)-
O n e possible p o i n t o f v i e w is n o w t o consider a n i t e r a t i o n o n residuals. To

t h i s end we p r e t e n d t o s t a r t w i t h o u r desired i n t e r p o l a n t TQ = V* on the entire
set X, a n d a n a p p r o p r i a t e sequence o f sets 34) k = 0 , 1 , . . . (we w i l l discuss some
possible heuristics for choosing these sets l a t e r ) . T h e n , j u s t as i n o u r earlier r e s i d u a l
i t e r a t i o n s , we i t e r a t i v e l y define
r +i
k = r k - V % , fc = 0,l,.... (33.1)
N o w , t h e energy s p l i t t i n g i d e n t i t y w i t h / = r k gives us
IMlkcn) = IK - PÎIAW) + ll^ll^n) (33.2)

or, u s i n g t h e i t e r a t i o n f o r m u l a ( 3 3 . 1 ) ,
H fcllAf*(n) = W k+i\\lr^(Q)
r r
+ Ikfc - r f c + i l l ^ n ) - (33.3)
291
T h e r e f o r e , u s i n g (33.1) a n d ( 3 3 . 3 ) , w e have t h e f o l l o w i n g t e l e s c o p i n g s u m for t h e

p a r t i a l s u m s o f t h e n o r m o f t h e r e s i d u a l u p d a t e s V~?*
K K
fc=0 k=0
k=0
K
k=0
= IMIA/WW
_
W K+i\\%-^(n)
R
^ ll o|lA^()-
r
T h i s e s t i m a t e shows t h a t t h e sequence o f p a r t i a l s u m s is m o n o t o n e i n c r e a s i n g a n d
b o u n d e d , a n d t h e r e f o r e c o n v e r g e n t even for a p o o r choice o f t h e sets 3 4 - If
w e c a n show t h a t t h e residuals r k converge t o zero, t h e n w e w o u l d have t h a t the
iteratively computed approximation
K K
V k
u +i
K = J2 r k = ^ 2 ( r k - r k + 1 ) = r 0 - r K + 1 (33.4)
k=0 k=0
converges t o t h e o r i g i n a l i n t e r p o l a n t rrj = V*.
W h i l e t h i s r e s i d u a l i t e r a t i o n a l g o r i t h m has some s t r u c t u r a l s i m i l a r i t i e s w i t h t h e
f i x e d level a l g o r i t h m o f C h a p t e r 3 1 w e n o w are c o n s i d e r i n g a w a y t o efficiently
c o m p u t e t h e i n t e r p o l a n t Vf o n some fine set X b y u s i n g a n i t e r a t i o n o n subsets o f
t h e d a t a . E a r l i e r we a p p r o x i m a t e d t h e i n t e r p o l a n t b y i t e r a t i n g a n approximation
method o n t h e f u l l d a t a set, w h e r e a s n o w w e are a p p r o x i m a t i n g t h e i n t e r p o l a n t b y
i t e r a t i n g a n interpolation method o n n e s t e d ( i n c r e a s i n g ) a d a p t i v e l y chosen subsets
of the data.
T h e present m e t h o d also has some s i m i l a r i t i e s w i t h t h e m u l t i l e v e l a l g o r i t h m s
o f C h a p t e r 32. H o w e v e r , n o w w e are i n t e r e s t e d i n c o m p u t i n g t h e i n t e r p o l a n t Vf
o n t h e set X based o n a single f u n c t i o n <&, whereas earlier, o u r f i n a l i n t e r p o l a n t
was t h e r e s u l t o f u s i n g t h e spaces was a n a p p r o p r i a t e l y
scaled v e r s i o n o f t h e basic f u n c t i o n . M o r e o v e r , t h e g o a l i n C h a p t e r 32 was t o
approximate / , not Vf.
I n o r d e r t o p r o v e convergence o f t h e r e s i d u a l i t e r a t i o n , let us assume t h a t w e
c a n f i n d sets o f p o i n t s 34 such t h a t a t step k a t least some f i x e d p e r c e n t a g e o f t h e
energy o f t h e r e s i d u a l is p i c k e d u p b y i t s i n t e r p o l a n t , i.e.,
Il^ll^(n)>7l|r ||5r, f c ( n ) (33.5)
w i t h some fixed 7 e ( 0 , 1 ] . T h e n (33.3) a n d t h e i t e r a t i o n f o r m u l a (33.1) i m p l y
and therefore
r
\\ k+i\\lr*{n) ^ I M I A M O )
r
- l\\ k\\x*{n) = (l-7)lk*llAV(n)-
A p p l y i n g this b o u n d recursively yields

T h e o r e m 3 3 . 1 . If the choice of sets y k satisfies (33.5), then the residual iteration

(33.4) converges linearly in the native space norm, and after K steps of iterative
refinement there is an error bound
r l K
\\r 0 - UKr||JV(fj) = W K\\lf*(n) < ( ~ l) IMIAV*^)-
T h i s t h e o r e m has v a r i o u s l i m i t a t i o n s . I n p a r t i c u l a r , t h e n o r m involves t h e func-

t i o n $ w h i c h makes i t d i f f i c u l t t o f i n d sets y k t h a t satisfy ( 3 3 . 5 ) . M o r e o v e r , the
n a t i v e space n o r m o f t h e i n i t i a l r e s i d u a l r n is n o t k n o w n , e i t h e r . Therefore, using
a n equivalent discrete n o r m o n t h e set X, Schaback and W e n d l a n d establish an
estimate of the form
K/2
\r -u )\%
0 K ( l - 5 ^ ) \\ro\\%,
w h e r e c a n d C are c o n s t a n t s d e n o t i n g t h e n o r m equivalence, i.e.,
c\\u\\ x < H U H A ^ ) < C||u||*
for any u G A / $ ( f 2 ) , a n d w h e r e S is a c o n s t a n t a n a l o g o u s t o 7 ( b u t based o n use o f

t h e discrete n o r m || | | i n ( 3 3 . 5 ) ) . I n fact, a n y d i s c r e t e l v n o r m o n X c a n be used.
I n t h e i m p l e m e n t a t i o n b e l o w we w i l l use t h e m a x i m u m n o r m .
I n [Schaback a n d W e n d l a n d (2000b)] a basic v e r s i o n o f t h i s a l g o r i t h m w h e r e
t h e sets y k consist o f a single p o i n t is d e s c r i b e d and tested. The resulting
a p p r o x i m a t i o n y i e l d s t h e best K-term approximation t o t h e i n t e r p o l a n t . T h i s idea
is r e l a t e d t o t h e c o n c e p t o f greedy approximation algorithms (see, e.g., [Temlyakov
(1998)]) a n d sparse approximation (see, e.g., [ G i r o s i ( 1 9 9 8 ) ] ) .
f c
I f t h e set y k consists o f o n l y a single p o i n t y , k then the p a r t i a l interpolant P^
is p a r t i c u l a r l y s i m p l e , n a m e l y
=/?*(-, y) k
with
(yk,yk)'
T h i s follows i m m e d i a t e l y f r o m t h e u s u a l R B F e x p a n s i o n ( w h i c h consists o f o n l y one
t e r m here) a n d t h e i n t e r p o l a t i o n c o n d i t i o n V^(y ) k r (y ).
k k
The point y k is p i c k e d t o be t h e p o i n t i n X w h e r e t h e r e s i d u a l is largest, i.e.,

\r (y )\
k k = I j r-fc j j 00 - T h i s choice o f "set" y k c e r t a i n l y satisfies t h e c o n s t r a i n t (33.5)
since <E> is s t r i c t l y p o s i t i v e d e f i n i t e a n d therefore has i t s m a x i m u m at t h e o r i g i n (cf.
P r o p e r t y (4) i n T h e o r e m 3.1). M o r e o v e r , t h e i n t e r p o l a t i o n p r o b l e m is ( a p p r o x i -
m a t e l y ) solved w i t h o u t h a v i n g t o i n v e r t a n y l i n e a r systems. T h e a l g o r i t h m c a n be
summarized as
A l g o r i t h m 3 3 . 1 . Greedy one-point a l g o r i t h m
I n p u t d a t a l o c a t i o n s X, associated values o f / , t o l e r a n c e t o l > 0

Set i n i t i a l r e s i d u a l rn = V*, i n i t i a l i z e UQ 0, e = oo, k = 0
Choose s t a r t i n g p o i n t y k X
W h i l e e > t o l do
For 1 < i < N do
r f c + i ( a ? i ) = r {xi)
k - P(xi,y ) k
u +i(xi)
k = u {xi) k + p$(xi,y ) k
end
F i n d e = m a x | r A ; + i | a n d t h e p o i n t y +i k w h e r e i t occurs
Increment k = k + 1
end
A M A T L A B i m p l e m e n t a t i o n o f t h e g r e e d y o n e - p o i n t a l g o r i t h m is presented as
Program 33.1. T h e i m p l e m e n t a t i o n is q u i t e s t r a i g h t f o r w a r d u s i n g o u r function
D i s t a n c e M a t r i x i n conjunction w i t h the anonymous function r b f to compute b o t h
^{yk,Vk) ( o n lines 18 a n d 19) a n d <&(xi, y ) k needed for t h e u p d a t e s o f t h e r e s i d u a l
r +i
k a n d t h e a p p r o x i m a t i o n Uk+i o n lines 2 1 - 2 4 . The algorithm demands that
we c o m p u t e t h e residuals r k o n t h e d a t a sites. O n the other hand, the p a r t i a l
a p p r o x i m a n t s Uk t o t h e i n t e r p o l a n t c a n b e e v a l u a t e d a n y w h e r e . I f w e choose t o
d o t h i s also at t h e d a t a sites, t h e n we are r e q u i r e d t o use a p l o t t i n g r o u t i n e t h a t
differs f r o m o u r u s u a l one (such as t r i s u r f built on a triangulation of the data
sites o b t a i n e d w i t h t h e h e l p o f d e l a u n a y n ) . W e i n s t e a d choose t o f o l l o w t h e same
p r o c e d u r e as i n a l l o f o u r o t h e r p r o g r a m s , i.e., t o e v a l u a t e Uk o n a 40 x 40 g r i d o f
e q u a l l y spaced p o i n t s . T h i s has been i m p l e m e n t e d o n lines 2 1 - 2 5 o f t h e p r o g r a m .
N o t e t h a t t h e u p d a t i n g p r o c e d u r e has b e e n v e c t o r i z e d i n M A T L A B a l l o w i n g us t o
a v o i d t h e f o r - l o o p over i i n t h e a l g o r i t h m .
I t is i m p o r t a n t t o realize t h a t we never a c t u a l l y c o m p u t e t h e i n i t i a l r e s i d u a l
rn = Vf. A l l we r e q u i r e are t h e values o f rn o n t h e g r i d X o f d a t a sites. H o w e v e r ,
since Vf\x f\x t h e values ro(xi) are g i v e n b y t h e i n t e r p o l a t i o n d a t a f(xi) (see
line 13 o f t h e c o d e ) . M o r e o v e r , since t h e sets y k are subsets o f X t h e v a l u e r (y )
k k
r e q u i r e d t o d e t e r m i n e (3 is a c t u a l l y one o f t h e c u r r e n t r e s i d u a l values (see line 20 o f

t h e code).
T h e f i n a l a p p r o x i m a t i o n t o t h e i n t e r p o l a n t a n d t h e a p p r o x i m a t i o n e r r o r are
p l o t t e d w i t h t h e c o m m a n d s g i v e n o n lines 3 6 - 3 8 . T h e c o m m a n d s for t h e p l o t s o f
the points y k selected b y t h e a l g o r i t h m a n d t h e n o r m o f t h e r e s i d u a l d i s p l a y e d i n
F i g u r e s 33.1 a n d 33.3 are i n c l u d e d o n lines 39 a n d 40.
P r o g r a m 3 3 . 1 . R B F G r e e d y 0 n e P o i n t 2 D .m
7, R B F G r e e d y 0 n e P o i n t 2 D
/o S c r i p t t h a t p e r f o r m s g r e e d y one p o i n t a l g o r i t h m f o r adaptive
% 2D RBF i n t e r p o l a t i o n
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; 7. Gaussian RBF

2 ep = 5.5; '/ Parameter f o r b a s i s f u n c t i o n
7 Define Franke's f u n c t i o n as t e s t f u n c t i o n
3 f l = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x - 2 ) . ~ 2 + ( 9 * y - 2 ) . ~ 2 ) / 4 ) ;
4 f 2 = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x + l ) . ~ 2 / 4 9 + ( 9 * y + l ) . ~ 2 / 1 0 ) ) ;
5 f 3 = @(x,y) 0 . 5 * e x p ( - ( ( 9 * x - 7 ) . ~ 2 + ( 9 * y - 3 ) . ~ 2 ) / 4 ) ;
6 f 4 = @(x,y) 0 . 2 * e x p ( - ( ( 9 * x - 4 ) . ~ 2 + ( 9 * y - 7 ) . " 2 ) ) ;
7 t e s t f u n c t i o n = @(x,y) f l ( x , y ) + f 2 ( x , y ) + f 3 ( x , y ) - f 4 ( x , y ) ;
V, Number and type of d a t a p o i n t s
8 N = 16641; g r i d t y p e = 'h';
9 n e v a l = 40; g r i d = l i n s p a c e ( 0 , 1 , n e v a l ) ;
7. T o l e r a n c e ; stopping c r i t e r i o n
11 t o l = l e - 5 ; kmax = 1000;
7o Load d a t a p o i n t s
12 name = s p r i n t f ('Data2D_7od7.s' ,N,gridtype) ; load(name)
7, I n i t i a l i z e r e s i d u a l and f i t
13 r_old = testfunction(dsites(:,1),dsites(:,2));
14 u_old = 0;
15 k = 1; maxres(k) = 999999;
7. Use an ( a r b i t r a r y ) i n i t i a l p o i n t
16 ykidx = (N+D/2; y k ( k , : ) = d s i t e s ( y k i d x , :) ;
17 w h i l e (maxres(k) > t o l && k < kmax)
7 E v a l u a t e b a s i s f u n c t i o n a t yk
18 DM_data = D i s t a n c e M a t r i x ( y k ( k , : ) , y k ( k , : ) ) ;
20 beta = r_old(ykidx)/IM;
7o Compute e v a l u a t i o n m a t r i c e s f o r r e s i d u a l and f i t
21 DM_res = D i s t a n c e M a t r i x ( d s i t e s , y k ( k , : ) ) ;
22 RM = rbf(ep,DM_res);
23 DM_eval = D i s t a n c e M a t r i x ( e p o i n t s , y k ( k , : ) ) ;
7o Update r e s i d u a l and f i t
25 r = r _ o l d - beta*RM; u = u_old + beta*EM;
7o F i n d new p o i n t t o add
26 [sr.idx] = sort(abs(r));
27 maxres(k+l) = s r ( e n d ) ;
28 ykidx = i d x ( e n d ) ; yk(k+l,:) = d s i t e s ( y k i d x , : ) ;
29 r_old = r; u _ o l d = u;
30 k = k + 1;
31 end
7o Compute exact s o l u t i o n
32 exact = testfunction(epoints(:,1),epoints(:,2));
33 maxerr = n o r m ( u - e x a c t , i n f ) ; r m s _ e r r = n o r m ( u - e x a c t ) / n e v a l ;
34 f p r i n t f ('RMS error: 7.e\n', r m s _ e r r )
36 f v i e w = [160,20] ; 7, v i e w i n g a n g l e s f o r p l o t
37 PlotSurf(xe,ye,u,neval,exact,maxerr,fview);
38 PlotError2D(xe,ye,u,exact,maxerr,neval,fview);
39 figure; plot(yk(:,1),yk(:,2),'ro')
40 figure; semilogy(maxres,'b');
To i l l u s t r a t e t h e g r e e d y o n e - p o i n t a l g o r i t h m w e p e r f o r m t w o e x p e r i m e n t s . Both
tests use d a t a o b t a i n e d b y s a m p l i n g F r a n k e ' s f u n c t i o n at 16641 H a l t o n p o i n t s i n
2
[0, l ] . H o w e v e r , t h e first t e s t is based o n Gaussians, w h i l e t h e second one uses
inverse m u l t i q u a d r i c s . F o r b o t h tests w e use t h e same shape p a r a m e t e r e = 5.5.
T h i s results i n t h e inverse m u l t i q u a d r i c s h a v i n g a m o r e g l o b a l influence t h a n the
Gaussians. T h i s effect is c l e a r l y e v i d e n t i n t h e f i r s t few a p p r o x i m a t i o n s t o the
i n t e r p o l a n t s i n F i g u r e s 33.2 a n d 33.4.
F i g u r e 33.4, i n p a r t i c u l a r , shows t h a t t h e g r e e d y a l g o r i t h m enforces i n t e r p o l a t i o n
of t h e d a t a o n l y o n t h e m o s t recent set 3 4 (i.e., for t h e o n e - p o i n t a l g o r i t h m s t u d i e d
here o n l y at a single p o i n t ) . I f one w a n t s t o m a i n t a i n t h e i n t e r p o l a t i o n achieved i n
p r e v i o u s i t e r a t i o n s , t h e n t h e sets 3 4 s h o u l d be n e s t e d . T h i s , however, w o u l d have a
significant effect o n t h e e x e c u t i o n t i m e o f t h e a l g o r i t h m since t h e m a t r i c e s at each
step w o u l d increase i n size.
x # Iterations
Fig. 33.1 1000 selected points and residual for greedy one point algorithm with Gaussian R B F s
and N = 16641 data points.
I n o r d e r t o o b t a i n o u r a p p r o x i m a t e i n t e r p o l a n t s w e used a t o l e r a n c e o f 1 0 - 5
a l o n g w i t h a n a d d i t i o n a l u p p e r l i m i t o f kmax=1000 o n t h e n u m b e r o f i t e r a t i o n s . F o r
b o t h tests t h e a l g o r i t h m uses u p a l l 1000 i t e r a t i o n s . T h e final m a x i m u m residual
for Gaussians is maxres = 0.0075, w h i l e for inverse M Q s we have maxres = 0.0035.
I n b o t h cases t h e r e o c c u r r e d several m u l t i p l e p o i n t selections. Contrary to interpo-
y x y x
Fig. 33.2 Fits of Franke's function for greedy one point algorithm with Gaussian R B F s and
N = 16641 data points. Top left to bottom right: 1 point, 2 points, 4 points, final fit with 1000
points.
l a t i o n p r o b l e m s based o n t h e s o l u t i o n o f a l i n e a r s y s t e m , m u l t i p l e p o i n t selections
do n o t pose a p r o b l e m here.
O n e a d v a n t a g e o f t h i s v e r y s i m p l e a l g o r i t h m is t h a t n o l i n e a r systems need t o be
solved. T h i s allows us t o a p p r o x i m a t e t h e i n t e r p o l a n t s for l a r g e d a t a sets even for
g l o b a l l y s u p p o r t e d basis f u n c t i o n s , a n d also w i t h s m a l l values o f e ( a n d t h e r e f o r e a n
associated i l l - c o n d i t i o n e d i n t e r p o l a t i o n m a t r i x ) . One should n o t expect t o o m u c h
i n t h i s case, however, as t h e r e s u l t s i n F i g u r e 33.5 s h o w w h e r e we used a v a l u e o f
e = 0.1 for t h e shape p a r a m e t e r . A s w i t h t h e f i x e d level i t e r a t i o n o f a p p r o x i m a t e
M L S a p p r o x i m a n t s based o n flat g e n e r a t i n g f u n c t i o n s , a l o t o f s m o o t h i n g o c c u r s so
t h a t t h e convergence t o t h e R B F i n t e r p o l a n t is v e r y slow.
M o r e o v e r , i n t h e pseudo-code o f t h e a l g o r i t h m m a t r i x - v e c t o r m u l t i p l i c a t i o n s are
not required, either. H o w e v e r , M A T L A B a l l o w s for a v e c t o r i z a t i o n o f t h e f o r - l o o p
w h i c h does r e s u l t i n t w o m a t r i x - v e c t o r m u l t i p l i c a t i o n s .
F o r p r a c t i c a l s i t u a t i o n s , e.g., for s m o o t h r a d i a l basis f u n c t i o n s a n d densely dis-
t r i b u t e d p o i n t s i n X t h e convergence c a n be r a t h e r slow. T h e s i m p l e g r e e d y algo-
r i t h m described above is e x t e n d e d i n [Schaback a n d W e n d l a n d (2000b)] t o a v e r s i o n
t h a t a d a p t i v e l y uses basis f u n c t i o n s o f v a r y i n g scales.
1 0.8 0.6 0.4 0.2 0 0 200 400 600 800 1000

x # Iterations
Fig. 33.3 1000 selected points and residual for greedy one point algorithm with I M Q R B F s and
N = 16641 data points.
33.2 T h e Faul-Powell Algorithm
A n o t h e r i t e r a t i v e a l g o r i t h m was suggested i n [ F a u l a n d P o w e l l ( 1 9 9 9 ) ; F a u l a n d
P o w e l l ( 2 0 0 0 ) ] . F r o m o u r earlier discussions we k n o w t h a t i t is possible t o express
t h e r a d i a l basis f u n c t i o n i n t e r p o l a n t i n t e r m s o f c a r d i n a l f u n c t i o n s Uj, j = 1 , . . . , TV,
i.e.,
V (x)
f = J2f(x> (x).
j
j=l
T h e basic idea o f t h e F a u l - P o w e l l a l g o r i t h m is t o use approximate cardinal functions
tyj i n s t e a d . O f course, t h i s w i l l o n l y g i v e a n a p p r o x i m a t e v a l u e for t h e i n t e r p o l a n t ,

a n d therefore a n i t e r a t i o n o n t h e residuals is suggested t o i m p r o v e t h e a c c u r a c y o f
this approximation.
T h e basic p h i l o s o p h y o f t h i s a l g o r i t h m is v e r y s i m i l a r t o t h a t o f t h e f i x e d level
i t e r a t i o n o f C h a p t e r 3 1 . I n p a r t i c u l a r , t h e F a u l - P o w e l l a l g o r i t h m c a n also be i n t e r -
p r e t e d as a K r y l o v subspace m e t h o d . H o w e v e r , i n s t e a d o f t a k i n g a p p r o x i m a t e M L S
g e n e r a t i n g f u n c t i o n s , t h e a p p r o x i m a t e c a r d i n a l f u n c t i o n s ^ j , j = l , . . . , N , are de-
t e r m i n e d as l i n e a r c o m b i n a t i o n s o f t h e basis f u n c t i o n s <&(-,X() for t h e i n t e r p o l a n t ,
i.e.,
* i = a*), (33.6)
where L 3 is a n i n d e x set c o n s i s t i n g o f n (n ?a 50) indices t h a t are used t o d e t e r m i n e

t h e a p p r o x i m a t e c a r d i n a l f u n c t i o n . F o r e x a m p l e , t h e n nearest n e i g h b o r s o f X j w i l l
u s u a l l y do. I n general, t h e choice o f i n d e x sets a l l o w s m u c h f r e e d o m , a n d t h i s is t h e
reason w h y we i n c l u d e t h e a l g o r i t h m i n t h i s c h a p t e r o n a d a p t i v e i t e r a t i v e m e t h o d s .
A l s o , as p o i n t e d o u t at t h e e n d o f t h i s s e c t i o n , t h e r e is a c e r t a i n d u a l i t y b e t w e e n
the Faul-Powell a l g o r i t h m and the greedy a l g o r i t h m o f the previous section.
Fig. 33.4 F i t s of Franke's function for greedy one point algorithm with I M Q R B F s and N = 16641
data points. Top left to bottom right: 1 point, 2 points, 4 points, final fit with 1000 points.
For every j = 1,..., N, t h e coefficients bji are f o u n d as s o l u t i o n o f t h e ( r e l a t i v e l y

small) n x n linear system
*j(xi) = 6,
jk ieCj. (33.7)
These a p p r o x i m a t e c a r d i n a l f u n c t i o n s are c o m p u t e d i n a pre-processing step.
A s before, i n its simplest f o r m t h e r e s i d u a l i t e r a t i o n c a n be f o r m u l a t e d as
N
ti<>(aO
3= 1
N
u { k + 1
\ x ) = u { k
\ x ) + Y, x
[f( o) - w (ajj)l *J(SB),
(fc)
k = 0,1,....
3= 1
I n s t e a d o f a d d i n g t h e c o n t r i b u t i o n o f a l l a p p r o x i m a t e c a r d i n a l f u n c t i o n s at t h e
same t i m e , t h i s is done i n a t h r e e - s t e p process i n t h e F a u l - P o w e l l a l g o r i t h m . T o t h i s
e n d , we choose i n d e x sets Cj, j = 1 , . . . , Nn, such t h a t Cj C {j, j + . . . , N} while
m a k i n g sure t h a t j Cj. A l s o , i f one w a n t s t o use t h i s a l g o r i t h m t o a p p r o x i m a t e
t h e i n t e r p o l a n t based o n c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s o f o r d e r m, then
one needs t o ensure t h a t t h e c o r r e s p o n d i n g centers f o r m a n (m l ) - u n i s o l v e n t set
and append a p o l y n o m i a l t o the local expansion (33.6).
Fig. 33.5 1000 selected points (only 20 of them distinct) and fit of Franke's function for greedy
one point algorithm with flat Gaussian R B F s (e = 0.1) and N = 16641 data points.
k k
N o w , i n t h e first step we define u ^ 0 = u^ \ and then iterate
u {k)
= uf2 + Of >tf
x j = 1 , . . . ,/V - , n (33.8)
with
(33.9)
T h e stepsize 9^ is chosen so t h a t t h e n a t i v e space best a p p r o x i m a t i o n t o the

r e s i d u a l Vf Uj _}i f r o m t h e space s p a n n e d b y t h e a p p r o x i m a t e c a r d i n a l f u n c t i o n s
k
tyj is a d d e d . Using the representation (33.6) o f \I/f i n t e r m s o f t h e g l o b a l basis

{&(-,Xi) : i = 1,...,N}, t h e r e p r o d u c i n g k e r n e l p r o p e r t y o f <&, a n d t h e ( l o c a l )
c a r d i n a l i t y p r o p e r t y (33.7) o f ^j we c a n c a l c u l a t e t h e d e n o m i n a t o r o f (33.9) as
<*i,*i>A/-(n) = <*j, E 6 a;
j^(- /)>A^(n)
= E ^(*J.*(-> a J
^(r2)
= E bjiVjixt) = bjj
since we have j G Cj b y c o n s t r u c t i o n o f t h e i n d e x set Cj. S i m i l a r l y , we get for t h e

numerator
{ V f - u f l ^ j ) ^ ) = (V -u f
{ (fc)
k
\, bje$(;x ))^
e m
tCj
,(fc)
= Y b
i * (/(**)-^-ite))
T h e r e f o r e (33.8) a n d (33.9) c a n be w r i t t e n as
uy
l
3
= uf\ + bn ( / ( a * ) - uf\ ( j)
Xi , j = l,...,N-n.
3 3
iec
3
I n t h e second s t e p o f t h e F a u l - P o w e l l a l g o r i t h m t h e r e s i d u a l is i n t e r p o l a t e d o n
t h e r e m a i n i n g n p o i n t s ( c o l l e c t e d v i a t h e i n d e x set * ) . T h u s , we find a f u n c t i o n
i n span{<&(-, Xj) : j G *} such t h a t
t;W(aj ) = / ( * i ) - S L ( a ! < ) ,
i i e T ,
a n d t h e a p p r o x i m a t i o n is u p d a t e d , i.e.,
I n t h e t h i r d step t h e residuals are u p d a t e d , i.e.,
rf + 1 )
= f( ) Xi - u < * > ( ),+1
Xi i = l,...,N. (33.10)
T h e o u t e r i t e r a t i o n ( o n k) is n o w r e p e a t e d unless t h e largest o f these residuals is

small enough.
W e c a n s u m m a r i z e t h i s a l g o r i t h m as
A l g o r i t h m 33.2. Faul-Powell a l g o r i t h m
I n p u t d a t a l o c a t i o n s X {x\,... ,XN}, associated values o f / , a n d t o l e r a n c e

tol > 0
Pre-processing s t e p
Choose n
For 1 < j < N - n d o
D e t e r m i n e t h e i n d e x set Cj
F i n d t h e coefficients bjg o f t h e a p p r o x i m a t e c a r d i n a l f u n c t i o n \Vj b y
solving
Vj(xi) = 6jk, i^Cj

end
Set k = 0 a n d u 0
k)
= 0
I n i t i a l i z e residuals r f ^ = f(xi), i = 1,..., N
Set e max Irf Î
W h i l e e > t o l do
For 1 < j < N - n d o
Update
}
= < i + fÊ (/(*<) - S?iM
end
Solve t h e i n t e r p o l a t i o n p r o b l e m
vWfa) = f(xi) - iiL (*i), n i e c
Update the approximation

u (k+l) _ (fc) ( f c )
u
0
u v
N-n +
k+1 k + 1
C o m p u t e n e w residuals r\ ^ = u 0 \xi), i = 1,..., N
k+1
Set n e w value for e = max \A ^ I
1
i=l,...,JV
Increment k = k + 1
end
F a u l a n d P o w e l l p r o v e t h a t t h i s a l g o r i t h m converges t o t h e s o l u t i o n o f t h e o r i g i n a l
i n t e r p o l a t i o n p r o b l e m . S i m i l a r t o some o f t h e o t h e r a l g o r i t h m s , one needs t o m a k e
sure t h a t t h e residuals are e v a l u a t e d efficiently b y using, e.g., a fast multipole
e x p a n s i o n , fast F o u r i e r t r a n s f o r m , o r c o m p a c t l y s u p p o r t e d functions.
I n i t s m o s t basic f o r m t h e K r y l o v subspace a l g o r i t h m o f F a u l a n d P o w e l l c a n
also be e x p l a i n e d as a d u a l a p p r o a c h t o t h e g r e e d y r e s i d u a l i t e r a t i o n a l g o r i t h m o f
Schaback a n d W e n d l a n d . I n s t e a d o f d e f i n i n g a p p r o p r i a t e sets o f p o i n t s y , k i n the
F a u l a n d P o w e l l a l g o r i t h m one p i c k s c e r t a i n subspaces U k o f t h e n a t i v e space. I n
particular, i f U k is t h e o n e - d i m e n s i o n a l space U k = span{\I/fc} ( w h e r e ty k is a l o c a l
a p p r o x i m a t i o n t o t h e c a r d i n a l f u n c t i o n ) we get t h e a l g o r i t h m d e s c r i b e d above. For
m o r e details see [Schaback a n d W e n d l a n d (2000b)].
W e leave t h e i m p l e m e n t a t i o n o f t h i s a l g o r i t h m t o t h e reader.
Chapter 34
Improving the Condition Number of the

Interpolation Matrix
I n C h a p t e r 16 we n o t e d t h a t t h e s y s t e m m a t r i c e s a r i s i n g i n s c a t t e r e d d a t a i n t e r p o l a -
t i o n w i t h r a d i a l basis f u n c t i o n s t e n d t o b e c o m e v e r y i l l - c o n d i t i o n e d as t h e m i n i m a l
s e p a r a t i o n d i s t a n c e qx b e t w e e n t h e d a t a sites x \ , . . . , CCJV, is r e d u c e d . T h e r e f o r e i t
is n a t u r a l t o devise strategies t o p r e v e n t such i n s t a b i l i t i e s b y e i t h e r p r e c o n d i t i o n i n g
t h e system, or b y f i n d i n g a b e t t e r basis for t h e a p p r o x i m a t i o n space we are u s i n g .
T h e f o r m e r a p p r o a c h is s t a n d a r d p r o c e d u r e i n n u m e r i c a l l i n e a r algebra, a n d i n fact
we c a n use a n y o f t h e w e l l - e s t a b l i s h e d m e t h o d s (such as p r e c o n d i t i o n e d conjugate
g r a d i e n t i t e r a t i o n ) t o i m p r o v e t h e s t a b i l i t y a n d convergence o f t h e i n t e r p o l a t i o n
systems t h a t arise for s t r i c t l y p o s i t i v e d e f i n i t e f u n c t i o n s . I n p a r t i c u l a r , t h e sparse
systems t h a t arise i n ( m u l t i l e v e l ) i n t e r p o l a t i o n w i t h c o m p a c t l y s u p p o r t e d radial
basis f u n c t i o n s c a n be solved efficiently w i t h t h e p r e c o n d i t i o n e d c o n j u g a t e gradi-
ent m e t h o d . H o w e v e r , i n o u r i m p l e m e n t a t i o n (see t h e discussion i n S e c t i o n 12.1)
we use M A T L A B ' S s p a r s e f u n c t i o n w h i c h takes a d v a n t a g e o f state-of-the-art direct
m e t h o d s for sparse l i n e a r systems.
T h e second a p p r o a c h t o i m p r o v i n g t h e c o n d i t i o n n u m b e r o f t h e i n t e r p o l a t i o n
s y s t e m , i.e., t h e idea o f u s i n g a m o r e s t a b l e basis, is w e l l k n o w n f r o m u n i v a r i a t e
p o l y n o m i a l a n d spline i n t e r p o l a t i o n . T h e L a g r a n g e basis f u n c t i o n s for u n i v a r i a t e
p o l y n o m i a l i n t e r p o l a t i o n are t h e i d e a l basis i f we are i n t e r e s t e d i n s t a b l y s o l v i n g
t h e i n t e r p o l a t i o n e q u a t i o n s since t h e r e s u l t i n g i n t e r p o l a t i o n m a t r i x is t h e i d e n t i t y
m a t r i x ( w h i c h is c e r t a i n l y m u c h b e t t e r c o n d i t i o n e d t h a n , e.g., t h e Vandermonde
m a t r i x t h a t we get i f we use a m o n o m i a l basis). S i m i l a r l y , 75-splines give rise t o
d i a g o n a l l y d o m i n a n t , sparse s y s t e m m a t r i c e s w h i c h are m u c h easier t o d e a l w i t h
t h a n t h e m a t r i c e s we w o u l d get i f we were t o represent a s p l i n e i n t e r p o l a n t u s i n g
t h e a l t e r n a t i v e t r u n c a t e d p o w e r basis. B o t h o f these e x a m p l e s are s t u d i e d i n g r e a t
d e t a i l i n s t a n d a r d n u m e r i c a l analysis t e x t s (see, e.g., [ K i n c a i d a n d C h e n e y (2002)])
or i n t h e l i t e r a t u r e o n splines (see, e.g., [Schumaker ( 1 9 8 1 ) ] ) . W e w i l l address a n
analogous a p p r o a c h for r a d i a l basis f u n c t i o n s i n S e c t i o n 34.4 b e l o w .
Before we describe a n y o f t h e specialized p r e c o n d i t i o n i n g p r o c e d u r e s for r a d i a l
basis f u n c t i o n i n t e r p o l a t i o n m a t r i c e s we g i v e t w o e x a m p l e s p r e s e n t e d i n t h e e a r l y
RBF p a p e r [Jackson (1989a)] t o i l l u s t r a t e t h e effects o f a n d m o t i v a t i o n for p r e c o n -
d i t i o n i n g i n t h e c o n t e x t o f r a d i a l basis f u n c t i o n s .
303
34.1 Preconditioning: T w o Simple E x a m p l e s
E x a m p l e 3 4 . 1 . L e t s 1 a n d consider i n t e r p o l a t i o n based o n <yc(r) = r w i t h n o

p o l y n o m i a l t e r m s added. A s d a t a sites w e choose X = { 1 , 2 , . . . , 1 0 } . T h i s leads t o
the system m a t r i x
~0 1 2 3 . . . 9
1 0 1 2 ... 8
2 1 0 1 ... 7
A = 3 2 1 0 ... 6
9 8 76 0
w i t h ^ - c o n d i t i o n n u m b e r c o n d ( A ) 67. I n s t e a d o f s o l v i n g t h e l i n e a r s y s t e m Ac =
T 1 0
y, w h e r e y [ y i , . . . ,yio] E K is a v e c t o r o f g i v e n r e a l n u m b e r s ( d a t a v a l u e s ) ,
we c a n find a s u i t a b l e m a t r i x B t o p r e - m u l t i p l y b o t h sides o f t h e e q u a t i o n s u c h
t h a t t h e s y s t e m is s i m p l e r t o solve. I d e a l l y , t h e n e w s y s t e m m a t r i x BA s h o u l d be
t h e i d e n t i t y m a t r i x , i.e., B s h o u l d be a n approximate inverse o f A. Thus, having
f o u n d a n a p p r o p r i a t e m a t r i x B, we m u s t n o w solve t h e l i n e a r s y s t e m BAc = By.
T h e m a t r i x B is u s u a l l y referred t o as t h e preconditioner o f the linear system. For
t h e m a t r i x A above we c a n choose a p r e c o n d i t i o n e r B as
1 0 0 0 0 0
l -1 h1 0 0 0
2 2
1
0 2 - ]1 \ 0 0
1 0 0
B = 0 0 2
0 0 0 0 - 1 *
0 0 0 0 0 1
T h i s leads t o t h e f o l l o w i n g p r e c o n d i t i o n e d s y s t e m m a t r i x
"0 1 2 . . . 8 9"
0 10 0 0
0 0 1 0 0
BA
0 0 0 1 0
9 8 7 1 0
i n t h e s y s t e m BAc = By. N o t e t h a t BA is a l m o s t a n i d e n t i t y m a t r i x . O n e c a n
easily check t h a t n o w c o n d ( B A ) 45.
T h e m o t i v a t i o n for t h i s choice o f B is t h e f o l l o w i n g . T h e f u n c t i o n (fi(r) = r
or $ ( r r ) = is a f u n d a m e n t a l s o l u t i o n o f t h e L a p l a c i a n A ( = i n the one-
d i m e n s i o n a l case), i.e.
34- Improving the Condition Number of the Interpolation Matrix 305
w h e r e <5n is t h e D i r a c d e l t a f u n c t i o n c e n t e r e d a t zero. Thus, B is chosen as a

d i s c r e t i z a t i o n o f t h e L a p l a c i a n w i t h s p e c i a l choices a t t h e e n d p o i n t s o f t h e d a t a set.
Example 3 4 . 2 . F o r n o n - u n i f o r m l y d i s t r i b u t e d d a t a w e c a n use a different dis-

cretization of the Laplacian A for each r o w o f B. T o see this, let s = 1,
X = {1,|,|,4, |}, and again consider i n t e r p o l a t i o n w i t h the radial function
(p(r) = r. Then
0 1o o1 3 n
2 2
0 1
A = 1 0
3 | 0
3 2 ^ 0
w i t h cond(>l) 18.15, a n d i f we choose
1 0 0 0
3 I
1 - - 2
0
x
2
1
B = oi 1
5
6 3
_ 4
0 0 3
3
0 0 0 0
based o n second-order b a c k w a r d differences o f t h e p o i n t s i n X, then the precondi-
t i o n e d s y s t e m t o be solved becomes
n I 3 o 1
u
2 2 2
0 10 0 0
c = By.
0 0 10 0
0 0 0 1 0
LI 3 2 I 0
O n c e m o r e , t h i s s y s t e m is a l m o s t t r i v i a l t o solve a n d has a n i m p r o v e d c o n d i t i o n
number of cond(B^4) 8.94.
34.2 Early Preconditioners
I l l - c o n d i t i o n i n g o f t h e i n t e r p o l a t i o n m a t r i c e s was i d e n t i f i e d as a serious p r o b l e m
v e r y early, a n d N i r a D y n a l o n g w i t h some o f her c o - w o r k e r s (see, e.g., [ D y n (1987);
D y n (1989); D y n a n d L e v i n ( 1 9 8 3 ) ; D y n et al. ( 1 9 8 6 ) ] ) p r o v i d e d some o f t h e first
p r e c o n d i t i o n i n g s t r a t e g i e s t a i l o r e d e s p e c i a l l y t o r a d i a l basis f u n c t i o n i n t e r p o l a n t s .
For t h e f o l l o w i n g discussion w e c o n s i d e r t h e g e n e r a l i n t e r p o l a t i o n p r o b l e m t h a t
includes p o l y n o m i a l r e p r o d u c t i o n (see C h a p t e r 6 ) . T h e r e f o r e , w e h a v e t o solve t h e
following system of linear equations
" A P~ c y
T
(34.1)
P O d 0
w i t h t h e i n d i v i d u a l pieces g i v e n b y Ajk = y ( | | c c j ajfc||), j , k = 1 , . . . ,7V, Pji

T T
Pefaj), j = 1,...,N, i = 1,...,M, c = [c ,...
x ,c \ , N d = [d , , d ] ,
x M V =
T
[yi,... ,yN] i O an M x M zero m a t r i x , a n d 0 a zero v e c t o r o f l e n g t h M with
M = dim H _. m 1 H e r e , as discussed e a r l i e r , <p s h o u l d be s t r i c t l y c o n d i t i o n a l l y
s
positive definite of order m and r a d i a l o n R a n d t h e set X = {x\,..., CCJV} s h o u l d
be ( m l ) - u n i s o l v e n t .
T h e p r e c o n d i t i o n i n g scheme p r o p o s e d b y D y n a n d her c o - w o r k e r s is a g e n e r a l -
i z a t i o n o f t h e s i m p l e d i f f e r e n c i n g scheme discussed above. I t is m o t i v a t e d b y t h e
fact t h a t t h e p o l y h a r m o n i c splines (i.e., t h i n p l a t e splines a n d r a d i a l p o w e r s )
f 2fc-s i g
r 0 r ? s even,
^ = { r ^ , sodd,
s
2k > s, are f u n d a m e n t a l s o l u t i o n s o f t h e k-th i t e r a t e d L a p l a c i a n i n M , i.e.
k
A v(\\x\\) = c5 (x), 0
w h e r e 5n is t h e D i r a c d e l t a f u n c t i o n c e n t e r e d a t t h e o r i g i n , a n d c is a n a p p r o p r i a t e
constant.
2 : L 2
F o r t h e (inverse) m u l t i q u a d r i c s (f(r) = (1 + r ) / , w h i c h are also discussed i n
t h e p a p e r s m e n t i o n e d above, a p p l i c a t i o n o f t h e L a p l a c i a n y i e l d s a s i m i l a r l i m i t i n g
behavior, i.e.
l i m A (p(r)
k
= 0,
r>oo
a n d for r * 0
A cp(r)
k
1.
One now wants t o discretize t h e L a p l a c i a n o n the (irregular) mesh given b y the

( s c a t t e r e d ) d a t a sites in X. T o t h i s e n d t h e a u t h o r s o f [ D y n et al. (1986)] suggest
2
t h e f o l l o w i n g p r o c e d u r e for t h e case o f s c a t t e r e d d a t a i n t e r p o l a t i o n over M.
(1) S t a r t w i t h a t r i a n g u l a t i o n o f t h e set X, e.g., t h e w i l l do. T h i s t r i a n g u l a t i o n

c a n be v i s u a l i z e d as follows.
(a) B e g i n w i t h t h e p o i n t s i n X a n d c o n s t r u c t t h e i r or Voronoi diagram. The

2
D i r i c h l e t t i l e o f a p a r t i c u l a r p o i n t x is t h a t subset o f p o i n t s i n R w h i c h are
closer t o x t h a n t o a n y o t h e r p o i n t i n X. T h e d a s h e d lines i n F i g u r e 3 4 . 1
d e n o t e t h e D i r i c h l e t t e s s e l a t i o n for t h e set o f 25 H a l t o n p o i n t s (circles) i n
2
[0,1] .
( b ) C o n s t r u c t t h e D e l a u n a y t r i a n g u l a t i o n , w h i c h is t h e d u a l o f t h e D i r i c h l e t
t e s s e l a t i o n , i.e., connect a l l strong neighbors i n the D i r i c h l e t tesselation,
i.e., p o i n t s whose t i l e s s h a r e a c o m m o n edge. T h e s o l i d lines i n F i g u r e 3 4 . 1
d e n o t e t h e c o r r e s p o n d i n g D e l a u n a y t r i a n g u l a t i o n o f t h e 25 H a l t o n p o i n t s .
0 0.2 0.4 0.6 0.8 1
Fig. 34.1 Dirichlet tesselation (dashed lines) and corresponding Delaunay triangulation (solid
lines) of 25 Halton points (circles).
(2) D i s c r e t i z e t h e L a p l a c i a n o n t h i s t r i a n g u l a t i o n . I n o r d e r t o also t a k e i n t o a c c o u n t
t h e b o u n d a r y p o i n t s D y n , L e v i n a n d R i p p a i n s t e a d use a d i s c r e t i z a t i o n o f a n
2
i t e r a t e d Green's f o r m u l a w h i c h has t h e space I I n _ 1 as i t s n u l l space. The
necessary p a r t i a l d e r i v a t i v e s are t h e n a p p r o x i m a t e d o n t h e t r i a n g u l a t i o n u s i n g
c e r t a i n sets o f vertices o f t h e t r i a n g u l a t i o n . ( t h r e e p o i n t s for first o r d e r p a r t i a l s ,
six for second o r d e r ) .
F i g u r e 3 4 . 1 was c r e a t e d i n M A T L A B u s i n g t h e c o m m a n d s
load('Data2D_25h')
tes = delaunayn(dsites);
triplot(tes,dsites(:,1),dsites(:,2),'k-')
h o l d on
[vx, vy] = voronoi(dsites(:,1),dsites(:,2),tes);
plot(dsites(:,1),dsites(:,2),'ko',vx,vy,'k')
axis([0 1 0 1])
A s i n o u r o t h e r M A T L A B e x a m p l e s , t h e file Data2D_25h c o n t a i n s t h e coordinates

of t h e 25 H a l t o n p o i n t s i n t h e a r r a y d s i t e s .
T h e d i s c r e t i z a t i o n described above y i e l d s t h e m a t r i x B = {bji)f i=1 as t h e p r e -
conditioning m a t r i x i n a w a y analogous t o the previous section. W e n o w o b t a i n
N
(BA) jk = E bjM\\xi - x \\)
k
m
A <p(\\ --XklDixj), j,k = l , . . . , N. (34.2)
i=l
T h i s m a t r i x has t h e p r o p e r t y t h a t t h e entries close t o t h e d i a g o n a l are l a r g e c o m -

p a r e d t o those away f r o m t h e d i a g o n a l , w h i c h decay t o zero as t h e d i s t a n c e b e t w e e n
t h e t w o p o i n t s i n v o l v e d goes t o i n f i n i t y . Since t h e c o n s t r u c t i o n o f B ( i n s t e p 2 a b o v e )
ensures t h a t p a r t o f t h e p r e c o n d i t i o n e d b l o c k m a t r i x vanishes, n a m e l y BP = O,
one m u s t n o w solve t h e n o n - s q u a r e system
' BA' By'

P T c
0
A c t u a l l y , t h e t o p p a r t o f t h e s y s t e m , t h e s q u a r e s y s t e m BAc = By, is s i n g u l a r ,
but i t is s h o w n i n t h e p a p e r [ D y n et al. (1986)] t h a t the a d d i t i o n a l constraints
T
Pc = 0 g u a r a n t e e existence o f a u n i q u e s o l u t i o n . F u r t h e r m o r e , t h e coefficients d
i n t h e o r i g i n a l e x p a n s i o n o f t h e i n t e r p o l a n t Vf c a n be o b t a i n e d b y s o l v i n g
Pd y Ac,
i.e., b y f i t t i n g t h e p o l y n o m i a l p a r t o f t h e e x p a n s i o n t o t h e r e s i d u a l y Ac.
T h e a p p r o a c h j u s t d e s c r i b e d leads t o l o c a l i z e d basis f u n c t i o n s \ I / t h a t are l i n e a r
c o m b i n a t i o n s o f t h e o r i g i n a l basis f u n c t i o n s (p. M o r e precisely,
N
*j(x) = 5><V(H* - a*||) A " V ( | | -XJWKX), (34.3)
w h e r e t h e coefficients b i are d e t e r m i n e d v i a t h e d i s c r e t i z a t i o n d e s c r i b e d above.

3
T h e l o c a l i z e d basis f u n c t i o n s tyj,j = l,...,N, (see ( 3 4 . 3 ) ) c a n be v i e w e d as a n

a l t e r n a t i v e ( b e t t e r c o n d i t i o n e d ) basis for t h e a p p r o x i m a t i o n space s p a n n e d b y t h e
functions <&j = ip(\\ Xj\\). W e w i l l c o m e b a c k t o t h i s i d e a i n S e c t i o n 34.4.
I n [ D y n et al. (1986)] t h e a u t h o r s describe h o w t h e p r e c o n d i t i o n e d m a t r i c e s c a n
be used efficiently i n c o n j u n c t i o n w i t h v a r i o u s i t e r a t i v e schemes s u c h as C h e b y s h e v
i t e r a t i o n or a v e r s i o n o f t h e c o n j u g a t e g r a d i e n t m e t h o d . T h e a u t h o r s also m e n -
t i o n s m o o t h i n g o f n o i s y d a t a , o r low-pass f i l t e r i n g as o t h e r a p p l i c a t i o n s for t h i s
p r e c o n d i t i o n i n g scheme.
T h e effectiveness o f t h e above p r e c o n d i t i o n i n g s t r a t e g y was i l l u s t r a t e d w i t h some
n u m e r i c a l e x a m p l e s i n [ D y n et al. ( 1 9 8 6 ) ] . W e l i s t some o f t h e i r r e s u l t s i n T a b l e 3 4 . 1 .
T h i n p l a t e splines a n d m u l t i q u a d r i c s were t e s t e d o n t w o d i f f e r e n t d a t a sets ( g r i d
2
I a n d g r i d I I ) i n M. . T h e s h a p e p a r a m e t e r e for t h e m u l t i q u a d r i c s w a s chosen t o
be t h e r e c i p r o c a l o f t h e average m e s h size. A l i n e a r t e r m was a d d e d for t h i n p l a t e
splines, a n d a c o n s t a n t for m u l t i q u a d r i c s .
Table 34.1 Condition numbers without and with preconditioning.
N grid I orig. grid I precond. grid I I orig. grid I I precond.
TPS 49 1181 4.3 1885 3.4

121 6764 5.1 12633 3.9
MQ 49 7274 69.2 17059 222.8
121 10556 126.0 107333 576.0
O n e c a n see t h a t t h e m o s t d r a m a t i c i m p r o v e m e n t is a c h i e v e d for t h i n plate

splines. T h i s is t o be e x p e c t e d since t h e m e t h o d d e s c r i b e d a b o v e is t a i l o r e d t o these
f u n c t i o n s . A s n o t e d earlier, for m u l t i q u a d r i c s a n a p p l i c a t i o n o f t h e L a p l a c i a n does
n o t y i e l d t h e d e l t a f u n c t i o n , b u t for values o f r close t o zero gives j u s t r e l a t i v e l y

large values.
A n o t h e r e a r l y p r e c o n d i t i o n i n g s t r a t e g y was suggested i n [ P o w e l l (1994a)]. P o w e l l
uses Householder t r a n s f o r m a t i o n s t o c o n v e r t t h e m a t r i x o f t h e i n t e r p o l a t i o n s y s t e m
(34.1) t o a s y m m e t r i c p o s i t i v e d e f i n i t e m a t r i x , a n d t h e n uses t h e c o n j u g a t e gradient
m e t h o d . H o w e v e r , P o w e l l r e p o r t s t h a t t h i s m e t h o d is n o t p a r t i c u l a r l y effective for
2
large t h i n p l a t e spline i n t e r p o l a t i o n p r o b l e m s i n R .
I n [ B a x t e r (1992a); B a x t e r (2002)] p r e c o n d i t i o n e d c o n j u g a t e g r a d i e n t m e t h o d s
for s o l v i n g t h e i n t e r p o l a t i o n p r o b l e m are discussed i n t h e case w h e n Gaussians
or m u l t i q u a d r i c s are used o n a r e g u l a r g r i d . T h e r e s u l t i n g m a t r i c e s are T o e p l i t z
m a t r i c e s , a n d a large b o d y o f l i t e r a t u r e e x i s t s for d e a l i n g w i t h m a t r i c e s h a v i n g t h i s
special s t r u c t u r e (see, e.g., [ C h a n a n d S t r a n g ( 1 9 8 9 ) ] ) .
34.3 Preconditioned G M R E S via Approximate Cardinal Functions
M o r e recently, B e a t s o n , C h e r r i e a n d M o u a t [ B e a t s o n et al. (1999)] p r o p o s e d a p r e -

c o n d i t i o n e r for t h e i t e r a t i v e s o l u t i o n o f r a d i a l basis f u n c t i o n i n t e r p o l a t i o n s y s t e m s
i n c o n j u n c t i o n w i t h t h e G M R E S m e t h o d o f [Saad a n d S c h u l t z ( 1 9 8 6 ) ] . T h e G M R E S
m e t h o d is a general p u r p o s e i t e r a t i v e solver t h a t c a n be a p p l i e d t o n o n s y m m e t r i c
( n o n d e f i n i t e ) systems. F o r fast convergence t h e m a t r i x s h o u l d be preconditioned
such t h a t i t s eigenvalues are c l u s t e r e d a r o u n d one a n d a w a y f r o m t h e o r i g i n . O b v i -
ously, i f t h e basis f u n c t i o n s for t h e r a d i a l basis f u n c t i o n space were c a r d i n a l func-
t i o n s , t h e n t h e m a t r i x w o u l d be t h e i d e n t i t y m a t r i x w i t h a l l i t s eigenvalues e q u a l
t o one. T h e r e f o r e , t h e G M R E S m e t h o d w o u l d converge i n a single i t e r a t i o n . C o n -
sequently, t h e p r e c o n d i t i o n i n g s t r a t e g y e m p l o y e d b y t h e a u t h o r s o f [ B e a t s o n et al.
(1999)] for t h e G M R E S m e t h o d is t o o b t a i n a p r e c o n d i t i o n i n g m a t r i x B t h a t is
close t o t h e inverse o f A.
Since i t is t o o expensive t o find t h e t r u e c a r d i n a l basis ( t h i s w o u l d i n v o l v e
at least as m u c h w o r k as s o l v i n g t h e i n t e r p o l a t i o n p r o b l e m ) , t h e i d e a pursued
in [Beatson et al. (1999)] (and suggested e a r l i e r i n [Beatson et al. (1996);
B e a t s o n a n d P o w e l l ( 1 9 9 3 ) ] ) is t o find approximate cardinal functions similar t o
the functions i n the previous subsection. N o w , however, t h e r e is also a n e m -
phasis o n efficiency, i.e., w e are i n t e r e s t e d i n local a p p r o x i m a t e c a r d i n a l f u n c t i o n s ,
i f possible (c.f. also t h e use o f a p p r o x i m a t e c a r d i n a l f u n c t i o n s i n t h e F a u l - P o w e l l
a l g o r i t h m o f S e c t i o n 3 3 . 2 ) . Several different s t r a t e g i e s for t h e c o n s t r u c t i o n o f these
a p p r o x i m a t e c a r d i n a l f u n c t i o n s were suggested i n [ B e a t s o n et al. ( 1 9 9 9 ) ] . W e w i l l
n o w e x p l a i n t h e basic idea.
G i v e n t h e centers x\,..., XN for t h e basis f u n c t i o n s i n t h e R B F i n t e r p o l a n t
N
C X X
v (x)
f = Y M \ \ - J)>
t h e j-th a p p r o x i m a t e c a r d i n a l f u n c t i o n is g i v e n as a l i n e a r c o m b i n a t i o n o f t h e basis
f u n c t i o n s <3>i = <p(|| Xi\\), w h e r e i r u n s over ( s o m e s u b s e t o f ) { 1 , . . . , AT}, i.e.,
N
34 4
*J = E^^H -^H) ( -)
i=l
H e r e pj is a p o l y n o m i a l i n Tl _ m 1 t h a t is u s e d o n l y i n t h e c o n d i t i o n a l l y p o s i t i v e
d e f i n i t e case, a n d t h e coefficients bji satisfy t h e u s u a l c o n d i t i o n s
N
b
E m(î) = 0 for a 1 1
Pj e n s
m -i- (34.5)
i=l
T h e k e y f e a t u r e i n d e s i g n i n g t h e a p p r o x i m a t e c a r d i n a l f u n c t i o n s is t o h a v e o n l y
a few n <C N coefficients i n ( 3 4 . 4 ) t o be n o n z e r o . I n t h a t case t h e f u n c t i o n s \&j
are f o u n d b y s o l v i n g s m a l l n x n l i n e a r s y s t e m s , w h i c h is m u c h m o r e efficient t h a n
dealing w i t h the original N x N system. F o r e x a m p l e , i n [ B e a t s o n et al. (1999)]
t h e a u t h o r s use n 50 for p r o b l e m s i n v o l v i n g u p t o 10000 centers. T h e r e s u l t i n g
p r e c o n d i t i o n e d s y s t e m is o f t h e s a m e f o r m as t h e e a r l i e r p r e c o n d i t i o n e r ( 3 4 . 2 ) , i.e.,
we n o w have t o solve t h e p r e c o n d i t i o n e d p r o b l e m
(BA)c = By,
w h e r e t h e e n t r i e s o f t h e m a t r i x BA are j u s t ^j(xk), j,k = 1,..., N.

T h e s i m p l e s t s t r a t e g y for d e t e r m i n i n g t h e coefficients bji is t o select t h e n nearest
n e i g h b o r s o f X j , a n d t o f i n d bji b y s o l v i n g t h e ( l o c a l ) c a r d i n a l i n t e r p o l a t i o n p r o b l e m
\Vj(xi)=5ij, i = l,...,n,
subject t o the m o m e n t constraint (34.5) l i s t e d a b o v e . H e r e 5ij is t h e Kronecker-

d e l t a , so t h a t \I>f is one a t X j a n d zero a t a l l o f t h e n e i g h b o r i n g centers a^.
T h i s basic s t r a t e g y is i m p r o v e d b y a d d i n g so-called special points t h a t are dis-
t r i b u t e d ( v e r y sparsely) t h r o u g h o u t t h e d o m a i n (for e x a m p l e n e a r c o r n e r s o f t h e
d o m a i n , or a t o t h e r s i g n i f i c a n t l o c a t i o n s ) .
A few n u m e r i c a l r e s u l t s for t h i n p l a t e s p l i n e a n d m u l t i q u a d r i c i n t e r p o l a t i o n i n
2
M f r o m [ B e a t s o n et al. (1999)] are l i s t e d i n T a b l e 34.2. T h e c o n d i t i o n n u m b e r s are
^ 2 - c o n d i t i o n n u m b e r s , a n d t h e p o i n t s were r a n d o m l y d i s t r i b u t e d i n t h e u n i t s q u a r e .
T h e " l o c a l p r e c o n d . " c o l u m n uses t h e n 50 nearest n e i g h b o r s t o d e t e r m i n e the
a p p r o x i m a t e c a r d i n a l f u n c t i o n s , w h e r e a s t h e r i g h t - m o s t c o l u m n uses t h e 4 1 nearest
neighbors plus nine special points placed u n i f o r m l y i n the u n i t square. T h e effect
of the preconditioning o n the performance o f t h e G M R E S a l g o r i t h m is, e.g., a
r e d u c t i o n f r o m 103 t o 8 i t e r a t i o n s for t h e 289 p o i n t d a t a set for t h i n p l a t e splines,
or f r o m 145 i t e r a t i o n s t o 11 for m u l t i q u a d r i c s .
A n e x t e n s i o n o f t h e ideas o f [ B e a t s o n et al. (1999)] t o linear systems arising
i n t h e c o l l o c a t i o n s o l u t i o n o f p a r t i a l d i f f e r e n t i a l e q u a t i o n s (see C h a p t e r 3 8 ) was
e x p l o r e d i n M o u a t ' s P h . D . thesis [ M o u a t (2001)] a n d also i n t h e r e c e n t p a p e r [ L i n g
and K a n s a (2005)].
,tx
Table 34.2 Condition numbers without and with preconditioning.
V N unprecond. local precond. local precond. w/special
TPS 289 4.005e+006 1.464e+003 5.721e+000

1089 2.753e+008 6.359e+005 1.818e+002
4225 2.605e+009 2.381e+006 1.040e+006
MQ 289 1.506e+008 3.185e+003 2.639e+002
1089 2.154e+009 8.125e+005 5.234e+004
4225 3.734e+010 1.390e+007 4.071e+004
34.4 Change o f Basis
A s p o i n t e d o u t at t h e b e g i n n i n g o f t h i s c h a p t e r , a n o t h e r a p p r o a c h t o o b t a i n i n g a
b e t t e r c o n d i t i o n e d i n t e r p o l a t i o n s y s t e m is t o w o r k w i t h a different basis for t h e
a p p r o x i m a t i o n space. W h i l e t h i s idea is i m p l i c i t l y addressed i n t h e p r e c o n d i t i o n i n g
strategies discussed above, w e w i l l n o w m a k e i t o u r p r i m a r y goal t o f i n d a b e t t e r
c o n d i t i o n e d basis for t h e R B F a p p r o x i m a t i o n space. U n i v a r i a t e piecewise linear
splines a n d n a t u r a l cubic splines c a n be i n t e r p r e t e d as r a d i a l basis f u n c t i o n s , and
we k n o w t h a t 73-splines f o r m stable bases for those spaces. T h e r e f o r e , i t s h o u l d be
possible t o generalize t h i s i d e a for o t h e r R B F s .
T h e process o f f i n d i n g a " b e t t e r " basis for c o n d i t i o n a l l y p o s i t i v e d e f i n i t e r a d i a l
basis f u n c t i o n s is closely c o n n e c t e d t o finding t h e r e p r o d u c i n g k e r n e l o f t h e associ-
a t e d n a t i v e space. Since we d i d n o t e l a b o r a t e o n t h e c o n s t r u c t i o n o f n a t i v e spaces
for c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s earlier, we w i l l n o w present t h e relevant
formulas w i t h o u t going i n t o any further details. I n p a r t i c u l a r , for p o l y h a r m o n i c
splines we w i l l be able t o find a basis t h a t is i n a c e r t a i n sense homogeneous, and
therefore t h e c o n d i t i o n n u m b e r o f t h e r e l a t e d i n t e r p o l a t i o n m a t r i x w i l l d e p e n d o n l y
on t h e n u m b e r N o f d a t a p o i n t s , b u t not o n t h e i r s e p a r a t i o n d i s t a n c e (c.f. the
discussion i n C h a p t e r 16). T h i s a p p r o a c h was suggested b y B e a t s o n , L i g h t and
B i l l i n g s [Beatson et al. ( 2 0 0 0 ) ] , a n d has i t s r o o t s i n [Sibson a n d Stone ( 1 9 9 1 ) ] .
Let $ be a s t r i c t l y c o n d i t i o n a l l y positive definite kernel of order m, and
X {x\,... ,XN} C O C R s
be a n (m l ) - u n i s o l v e n t set o f centers. T h e n the
r e p r o d u c i n g k e r n e l for t h e n a t i v e space A / * ( f 2 ) is g i v e n b y
M M
K(x,y) = $(x,y) - Y k( )$( k,y)

p x x
- Y,Pe(y)( , x x
e)
fc=i i=\
M M M
pk x pe y Xk Xe> + 34 6
+ E Y, ( ) ( )( > ) ^ptWptty)' ( -)
k=l e=l 1=1
w h e r e t h e p o i n t s {x\,..., X M } c o m p r i s e a n (m l ) - u n i s o l v e n t subset o f X a n d t h e
polynomials p, k k = 1 , . . . , M, f o r m a cardinal basis for Tl _ m 1 o n t h i s subset whose
d i m e n s i o n is M = ( ^ " ^ Y 1
) , i.e.,
Pi(x )=S , k kti k,e = l,...,M.

T h i s f o r m u l a t i o n o f t h e r e p r o d u c i n g k e r n e l for t h e c o n d i t i o n a l l y p o s i t i v e d e f i n i t e
case also appears i n t h e s t a t i s t i c s l i t e r a t u r e i n t h e c o n t e x t o f kriging (see, e.g.,
[ B e r l i n e t a n d T h o m a s - A g n a n ( 2 0 0 4 ) ] ) . I n t h a t c o n t e x t t h e k e r n e l K is a covariance
k e r n e l associated w i t h t h e g e n e r a l i z e d covariance <. T h e s e t w o kernels give rise t o
the k r i g i n g equations and d u a l k r i g i n g equations, respectively.
A n i m m e d i a t e consequence o f h a v i n g f o u n d t h e r e p r o d u c i n g k e r n e l K is t h a t w e
c a n express t h e r a d i a l basis f u n c t i o n i n t e r p o l a n t t o values o f some f u n c t i o n / g i v e n
on X i n the form
N
c K x x s
V (x)
f = Y i ( ' j ) > x e R .
3= 1
N o t e t h a t t h e k e r n e l K used here is a s t r i c t l y p o s i t i v e d e f i n i t e k e r n e l (since i t is

a reproducing kernel) w i t h b u i l t - i n p o l y n o m i a l precision. T h e coefficients Cj are
determined by satisfying the i n t e r p o l a t i o n conditions
Pf(xi) = f(xi), i = l,...,N.
W e w i l l see b e l o w ( i n Tables 34.3 a n d 34.4) t h a t t h i s basis a l r e a d y p e r f o r m s " b e t t e r "

(i.e., is b e t t e r c o n d i t i o n e d ) t h a n t h e s t a n d a r d basis {<>(-, x{), ...,<>(, X N ) } i f we
keep t h e n u m b e r o f centers fixed, a n d v a r y o n l y t h e i r s e p a r a t i o n distance.
T o o b t a i n t h e h o m o g e n e o u s basis referred t o above we m o d i f y K b y s u b t r a c t i n g
t h e tensor p r o d u c t p o l y n o m i a l , i.e.,
M
n(x,y) = K(x,y) - x
Y^P^ )P^v)-
e=i
Now, i f y denotes a n y one o f t h e p o i n t s x\,..., XM i n t h e (m l ) - u n i s o l v e n t subset
o f X used i n t h e c o n s t r u c t i o n o f K above, t h e n we h a v e
M M
(-,!/) = <s>(-,y) - YPk(-)^( k,y) x

- EMX/W. *) 3
fc=i e=i
M M
+E Y (') ( )( ' )
pk pe y xk xl
M M
= $(-,y) p x
- Y k(')^( k,y) - $(-,!/) + Y P k
( ' W X k y
> ) = 0
fc=i fc=i
since t h e p o l y n o m i a l s p k are c a r d i n a l o n i r i , . . . , XM, i-e-, o n l y one o f t h e Pk(y) will
"survive".
T h i s m e a n s t h a t t h e f u n c t i o n s K(-, X j ) , j 1 , . . . , N, c a n n o t be used as a basis
o f o u r a p p r o x i m a t i o n space. I n s t e a d we need t o r e m o v e t h e p o i n t s used t o define
t h e c a r d i n a l p o l y n o m i a l s a b o v e f r o m t h e set o f centers used for K. O n c e we d o t h i s
i t t u r n s o u t t h a t t h e m a t r i x C w i t h entries Cij = n(xi, X j ) , i,j = M+l,...,N,is
p o s i t i v e d e f i n i t e , a n d therefore we o b t a i n t h e f o l l o w i n g basis
{Pi, , P M } U {K(-,X +I), M -,K(-,X )} N

for t h e space s p a n { $ ( - , # 1 ) , . . . , <&(, X N ) } - Therefore t h e i n t e r p o l a n t can be repre-

sented i n t h e f o r m
M N
x
Vf( ) = ^2 jPj( )d x +
E CkK(x,x ),k x e R s
. (34.7)
j=l fc=M+l
T h e coefficients are determined as u s u a l by enforcing the interpolation condi-
t i o n s Vf(xi) = f(xi), i 1,...,7V. Since t h e p o l y n o m i a l s pj are c a r d i n a l o n
{xi,..., X M } a n d K was s h o w n t o be zero i f c e n t e r e d a t these p o i n t s , t h i s leads t o
the following linear system
' / o" d VP
(34.8)
P T
c_ c
with I an M x M identity matrix, O an M x (TV M) zero m a t r i x , C as
T
above, P io = Pj(xi), j = 1,...,M, i = M + 1,...,TV, c = [ c + i , ,
M c],
N
d [di,..., ^M] , T
a n d t h e r i g h t - h a n d side v e c t o r s y p [/(xi),..., /(XM)] T
and
2/ =
[/(^M+I), i f(xN)] - T
The identity block (cardinality of the p o l y n o m i a l
basis f u n c t i o n s ) i m p l i e s t h a t t h e coefficient v e c t o r d is g i v e n b y
dj = f(xj), j = l,...,M,
a n d therefore t h e s y s t e m ( 3 4 . 8 ) c a n be s o l v e d as
T
Cc = y K - P d. (34.9)
As m e n t i o n e d above, one c a n s h o w t h a t t h e m a t r i x C is s y m m e t r i c a n d p o s i t i v e
definite.
M o s t i m p o r t a n t l y , for p o l y h a r m o n i c splines, t h e ^ - c o n d i t i o n n u m b e r o f t h e m a -
h
t r i x C is i n v a r i a n t u n d e r a u n i f o r m s c a l i n g o f t h e centers, i.e., i f C = (n(hxi, hxj)),
then
h
cond{C ) = cond(C).
T h i s is p r o v e d t o v a r y i n g degrees o f d e t a i l i n t h e p a p e r s [ B e a t s o n et al. (2000);
Iske (2003a)] a n d t h e b o o k [ W e n d l a n d ( 2 0 0 5 a ) ] .
E x a m p l e 3 4 . 3 . T h e s i m p l e s t e x a m p l e is g i v e n b y t h e p o l y h a r m o n i c s p l i n e <p(r) r.
I n t h i s case M = 1 so t h a t t h e o n l y p o l y n o m i a l t e r m is g i v e n b y t h e c o n s t a n t p = 1.
F o r s i m p l i c i t y we use t h e o r i g i n as a s p e c i a l p o i n t . U s i n g these c o n v e n t i o n s w e h a v e
t h e f o l l o w i n g t h r e e r e p r e s e n t a t i o n s o f t h e v a r i o u s kernels:
&{x,y) = \\x-y\\,
K( ,y)
X = \\x-y\\-\\y\\-\\x\\ + l,
K(x,y) = ||JE - y\\ - \\y\\ - \\x\\.

N o t e t h a t i n t h i s case t h e c o n d i t i o n n u m b e r o f t h e m a t r i x C a s s o c i a t e d w i t h the
k e r n e l K is c l e a r l y i n v a r i a n t u n d e r u n i f o r m s c a l i n g o f t h e p r o b l e m . However, the
m a t r i x A associated w i t h t h e basic n o r m R B F $ enjoys t h e same i n v a r i a n c e . It
T
is o n l y w h e n we a d d t h e p o l y n o m i a l b l o c k s P a n d P t o ensure r e p r o d u c t i o n o f
constants t h a t the c o n d i t i o n number of the resulting block m a t r i x w i l l v a r y greatly
w i t h the p r o b l e m scaling. A s i m i l a r dependence o f t h e c o n d i t i o n n u m b e r of the
s y s t e m m a t r i x o n t h e s c a l i n g is a s s o c i a t e d w i t h t h e k e r n e l K.
34.5 Effect of t h e " B e t t e r " B a s i s o n t h e C o n d i t i o n N u m b e r of t h e

Interpolation M a t r i x
W e r e p r o d u c e some n u m e r i c a l e x p e r i m e n t s f r o m [ B e a t s o n et al. (2000)] b a s e d o n

2
t h e use o f t h i n p l a t e splines i n M . We compute the .^-condition numbers of the
i n t e r p o l a t i o n m a t r i x for t h e t h r e e d i f f e r e n t a p p r o a c h e s m e n t i o n e d a b o v e , i.e., using
t h e s t a n d a r d basis c o n s i s t i n g o f f u n c t i o n s $>(-,Xj) and monomials (obtained w i t h
t h e M A T L A B p r o g r a m R B F I n t e r p o l a t i o n 2 D l i n e a r . m o f C h a p t e r 6 ) , u s i n g t h e re-
p r o d u c i n g kernels K(-,Xj), a n d using t h e m a t r i x C based o n t h e kernel K. The
m a t r i x for t h e kernels K(-,Xj) is c o m p u t e d w i t h t h e p r o g r a m t p s K . m p r o v i d e d i n
Program 34.1. T h e t h r e e p o l y n o m i a l c a r d i n a l f u n c t i o n s are b a s e d o n t h e three
corners ( 0 , 0 ) , ( 0 , 1 ) , a n d ( 1 , 0 ) o f t h e u n i t square, i.e.,
Pi(z) = 1 - z x - 22,
p (z)
2 = z ,
x
w h e r e z = (z ,z )x 2 G R. 2
T h e p r o g r a m t p s K . m is c o m p l e t e l y v e c t o r i z e d , i.e., we i n p u t arrays of p o i n t s x
a n d y , a n d c r e a t e t h e e n t i r e m a t r i x w i t h e n t r i e s K(xi, X j ) , i,j 1,..., N (denoted
b y r b f i n t h e p r o g r a m ) . W e assemble t h e m a t r i x a c c o r d i n g t o t h e t e r m s i n ( 3 4 . 6 ) .
On lines 3 a n d 4 w e f i l l t w o m a t r i c e s , p x a n d p y , w h o s e c o l u m n s c o n t a i n t h e values
of the polynomials p , x p 2 a n d ps ( d e f i n e d as s e p a r a t e f u n c t i o n s a t t h e e n d o f t h e
p r o g r a m ) at a l l of the p o i n t s i n x a n d y, respectively. T h e first t e r m o f (34.6), the
m a t r i x <fr(x, y), is assembled o n l i n e 5 w h e r e w e c a l l t h e s u b r o u t i n e t p s .m l i s t e d as
P r o g r a m C . 4 i n A p p e n d i x C . N e x t , o n l i n e s 611 w e a d d t h e n e x t t w o s u m s f r o m
(34.6) s i m u l t a n e o u s l y . T h e d o u b l e s u m is a d d e d t o t h e m a t r i x r b f o n lines 12
17, a n d f i n a l l y t h e t e n o r p r o d u c t p o l y n o m i a l t e r m is c o m p u t e d a n d a d d e d o n lines
18-20.
P r o g r a m 34.1. t p s K . m
% rbf = tpsK(x,y)
% Computes matrix for thin plate spline kernel K with
% linear polynomials cardinal on (0,0), (1,0), (0,1)
% Calls on: tps
1 function rbf = tpsK(x,y)
Define points for cardinal polynomials
2 ppoints = [0 0; 1 0; 0 1];
3 px = [ p l ( x ) p2(x) p3(x)];
4 py = [pl(y) p2(y) p3(y)];
5 r = DistanceMatrix(x,y); rbf = tps(l,r);
6 for k=l:3
7 r = DistanceMatrix(ppoints(k,:),y);
8 rbf = rbf - px(:,k)*tps(1,r);

9 r = DistanceMatrix(x,ppoints(k,:));
10 rbf = rbf - tps(l,r)*py(:,k)';
11 end
12 for j=l:3
13 for k=l:3
14 r = DistanceMatrix(ppoints(j,:),ppoints(k,:));
15 rbf = rbf + px(:,j)*py(:,k)'*tps(1,r);
16 end
17 end
18 for k=l:3
19 rbf = rbf + px(:,k)*py(:,k)';
20 end
21 return
% The c a r d i n a l polynomials
22 function w = pl(z)
23 w = 1 - z(:,1) - z(:,2);
24 return
25 function w = p2(z)
26 w = z(: ,1);
27 return
29 w = z(:,2);
30 return
Since P r o g r a m 3 4 . 1 p r o d u c e s t h e e n t i r e i n t e r p o l a t i o n (or e v a l u a t i o n ) m a t r i x , w e
can use P r o g r a m 2.1 a n d replace lines 13 a n d 14 b y
IM = tpsK(dsites,ctrs);
a n d lines 15 a n d 16 b y
EM = tpsK(epoints,ctrs);
i n o r d e r t o solve t h e i n t e r p o l a t i o n p r o b l e m i n t h i s case.
T h e m a t r i x C is o b t a i n e d i n a s i m i l a r fashion b y u s i n g a p r o g r a m t p s H . m t h a t
is i d e n t i c a l t o t p s K . m e x c e p t t h a t lines 1 8 - 2 0 are r e m o v e d . I n a d d i t i o n , we n e e d t o
remove the corner points ( 0 , 0 ) , ( 1 , 0 ) , and (0,1) f r o m the c t r s and d s i t e s i n the
driver program.
I n the first experiment ( i l l u s t r a t e d i n T a b l e 34.3) t h e p r o b l e m is f o r m u l a t e d o n
2
t h e u n i t s q u a r e [0, l ] . H e r e b o t h t h e n u m b e r o f p o i n t s a n d t h e s e p a r a t i o n d i s t a n c e
v a r y f r o m one r o w i n t h e t a b l e t o t h e n e x t . T h e t h r e e different c o l u m n s l i s t t h e
^ 2 - c o n d i t i o n n u m b e r s o f t h e i n t e r p o l a t i o n m a t r i x for t h e t h r e e different a p p r o a c h e s
mentioned above. W i t h this setup a l l three methods perform comparably.
2
Table 34.3 Condition numbers for different thin plate spline bases on [0, l ]
with increasing number of points and varying separation distance.
spacing h standard matrix reproducing kernel homogeneous matrix
1/8 3.515800e+003 1.839030e+004 7.583833e+003

1/16 3.893850e+004 2.651373e+005 1.108581e+005
1/32 5.136252e+005 4.000679e+006 1.686431e+006
1/64 7.618277e+006 6.202918e+007 2.626402e+007
I n t h e s e c o n d e x p e r i m e n t ( s h o w n i n T a b l e 3 4 . 4 ) t h e n u m b e r o f p o i n t s is k e p t f i x e d
2
at 5 x 5 e q u a l l y spaced p o i n t s . H o w e v e r , t h e d o m a i n is scaled t o t h e s q u a r e [0, a]
w i t h scale p a r a m e t e r a ( t h i s c a n e a s i l y be d o n e u s i n g t h e s a m e p r o g r a m s as a b o v e
b y i n t r o d u c i n g a scale p a r a m e t e r a t t h e a p p r o p r i a t e places, see also P r o g r a m 3 4 . 2 ) .
T h e effect o f t h i s is t h a t o n l y t h e s e p a r a t i o n d i s t a n c e qx changes f r o m one r o w t o
t h e n e x t i n t h e t a b l e . N o w , c l e a r l y , t h e t w o n e w m e t h o d s s h o w less d e p e n d e n c e o n
the separation distance, w i t h the c o n d i t i o n n u m b e r o f the homogeneous m a t r i x C
b e i n g c o m p l e t e l y i n s e n s i t i v e t o t h e r e - s c a l i n g as c l a i m e d e a r l i e r .
2
Table 34.4 Condition numbers for different thin plate spline bases on [0, a ] with
fixed number of 25 points and varying separation distance.
scale parameter a standard matrix reproducing kernel homogeneous matrix
0.001 2.434883e+008 8.463509e+008 5.493771e+002

0.01 2.436378e+006 8.464002e+006 5.493771e+002
0.1 2.517866e+004 8.513354e+004 5.493771e+002
1.0 3.645782e+002 1.366035e+003 5.493771e+002
10 1.874215e+006 1.260864e+003 5.493771e+002
100 1.151990e+011 1.139634e+005 5.493771e+002
1000 3.548239e+015 1.138572e+007 5.493771e+002
W e close t h i s s e c t i o n b y p o i n t i n g o u t t h a t I s k e a n c o - w o r k e r s t a k e a d v a n t a g e o f
t h e scale i n v a r i a n c e o f p o l y h a r m o n i c s p l i n e s ( a n d t h i n p l a t e s p l i n e s i n p a r t i c u l a r ) i n
t h e c o n s t r u c t i o n o f a n u m e r i c a l m u l t i s c a l e s o l v e r for t r a n s p o r t p r o b l e m s (see, e.g.,
[ B e h r e n s et al. ( 2 0 0 2 ) ] ) .
34.6 Effect of the " B e t t e r " B a s i s on t h e A c c u r a c y of the

Interpolant
I n t h i s s e c t i o n w e p r o v i d e a n e x a m p l e i l l u s t r a t i n g t h e s u r p r i s i n g fact t h a t for p o l y -
h a r m o n i c splines n o t o n l y t h e h o m o g e n e o u s k e r n e l K c a n be u s e d successfully f o r
p o o r l y scaled p r o b l e m s , b u t also t h e s t a n d a r d k e r n e l 3>.
2
E x a m p l e 3 4 . 4 . W e use t h e t h i n p l a t e s p l i n e basic f u n c t i o n <p(r) = r log r and
a scaled v e r s i o n o f F r a n k e ' s t e s t f u n c t i o n t o g e n e r a t e t e s t d a t a o n a 5 x 5 u n i f o r m
2
g r i d i n t h e square [ 0 , a ] as i n T a b l e 34.4. H o w e v e r , n o w t h e scale p a r a m e t e r a w i l l
- 9 9
range from 1 0 t o 1 0 . W e w i l l present c o n d i t i o n n u m b e r s a n d r o o t - m e a n - s q u a r e
e r r o r s c o m p u t e d o n a 40 x 40 u n i f o r m g r i d for t h e t h r e e different kernels discussed
previously. W e list o n l y t h e M A T L A B code f o r t h e h o m o g e n e o u s case since the
t w o o t h e r p r o g r a m s are v e r y s i m i l a r t o p r e v i o u s ones. T h e f u n c t i o n t p s H . m c a l l e d
b y P r o g r a m 34.2 is a l m o s t t h e same as P r o g r a m 34.1 l i s t e d above. The required
m o d i f i c a t i o n s are n o t e d t h e r e .
M a n y p a r t s o f P r o g r a m 34.2 are f a m i l i a r . However, i n order t o deal w i t h the
k e r n e l K a n d t h e associated m a t r i x C w e need t o define t h e special p o i n t s at w h i c h
t h e c a r d i n a l p o l y n o m i a l s are defined. T h i s is d o n e o n l i n e 9, w h e r e t h e special
p o i n t s are t a k e n as t h r e e corners o f t h e scaled u n i t square. Since t h e k e r n e l is t h e
zero f u n c t i o n w h e n centered at these p o i n t s t h e y need t o be r e m o v e d f r o m t h e set
o f centers. T h i s is a c c o m p l i s h e d o n lines 1 1 , 12 a n d 14. T h e s c a l i n g o f t h e p r o b l e m
h a p p e n s o n lines 8, 9 a n d 13. T h e scale also e n t e r s i n a n u m b e r o f o t h e r places such
as t h e d e f i n i t i o n o f t h e e v a l u a t i o n g r i d o n l i n e 15, c o m p u t a t i o n o f t h e r i g h t - h a n d side
o n lines 1 7 - 1 9 , a n d t h e c o m p u t a t i o n o f t h e e x a c t s o l u t i o n o n l i n e 26. I n c o n t r a s t
t o m o s t o f o u r o t h e r i n t e r p o l a t i o n p r o g r a m s here we c o m p u t e t h e i n t e r p o l a t i o n
a n d e v a l u a t i o n m a t r i c e s w i t h a single s u b r o u t i n e (c.f. t h e calls t o t p s H o n lines 20
a n d 2 2 ) . N o t e t h a t t h e scale p a r a m e t e r a is passed t o t p s H . E q u a t i o n s 34.9 a n d
34.7 for t h e s o l u t i o n a n d e v a l u a t i o n o f t h e i n t e r p o l a n t are i m p l e m e n t e d t o g e t h e r o n
l i n e 25. F i n a l l y , t h e t h r e e c a r d i n a l p o l y n o m i a l s are c o d e d o n lines 3 5 - 4 3 . Since these
p o l y n o m i a l s are defined o n t h e u n i t square t h e y need t o be c a l l e d w i t h re-scaled
a r g u m e n t s (cf. lines 17 a n d 2 3 ) .
P r o g r a m 34.2. RBFInterpolation2DtpsH.m
% RBFInterpolation2DtpsH
V, S c r i p t t h a t performs 2D TPS i n t e r p o l a t i o n w i t h h o m o g e n e o u s k e r n e l
% C a l l s on: tpsH
1 function RBFInterpolation2DtpsH
'/ D e f i n e F r a n k e ' s f u n c t i o n as testfunction
2 f l = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x - 2 ) . ~ 2 + ( 9 * y - 2 ) . ~ 2 ) / 4 ) ;
3 f 2 = @(x,y) 0 . 7 5 * e x p ( - ( ( 9 * x + l ) . ~ 2 / 4 9 + ( 9 * y + l ) . ~ 2 / 1 0 ) ) ;
4 f 3 = <3(x,y) 0 . 5 * e x p ( - ( ( 9 * x - 7 ) . ~ 2 + ( 9 * y - 3 ) . ~ 2 ) / 4 ) ;
5 f 4 = <3(x,y) 0 . 2 * e x p ( - ( ( 9 * x - 4 ) . ~ 2 + ( 9 * y - 7 ) . " 2 ) ) ;
6 t e s t f u n c t i o n = @(x,y) fl(x,y)+f2(x,y)+f3(x,y)-f4(x,y);
7 N = 25; g r i d t y p e = 'u';
8 a = le9;
9 ppoints = a*[0 0; 1 0; 0 1] ;
% Load d a t a points
10 name = s p r i n t f ( ' D a t a 2 D _ y o d y , s ' , N , g r i d t y p e ) ; load(name)
% Remove (0,0), (1,0), ( 0 , 1 ) t o work w i t h C m a t r i x
11a r e m o v e = [ f i n d ( d s i t e s ( : , 1 ) = = 0 & d s i t e s ( : , 2 ) = = 0 ) ; . . .
lib find(dsites(:,1)==1 & dsites(:,2)==0);...

11c find(dsites(:,1)==0 & dsites(:,2)==1)]
12 d s i t e s (remove, : ) = [] ;
/ S c a l e problem t o square [ 0 , a ] ~ 2
0
13 d s i t e s = a * d s i t e s ;
% L e t centers coincide with data s i t e s
14 c t r s = d s i t e s ;
15 n e v a l = 40; g r i d = l i n s p a c e ( 0 , a , n e v a l ) ;
% C r e a t e r i g h t - h a n d s i d e f o r homogeneous problem
17 DP = [ p l ( d s i t e s / a ) p 2 ( d s i t e s / a ) p 3 ( d s i t e s / a ) ] ' ;
18 d = t e s t f u n c t i o n ( p p o i n t s ( : , 1 ) / a , p p o i n t s ( : , 2 ) / a ) ;
19 r h s = t e s t f u n c t i o n ( d s i t e s ( : , 1 ) / a , d s i t e s ( : , 2 ) / a ) - DP'*d;
% Compute i n t e r p o l a t i o n m a t r i x f o r t h e s p e c i a l case of TPS
% n a t i v e space k e r n e l (no need t o add p o l y n o m i a l s )
20 IM = t p s H ( d s i t e s , c t r s , a ) ;
% Compute c o n d i t i o n number of i n t e r p o l a t i o n m a t r i x
0
21 f p r i n t f ('12-condition : /,e\n' , cond(IM))
22 EM = t p s H ( e p o i n t s , c t r s , a ) ;
23 EP = [ p i ( e p o i n t s / a ) p 2 ( e p o i n t s / a ) p 3 ( e p o i n t s / a ) ] ;
24 EM = [EM E P ] ;
V, Compute RBF i n t e r p o l a n t
25 Pf = EM * [ ( I M \ r h s ) ; d ] ;
% Compute e x a c t s o l u t i o n
26 exact = t e s t f u n c t i o n ( e p o i n t s ( : , l ) / a , e p o i n t s ( : , 2 ) / a ) ;
/ Compute e r r o r s on e v a l u a t i o n g r i d
0
28 r m s _ e r r = n o r m ( P f - e x a c t ) / n e v a l ;
29 f p r i n t f ('RMS e r r o r : %e\n' , r m s . e r r )
30 f p r i n t f ( 'Maximum e r r o r : '/,e\n', maxerr)
31 f v i e w = [160,20] ; viewing angles f o r p l o t
34 return
% The c a r d i n a l p o l y n o m i a l s
35 f u n c t i o n w = p l ( z )
36 w = 1 - z ( : ,1) - z ( : , 2 ) ;
37 return
39 w = z(:,1);
40 return
42 w = z(:,2);
43 return
I n T a b l e 34.5 we l i s t t h e r o o t - m e a n - s q u a r e e r r o r s r e s u l t i n g f r o m t h e t h r e e dif-
ferent i n t e r p o l a t i o n m e t h o d s . T h e s c a l i n g o f t h e d o m a i n w a s chosen m o r e e x t r e m e
t h a n i n T a b l e 34.4 so t h a t t h e s e n s i t i v i t y o f t h e r e p r o d u c i n g k e r n e l K becomes
9
c l e a r l y v i s i b l e . I t s c o n d i t i o n n u m b e r o f t h e i n t e r p o l a t i o n m a t r i x for a 1 0 ~ was
9
5 . 3 5 4 1 3 4 e + 0 1 8 , w h i l e for a = 10 i t was 6.994062e+019. W h i l e b o t h o f these
c o n d i t i o n n u m b e r s are c l e a r l y v e r y h i g h a n d t h e r e f o r e i n d i c a t e t h a t we m i g h t ex-
pect n u m e r i c a l difficulties s o l v i n g a p r o b l e m o n these l e n g t h scales, t h e o t h e r t w o
m e t h o d s ( s t a n d a r d T P S basis f u n c t i o n s a n d t h e h o m o g e n e o u s k e r n e l K ) perform
p e r f e c t l y t h r o u g h o u t t h e e n t i r e r a n g e o f scalings. Moreover, the condition n u m -
bers for t h e s t a n d a r d T P S i n t e r p o l a t i o n m a t r i x are m u c h h i g h e r t h a n for t h e K
9 9
k e r n e l : 7 . 2 9 9 4 0 8 e + 0 2 1 for a = 1 C T , a n d e v e n 2 . 7 4 9 5 3 7 e + 0 3 8 for a = 1 0 . N e v e r -
theless, t h e s t a n d a r d T P S i n t e r p o l a n t does not suffer f r o m i n s t a b i l i t y due t o t h i s
i l l - c o n d i t i o n i n g . T h e same is t r u e for a l l o t h e r tests we h a v e p e r f o r m e d w i t h s t a n -
d a r d p o l y h a r m o n i c spline i n t e r p o l a n t s (such as, e.g., t h e n o r m basic f u n c t i o n ) .
2
Table 34.5 R M S errors for different thin plate spline interpolants on [ 0 , a ] with
fixed number of 25 points and varying separation distance.
scale parameter o standard matrix reproducing kernel homogeneous matrix
IO- 9
2.969478e-002 NAN 2.969478e-002
ICR 6
2.969478e-002 2.970740e-002 2.969478e-002
3
10- 2.969478e-002 2.969478e-002 2.969478e-002
1.0 2.969478e-002 2.969478e-002 2.969478e-002
10 3
2.969478e-002 2.969478e-002 2.969478e-002
6
10 2.969478e-002 2.969218e-002 2.969478e-002
9
10 2.969478e-002 1.207446e+003 2.969478e-002
Chapter 35
Other Efficient Numerical Methods
I n earlier c h a p t e r s we have a l r e a d y m e n t i o n e d v a r i o u s a l g o r i t h m s for meshfree scat-

t e r e d d a t a i n t e r p o l a t i o n t h a t are m o r e efficient t h a n t h e s t r a i g h t f o r w a r d s o l u t i o n
o f t h e linear s y s t e m o b t a i n e d b y e n f o r c i n g t h e i n t e r p o l a t i o n c o n d i t i o n s . I n par-
t i c u l a r , we suggested t h e use o f t h e n o n - u n i f o r m fast F o u r i e r t r a n s f o r m ( N F F T )
for fast e v a l u a t i o n o f g l o b a l l y s u p p o r t e d f u n c t i o n s , a fixed level i t e r a t i v e a l g o r i t h m
based o n a p p r o x i m a t e M L S a p p r o x i m a t i o n , t h e greedy a l g o r i t h m o f S c h a b a c k a n d
Wendland, the Faul-Powell a l g o r i t h m , and the preconditioned G M R E S m e t h o d of
B e a t s o n et a l .
W e n o w a d d t h r e e m o r e n u m e r i c a l t e c h n i q u e s t h a t c a n be used t o m a k e the
c o m p u t a t i o n w i t h g l o b a l l y s u p p o r t e d f u n c t i o n s o n large d a t a sets m o r e efficient
a n d also m o r e stable. I n t h e first t w o sections o f t h i s c h a p t e r we discuss t h e fast
m u l t i p o l e m e t h o d a n d fast t r e e codes, a n d h o w these m e t h o d s c a n be a d a p t e d t o
r a d i a l basis f u n c t i o n s . I n t h e t h i r d s e c t i o n we present a b r i e f i n t r o d u c t i o n t o d o m a i n
d e c o m p o s i t i o n m e t h o d s , w h i c h n o t o n l y m a k e t h e s o l u t i o n o f large i n t e r p o l a t i o n
p r o b l e m s m o r e efficient, b u t also p r o v i d e a w a y t o a v o i d t h e i l l - c o n d i t i o n i n g issue
b y b r e a k i n g t h e large p r o b l e m i n t o m a n y w e l l - c o n d i t i o n e d s m a l l e r ones.
35.1 T h e Fast Multipole Method
A n o t h e r t e c h n i q u e for d e a l i n g w i t h fast s u m m a t i o n p r o b l e m s is k n o w n as t h e fast

multipole method. I t was first p r o p o s e d b y G r e e n g a r d a n d R o k h l i n i n t h e l a t e 1980s
(see, e.g., t h e o r i g i n a l p a p e r [ G r e e n g a r d a n d R o k h l i n (1987)], t h e p o p u l a r discussion
[ G r e e n g a r d (1994)], or t h e s h o r t course t a i l o r e d t o r a d i a l basis f u n c t i o n s [Beatson
and Greengard ( 1 9 9 7 ) ] ) . T h i s m e t h o d has q u i c k l y b e c o m e r a t h e r p o p u l a r i n t h e
c o m p u t a t i o n a l sciences. T h e b r e a k t h r o u g h a c c o m p l i s h m e n t o f t h i s a l g o r i t h m was
t h e a b i l i t y t o p e r f o r m fast e v a l u a t i o n s o f s u m s o f t h e t y p e
V (x)
f = J2 k$(x,
c
x ),
k x e R s
.
k=l
321
I n p a r t i c u l a r , M s u c h e v a l u a t i o n s c a n be p e r f o r m e d i n 0(M log N) (or e v e n O(M))

o p e r a t i o n s i n s t e a d o f t h e s t a n d a r d 0(MN) o p e r a t i o n s for a n a i v e i m p l e m e n t a t i o n o f
t h e s u m m a t i o n . T h e n o n - u n i f o r m fast F o u r i e r t r a n s f o r m o f C h a p t e r 28 was a b l e t o
d o t h i s also, a n d i n a f a i r l y g e n e r a l w a y for a v e r y l a r g e class o f kernels <&. H o w e v e r ,
t h e fast m u l t i p o l e m e t h o d is a l i t t l e o l d e r a n d i t m a y be m o r e efficient t h a n the
N F F T since special e x p a n s i o n s are used t h a t are chosen w i t h t h e p a r t i c u l a r k e r n e l
<& i n m i n d .
W e w i l l n o w o u t l i n e t h e basic i d e a o f t h e fast Gauss transform [Greengard and
S t r a i n ( 1 9 9 1 ) ] . T h i s t r a n s f o r m c a n be a p p l i e d d i r e c t l y t o t h e a p p r o x i m a t e m o v i n g
least squares a p p r o x i m a n d s b a s e d o n G a u s s i a n s u s e d i n e a r l i e r c h a p t e r s (see the
numerical experiments reported i n Table 35.1 below). T h e higher-order Laguerre-
G a u s s i a n kernels, h o w e v e r , r e q u i r e a c o m p l e t e l y n e w d e r i v a t i o n . U s i n g o u r s t a n d a r d
a b b r e v i a t i o n e = 1 / ( y / V h ) , w e are n o w i n t e r e s t e d i n a fast s u m m a t i o n t e c h n i q u e for
M simultaneous evaluations o f the Gaussian quasi-interpolant (or discrete Gauss
transform)
GfiVj) = J2f( ^ ~ ~ > x e MVj Xk)r

j = 1,..., M . (35.1)
fc=i
I n [ G r e e n g a r d a n d S t r a i n (1991)] s u c h a n a l g o r i t h m w a s d e v e l o p e d , a n d i n [ S t r a i n
(1991)] a m o d i f i c a t i o n was s u g g e s t e d t o cover also t h e case o f v a r i a b l e scales e k as
needed w i t h q u a s i - i n t e r p o l a t i o n a t s c a t t e r e d sites o r w i t h v a r i a b l e shape p a r a m e t e r s .
O n e o f t h e c e n t r a l i n g r e d i e n t s for t h e fast G a u s s t r a n s f o r m are t h e multivariate
Hermite functions h a defined as
2
h (x)
a = (-i)ll> -IMI , e (35.2)
s
w h e r e ct = (a ,...,
x a) s G N is a m u l t i - i n d e x . T h e s e f u n c t i o n s are r e l a t e d t o t h e
multivariate Hermite polynomials H a via
s
2
H {x)
a = \ { H (x ) ad d = e^ h (x) a (35.3)
d=i
(see, e.g., t h e u n i v a r i a t e f o r m u l a ( 6 . 1 . 3 ) i n [ A n d r e w s et al. ( 1 9 9 9 ) ] ) . I t is benefi-

cial t h a t the H e r m i t e functions can be evaluated recursively using the (univariate)
recurrence r e l a t i o n
h i(x)
n+ = 2xh {x) n 2nh -. (x), n x n = 1, 2 , . . . ,
| x 2
h (x)
0 = e- l , hx(x) = 2xe~W\
w h i c h f o l l o w s i m m e d i a t e l y f r o m ( 3 5 . 3 ) a n d t h e r e c u r s i o n r e l a t i o n for H e r m i t e p o l y -
n o m i a l s (see, e.g., f o r m u l a ( 6 . 1 . 1 0 ) i n [ A n d r e w s et al. ( 1 9 9 9 ) ] ) .
The algorithm of Greengard a n d S t r a i n is based o n t h r e e basic e x p a n s i o n s
w h i c h w e l i s t b e l o w as T h e o r e m s 3 5 . 1 - 3 5 . 3 (see [Greengard a n d S t r a i n (1991);
G r e e n g a r d a n d S u n ( 1 9 9 8 ) ] ) . T h e m a i n effect o f these e x p a n s i o n s is t h e f a c t t h a t t h e
v a r i a b l e s yj a n d x k w i l l b e s e p a r a t e d . T h i s is t h e f u n d a m e n t a l " t r i c k " used w i t h a l l
35. Other Efficient Numerical Methods 323
fast s u m m a t i o n a l g o r i t h m s (see o u r discussion o f t h e N F F T based fast s u m m a t i o n

method i n Chapter 28). T h i s w i l l a l l o w for t h e p r e - c o m p u t a t i o n a n d storage o f
c e r t a i n moments below.
s
T h e first step i n t h e a l g o r i t h m is t o scale t h e p r o b l e m t o t h e u n i t b o x [0, l ] and
t h e n s u b d i v i d e t h e u n i t b o x i n t o s m a l l e r boxes B a n d C w h i c h u s u a l l y coincide.
T h e y can, however, also differ. T h e boxes B c o n t a i n t h e sources x k (i.e., our
centers), a n d t h e boxes C t h e targets yj (i.e., our evaluation points). F o r each
source b o x B one t h e n d e t e r m i n e s i t s interaction region IR(B). The interaction
r e g i o n o f B is a set o f nearest n e i g h b o r s o f B such t h a t t h e e r r o r o f t r u n c a t i n g
t h e s u m over a l l boxes is b e l o w a c e r t a i n t h r e s h o l d . D u e t o t h e fast decay o f t h e
s
Gaussians i t is suggested (see [ G r e e n g a r d a n d S u n (1998)]) t o use t h e 9 nearest
s
neighbors for single p r e c i s i o n a n d t h e 1 3 nearest n e i g h b o r s for d o u b l e p r e c i s i o n .
T h e o r e m 3 5 . 1 . Let IB be the index set denoting the sources x k that lie in a box
B with center XB and side length 1/e, and let yc be the center of the target box
C IR(B) of radius r c containing the targets yj. Then the Gaussian field due to
the sources in B ,
G?\vi)= E /(*fc)e- ||e(v

'-" fc)l|a
,
kei B
has the following Taylor expansion about yc '

a a 35 4
Qf\vo) = E <* - yc)) > ( -)
cx>0
where the coefficients a a are given by
a n = E f(x )h k a (e(x k - yc))

Oil
kei B
The error E T ( P ) due to truncating the series (35.4) after the p-th order terms
satisfies the bound
a
\ET( )\
P = I E" ~ vc)T I < ( 1 . 0 9 ) ' F < * > - = =
where = f c e 7 s \f(x )\.

k
Here we used t h e m u l t i - i n d e x n o t a t i o n at > 0 t o d e n o t e t h e c o n s t r a i n t s ad > 0

for a l l d 1,..., s. M o r e generally, for some integer p we say ex. > p i f ad > p for
all d = 1 , . . . , s. T h i s i m p l i e s t h a t we have a > p for some integer p, i f ex. > p a n d
a d > p for some d. W e also use ex > (3 i f a d > (3d for a l l d = 1 , . . . , s.
T h e e x p a n s i o n (35.4) w i l l be used i n t h e case w h e n t h e source b o x B c o n t a i n s
r e l a t i v e l y few sources, b u t t h e t a r g e t b o x C c o n t a i n s m a n y t a r g e t s .
B y reversing t h e role o f t h e H e r m i t e f u n c t i o n s a n d t h e s h i f t e d m o n o m i a l s one c a n
s
w r i t e a single G a u s s i a n as a m u l t i v a r i a t e H e r m i t e e x p a n s i o n a b o u t a p o i n t ZQ e M ,
i.e.,
e - M y j - X k ) l l * = J2 J _ ( ( Xk - Zo ))<* h a (e(Vj - z )).

0 (35.5)
T h i s idea is used i n
T h e o r e m 35.2 (Far-field e x p a n s i o n ) . Let I B be the index set denoting the

sources Xk that lie in a box B with center XB and side length 1/e. Then the Gaus-
sian field due to the sources in B,
B) M y k n
G (yj)
f = E f M e - ^ \
kei B
is equal to an Hermite expansion about x B '

b h XB 35 6
G?\v,) = E<*<* ^ ~ ^
tx>0
( -)
The moments b
a are given by
b a = . E x
f( k) x
((k - x B ) ) a
keiB
The error EH(P) due to truncating the series (35.6) after p-th order terms satisfies
the bound
P+I
=|E -

'). (r) c
~ <* B))
a
\EH{P)\ B H X
I < (1.09) F^
1 er c
ot>p
T h e o r e m 35.2 is used w h e n B c o n t a i n s m a n y sources, b u t C o n l y few t a r g e t s .

F i n a l l y , i n t h e case w h e n b o t h B a n d C c o n t a i n r e l a t i v e l y m a n y p o i n t s we use
T h e o r e m 35.3 ( T r a n s l a t i o n o p e r a t i o n ) . Let the sources Xk lie in a box B with

center XB and side length 1/e and let yj be an evaluation point in a box C with
center yc- Then the corresponding truncated Hermite expansion (35.6) can be ex-
panded as a Taylor series of the form
G f (yj) { BC)
= T,c0Wyi-yc)f- (35.7)
The coefficients cp are given by
(-1)101
= -pi 2^ <* <*+p ( ( B -
c
/3 b h
X
yc)),
a<p
with b a as in Theorem 35.2. The error ET{P) due to truncating the series (35.7)
after p-th order terms satisfies the bound
P+I
1 er
t^p V(P+1)! c
T h e o r e m 35.3 is based o n t h e m u l t i v a r i a t e T a y l o r series e x p a n s i o n o f t h e H e r m i t e

functions h , a i.e.,
h, (e(yj - x ) ) = Yl ~ g i ((i e y
~ y c
^ ha
+P
( ( X B
~ y c
^
B
(3>0 P'
N o t e t h a t t h e e r r o r e s t i m a t e s i n t h e o r i g i n a l p a p e r o n t h e fast Gauss t r a n s f o r m
[ G r e e n g a r d a n d S t r a i n (1991)] were i n c o r r e c t . I n t h e m e a n t i m e a n u m b e r o f o t h e r
a u t h o r s have p r o v i d e d a l t e r n a t e e r r o r b o u n d s i n t h e i r p a p e r s (see, e.g., [Baxter
a n d Roussos (2002); F l o r e n c e a n d v a n L o a n (2000); G r e e n g a r d a n d S u n ( 1 9 9 8 ) ;
W e n d l a n d (2004)]).
F o r I D c a l c u l a t i o n s o n t h e o r d e r o f p = 20 t e r m s are r e q u i r e d t o achieve d o u b l e
p r e c i s i o n accuracy. F o r t h e 2 D case one c a n get b y w i t h a s m a l l e r value o f p ( a b o u t
s
15), b u t t h e n u m b e r o f t e r m s is o f course m u c h h i g h e r ( o n t h e o r d e r o f p for
s-dimensional problems).
T h e basic o u t l i n e o f t h e a l g o r i t h m is as follows:
A l g o r i t h m 35.1. Fast Gauss t r a n s f o r m

s
(1) I f necessary, scale t h e p r o b l e m so t h a t t h e coarsest b o x Bo [0, l ] . S u b d i v i d e
Bo i n t o smaller boxes w i t h side l e n g t h 1/e p a r a l l e l t o t h e axes. A s s i g n each
source x k t o t h e b o x B i n w h i c h i t lies a n d each e v a l u a t i o n p o i n t yj t o t h e b o x
C i n w h i c h i t lies.
(2) Choose p so t h a t t h e t r u n c a t i o n e r r o r satisfies t h e desired accuracy, a n d for
each b o x B c o m p u t e a n d store t h e coefficients (or m o m e n t s )
boc =
'
E kei
x
f( *>) (
( X k
~ X
B)) L
, OL<P,
of i t s H e r m i t e e x p a n s i o n ( 3 5 . 6 ) .
(3) For each e v a l u a t i o n b o x C, d e t e r m i n e i t s i n t e r a c t i o n r e g i o n IR(C).
(4) For each e v a l u a t i o n b o x C t r a n s f o r m a l l H e r m i t e expansions i n source boxes
w i t h i n t h e i n t e r a c t i o n r e g i o n IR(C) i n t o a single T a y l o r e x p a n s i o n u s i n g ( 3 5 . 7 ) ,
i.e.,
/9<P
where
C
P = ^-4jj E E BelR(C) at<p
6
^ + / 3 (e(x B - yc))
For a s m a l l n u m b e r o f p o i n t s d i r e c t s u m m a t i o n is m o r e efficient t h a n t h e fast

t r a n s f o r m . U n f o r t u n a t e l y , t h e v a l u e o f t h e "crossover p o i n t " g r o w s w i t h t h e space
d i m e n s i o n s. T h i s makes t h e fast Gauss t r a n s f o r m i n i t s basic f o r m v i r t u a l l y unus-
able for 3 D a p p l i c a t i o n s .
N o t e t h a t t h e a l g o r i t h m p r e s e n t e d here does n o t use a h i e r a r c h i c a l d e c o m p o s i t i o n
of space as is t y p i c a l for so-called tree codes, as w e l l as m a n y o t h e r m o r e general fast
m u l t i p o l e a l g o r i t h m s . I n t h e a l g o r i t h m a b o v e t h e i n t e r a c t i o n r e g i o n is d e t e r m i n e d
s i m p l y based o n t h e fast d e c a y o f t h e G a u s s i a n .
C l e a r l y , t h e m a j o r i t y o f t h e w o r k has t o be p e r f o r m e d i n step 4. T h e p e r f o r m a n c e
of t h i s step c a n be i m p r o v e d b y u s i n g plane wave expansions t o d i a g o n a l i z e the
t r a n s l a t i o n o p e r a t o r s (see [ G r e e n g a r d a n d S u n ( 1 9 9 8 ) ] ) . I n o r d e r t o keep matters

as s i m p l e as possible, w e w i l l n o t discuss t h i s f e a t u r e .
A m o r e c o m p l e t e a l g o r i t h m ( d e s i g n e d for r a d i a l basis f u n c t i o n i n t e r p o l a t i o n w i t h
m u l t i q u a d r i c s a n d t h i n p l a t e splines) has b e e n d e v e l o p e d b y B e a t s o n a n d c o - w o r k e r s
(see, e.g., [ B e a t s o n a n d N e w s a m ( 1 9 9 2 ) ; C h e r r i e et al. ( 2 0 0 2 ) ] ) .
For t h e n u m e r i c a l e x p e r i m e n t s i n T a b l e 3 5 . 1 w e used t h e C-code FGT w h i c h
c a n also be used as a M E X - f i l e w i t h M A T L A B . T h e code w a s w r i t t e n b y A d a m F l o -
rence a n d c a n be o b t a i n e d a t h t t p : / / w w w . c s . C o r n e l l . e d u / a f l o r e n c / r e s e a r c h / -
fgt.html (see also [ F l o r e n c e a n d v a n L o a n ( 2 0 0 0 ) ] ) . T h e n u m e r i c a l results pre-
sented i n T a b l e 3 5 . 1 w e r e o b t a i n e d b y p e r f o r m i n g q u a s i - i n t e r p o l a t i o n o f t h e f o r m
^'()-(^)-'/E/(*).(^),
K 1
l
w i t h a G a u s s i a n <E> o n N = 2 + 1, 2 , 3 , 4 , . . . , 18, e q u a l l y spaced p o i n t s i n [ 0 , 1 ]
w i t h the mollified test f u n c t i o n
,-(*-2) /42 3 2
(x + l ) / 4 9 1 2
(*-7) /4 1 -(x-4) s
f(x) = 15e^^-D a
+ 6
4 ' 44" " +
" 22" '
e
5'
A l l e r r o r s were c o m p u t e d o n M = 524289 e q u a l l y spaced p o i n t s i n [ 0 , 1 ] . In
t h e " r a t e " c o l u m n we l i s t t h e n u m b e r r a t e = l n ( e ^ _ i / e ^ ) / l n 2 c o r r e s p o n d i n g t o t h e
T&te
e x p o n e n t i n t h e 0(h ) n o t a t i o n . O t h e r p a r a m e t e r s w e r e T> = 4, a n d t h e d e f a u l t
values for t h e FGT code ( i . e . , R = 0 . 5 ) . A l l t i m e s w e r e m e a s u r e d i n seconds.
Table 35.1 I D quasi-interpolation using fast Gauss transform.
direct fast
N max-error rate time max-error rate time speedup
5 3.018954e-00 1.93 5 495125e-00 1.07 1.80

9 2.037762e-00 0.57 3.40 2 037762e-00 1 43 5.31 0.64
17 9.617170e-01 1.08 6.39 9 617170e-01 1 08 5.33 1.20
33 3.609205e-01 1.41 12.28 3 609205e-01 1 41 5.35 2.30
65 1.190192e-01 1.60 24.72 1 190192e-01 1 60 5.39 4.59
129 3.354132e-02 1.83 53.38 3 354132e-02 1 83 5.46 10.14
257 8.702868e-03 1.95 113.35 8 702868e-03 1 95 5.61 20.20
513 2.196948e-03 1.99 226.15 2 196948e-03 1 99 5.94 38.07
1025 450* 5 505832e-04 2 00 6.67 67.47
2049 900* 1 377302e-04 2 00 7.87 114.36
4097 1800* 3 443783e-05 2 00 10.56 170.45
8193 3600* 8 609789e-06 2 00 15.78 228.14
16385 7200* 2 152468e-06 2 00 26.27 274.08
32769 14400* 5 381182e-07 2 00 47.39 303.86
65537 28800* 1 .345296e-07 2 .00 89.91 320.32
131073 57600* 3 .363241e-08 2 .00 174.74 329.63
262145 115200* 8 .408103e-09 2 .00 343.59 335.28
A n asterisk * o n t h e entries i n t h e lower p a r t o f the "direct" c o l u m n indicates

e s t i m a t e d t i m e s . T h e fast Gauss t r a n s f o r m y i e l d s a s p e e d u p o f r o u g h l y a f a c t o r o f
300. A n o t h e r w a y t o i n t e r p r e t these r e s u l t s is t h a t for r o u g h l y t h e same a m o u n t

o f w o r k we c a n o b t a i n a n answer w h i c h is a b o u t 100000 t i m e s m o r e a c c u r a t e . T h e
2
p r e d i c t e d 0(h ) convergence o f t h e G a u s s i a n q u a s i - i n t e r p o l a n t (c.f. C h a p t e r 26) is
p e r f e c t l y i l l u s t r a t e d b y t h e entries i n t h e " r a t e " c o l u m n s .
35.2 Fast Tree Codes
A n a l t e r n a t i v e t o fast m u l t i p o l e m e t h o d s are so-called fast tree codes. These k i n d

o f a l g o r i t h m s o r i g i n a t e d i n c o m p u t a t i o n a l c h e m i s t r y . For t h e i n t e r e s t e d reader w e
r e c o m m e n d recent m a t h e m a t i c a l p a p e r s b y K r a s n y a n d c o - w o r k e r s (e.g., [Duan and
K r a s n y ( 2 0 0 1 ) ; L i n d s a y a n d K r a s n y ( 2 0 0 1 ) ] ) . A n a d v a n t a g e o f fast t r e e code m e t h -
ods is t h a t t h e y m a k e use o f s t a n d a r d T a y l o r e x p a n s i o n s i n s t e a d o f t h e s p e c i a l i z e d
e x p a n s i o n s t h a t are used i n t h e c o n t e x t o f t h e fast m u l t i p o l e e x p a n s i o n s o f t h e
p r e v i o u s s e c t i o n (such as, e.g, i n t e r m s o f H e r m i t e f u n c t i o n s , s p h e r i c a l h a r m o n -
ics, s p h e r i c a l H a n k e l f u n c t i o n s , p l a n e waves, o r h y p e r g e o m e t r i c f u n c t i o n s [ C h e r r i e
et al. (2002)]). T h i s simplifies t h e i r i m p l e m e n t a t i o n . However, their convergence
p r o p e r t i e s are p r o b a b l y n o t as g o o d as t h o s e o f fast m u l t i p o l e e x p a n s i o n s .
W e n o w present a v e r y g e n e r a l d i s c u s s i o n o f fast s u m m a t i o n v i a T a y l o r e x p a n -
sions. T h e p r e s e n t a t i o n o f t h i s m a t e r i a l is m o t i v a t e d b y t h e w o r k o f K r a s n y a n d
co-workers (see, e.g., [ D u a n a n d K r a s n y ( 2 0 0 1 ) ; L i n d s a y a n d K r a s n y ( 2 0 0 1 ) ] ) as
w e l l as t h e a l g o r i t h m for t h e fast Gauss t r a n s f o r m r e v i e w e d i n t h e p r e v i o u s s e c t i o n .
Since we are i n t e r e s t e d i n m a n y s i m u l t a n e o u s e v a l u a t i o n s o f o u r q u a s i - i n t e r p o l a n t s
(or o t h e r r a d i a l basis f u n c t i o n e x p a n s i o n ) , we s p l i t t h e set o f M e v a l u a t i o n p o i n t s
yj i n t o g r o u p s ( c o n t a i n e d i n boxes C w i t h centers yc)- W e also s p l i t t h e N data
l o c a t i o n s Xk i n t o boxes B w i t h centers X B , a n d use t h e i n d e x set IB t o d e n o t e t h e
points i n B .
I n o r d e r t o set t h e stage for a fast s u m m a t i o n o f t h e q u a s i - i n t e r p o l a n t
N
Qfiyj) =
%2f( k)&(vj
1
x
- x)
k
= EE /(**)*(Vi - **) (35-8)

B kei
B
w i t h g e n e r a t i n g f u n c t i o n <& w e r e q u i r e t h e m u l t i v a r i a t e T a y l o r e x p a n s i o n o f a b o u t
s
a p o i n t ZQ R , i.e.,
*(*) = E D ^ ( z ) \ z = Z 0
{ z
' ^ r
, (35.9)
a>0
w h e r e ot is a m u l t i - i n d e x . N o w as for t h e fast Gauss t r a n s f o r m we consider

t h r e e basic expansions.
T h e o r e m 35.4 ( T a y l o r Series E x p a n s i o n about C e n t e r s of T a r g e t B o x e s ) .

Let IB be the index set denoting the sources Xk that lie in a box B with center XB,
and let yc be the center of the target box C containing an evaluation point yj. Then
the quasi-interpolant due to sources in B
Q?\vi) E = HxkMyj-Vk)
kei B
can be written as a Taylor expansion about yc:
oc>0
where
dry t ^ l . ^ f(x )T (y ,x )
k a c k
a!
kei B
withT (y ,x )
a c k = (-l)WD$(z)\ _ . z=yc Xk
Proof. W e c o m b i n e t h e c o n t r i b u t i o n for t h e source b o x B o f (35.8) w i t h ( 3 5 . 9 ) ,

a n d let z yj x k a n d ZQ yc x . T h e n (35.8) becomes
k
,{B)(,..\- f(. \ n a ^ ^ i iVi ~ Vc) a
Qf(yj) = E /(**) E D *(z)\ =y -

a
z c
ex
kel B oc>o
O L
U s i n g t h e a b b r e v i a t i o n T (yc,x ) a k = (\)^D ^(z)\ z = y c - X k we can rewrite this
as
Q f (yj){
)
= T , * ( y - y c ) j
a
,
a>0
where
E f{*k)T {y ,x ). a c k
cx\
kei B
E x a m p l e 3 5 . 1 . I f we t a k e 3>(cc) = e ^ then
T (y ,x )
a c k = h (yc a - x) k = h (xk a - yc),
a n d T h e o r e m 35.4 is e q u i v a l e n t t o T h e o r e m 3 5 . 1 g i v e n a b o v e .
W e c a n see t h a t t h e T a y l o r e x p a n s i o n has a l l o w e d us t o s e p a r a t e t h e e v a l u a t i o n
p o i n t s yj from the data points x . k
T h e o r e m 35.5 ( T a y l o r Series E x p a n s i o n a b o u t C e n t e r s of S o u r c e Boxes).

Let IB be the index set denoting the sources x k that lie in a box B with center XB-
Then the quasi-interpolant due to sources in B
B
Qf \vj)= E fMHyj-Xk)
kei B
can be written as a reversed Taylor expansion about XB-'

( ) b T ai
Q f (vj) = J2 <* <*(yj> B),
ot>0
with the m o m e n t s b a given by
a
be* = ^ 7 E f(xk)(x k - x B ) ,
cx\
kl B
a
andT { ,x )
a yj B = (-l)WD $(z)\ Vj-x z=
B
r
Proof. W e c o m b i n e t h e c o n t r i b u t i o n for t h e source b o x B o f (35.8) w i t h ( 3 5 . 9 ) ,

a n d let z = y 3 - x k and z 0 = y 3 - x . T h e n (35.8) becomes
B
fc6/ B tx>0
| o t | Q :
U s i n g t h e a b b r e v i a t i o n T (y ,x ) a 3 B = (-l) iP $(2:)| 2 ; = : y ._
j : C s w e c a n reverse t h e
r o l e o f t h e T a y l o r coefficients a n d t h e p o l y n o m i a l s t o w r i t e t h i s as
6 r
Q/ (Wj) = Z ) ( f i , a 5 B ) ,
B)
<x>0
with
& a = x a
a! ^ f ( k ) ( x k ~ x B ) .
' kei B j-j
- 2
E x a m p l e 3 5 . 2 . U s i n g $(x) = e "*" t h i s is e q u i v a l e n t t o T h e o r e m 35.2.
The moments b a c a n be p r e - c o m p u t e d a n d s t o r e d d u r i n g t h e s e t u p phase o f t h e

algorithm.
T h e o r e m 35.6 ( C o n v e r s i o n ) . Let I B be the index set denoting the sources x k
that lie in a box B with center x B , and let yc be the center of the target box C
containing y.
3 Then a fast summation formula for the quasi-interpolant
N
Qfivj) = ^2f(xk)$(yj - x k )
k=l
can be given as an expansion about yc'
where
(-1)1^1 ^ ,
C
P = m 2^ 2^T B(yc,x )b ,
a+ B a
T B(yc,x )
a+ B = (l)\ a + f 3
\D a + ( 3
<!<>(z)\ =y -. z c X B , and the moments b a are as in
Theorem 35.5.
Proof. W e c o m b i n e (35.8) w i t h ( 3 5 . 9 ) , a n d n o w replace z b y y 3 x k and z 0 by

Vc x B . T h e n (35.8) becomes
QfM = E E /(-*) E ^*(*)i

-0 ^ - " - ( f ".
B kel B >0
U s i n g t h e a b b r e v i a t i o n T (yc, a x) B = (l)^D ^(z)\ ^ OL

z=yc XB along w i t h the m u l -
t i v a r i a t e b i n o m i a l t h e o r e m w e c a n r e w r i t e t h i s as
Qfto) = E /(**)E ( - i ) M T
( y
' " B )
*
B kl B >0
3<Q
= E E ( - u | c
" ^ f e c , s s ) E(-i) : 2v^ >
< w(>,
V
f
,c
<*>0 B /3<a
E / ( ^ ) ^ ^ -
I n fact, we c a n i n t r o d u c e t h e m o m e n t s o f T h e o r e m 35.5 a n d w r i t e
V
a>0 B /3<a
where
6
0 =
E f ( x
k ) ( x
k - XB) - 13
p
' k e i B
A fast a l g o r i t h m is n o w o b t a i n e d b y t r u n c a t i n g t h e i n f i n i t e series after t h e p - t h

order terms, i.e.,
[ H J
Oi<p B B<cx '
U s i n g t h e fact t h a t
a b a 3 = b a a a = b a a
E E ~ E E E E <*+0'
ct <p B<cx o;<p ct</3<p a<p c*+/3<p
w h i c h c a n be v e r i f i e d b y a s i m p l e r e a r r a n g e m e n t o f t h e s u m m a t i o n s a n d a n i n d e x
transformation, we o b t a i n (interchanging the role of a a n d (3) t h e f o l l o w i n g fast
summation formula:
| a |
G/(i/i)E E (-i) 4iE(- ) 1 | o t + / 3 r
' +/3(yc,a:B)6c,(yi-yc) . / 3
0<pa+3<p B
T h i s is e q u i v a l e n t t o t h e s t a t e m e n t o f t h e t h e o r e m .
- 3 3 2
E x a m p l e 3 5 . 3 . U s i n g $(cc) = e " " T h e o r e m 35.6 is a l m o s t e q u i v a l e n t t o T h e o -
r e m 35.2. H o w e v e r , o u r a l t e r n a t e f o r m u l a is m o r e efficient since o n l y H e r m i t e f u n c -
t i o n s u p t o o r d e r p are r e q u i r e d (as o p p o s e d t o o r d e r 2p i n t h e Greengard/Strain
v e r s i o n ) . T h i s g a i n is a c h i e v e d b y u s i n g t h e b i n o m i a l t h e o r e m i n s t e a d o f a second
T a y l o r e x p a n s i o n . T h e H e r m i t e series e x p a n s i o n used i n t h e t r a d i t i o n a l fast Gauss
t r a n s f o r m is e q u i v a l e n t t o a T a y l o r e x p a n s i o n .
N o t e t h a t t h e T a y l o r coefficients T (yc, a XB) d e p e n d o n l y o n t h e b o x centers yc

and XB-
I n order t o m a k e t h e a l g o r i t h m efficient one w i l l use a d e c i s i o n r u l e (as i n S t r a i n ' s

code for t h e fast Gauss t r a n s f o r m ) t o decide w h e n t o use w h i c h o f t h e t h r e e e x p a n -
sions. E r r o r e s t i m a t i o n is v e r y s i m i l a r t o G r e e n g a r d / S t r a i n . T h e o n l y difference is
t h a t one needs b o u n d s o n t h e T a y l o r coefficients i n s t e a d o f t h e H e r m i t e f u n c t i o n s .
I n o r d e r t o a d a p t t h i s fast t r a n s f o r m t o L a g u e r r e - G a u s s i a n g e n e r a t i n g f u n c t i o n s
(or a n y o t h e r g e n e r a t i n g f u n c t i o n ) one needs t o c o m p u t e t h e r e q u i r e d T a y l o r coef-
ficients. T h i s is a t a s k t h a t goes b e y o n d t h e scope o f t h i s b o o k .
35.3 Domain Decomposition
F i n a l l y , a n o t h e r m e t h o d c o m m o n l y used t o d e a l w i t h large c o m p u t a t i o n a l p r o b l e m s
is t h e domain decomposition method. D o m a i n d e c o m p o s i t i o n is f r e q u e n t l y imple
m e n t e d o n p a r a l l e l c o m p u t e r s i n o r d e r t o speed u p t h e c o m p u t a t i o n . A s t a n d a r d
reference (based m o s t l y o n f i n i t e difference a n d f i n i t e element m e t h o d s ) is t h e b o o k
b y S m i t h , B j 0 r s t a d a n d G r o p p [ S m i t h et al. ( 1 9 9 6 ) ] . F o r r a d i a l basis f u n c t i o n s
t h e r e is a recent p a p e r b y B e a t s o n , L i g h t a n d B i l l i n g s [ B e a t s o n et al. ( 2 0 0 0 ) ] .
T h e m a i n a i m o f t h e p a p e r [ B e a t s o n et al. (2000)] is t o solve t h e r a d i a l basis
f u n c t i o n i n t e r p o l a t i o n p r o b l e m discussed m a n y t i m e s i n p r e v i o u s c h a p t e r s . I n p a r -
t i c u l a r , a so-called multiplicative Schwarz a l g o r i t h m ( w h i c h is analogous t o Gauss-
Seidel i t e r a t i o n ) is presented, a n d l i n e a r convergence o f t h e a l g o r i t h m is p r o v e d . A
s e c t i o n w i t h n u m e r i c a l e x p e r i m e n t s r e p o r t s r e s u l t s for a n additive Schwarz method
( w h i c h is analogous t o J a c o b i i t e r a t i o n ) .
I n p a r t i c u l a r , t h e a u t h o r s i m p l e m e n t e d p o l y h a r m o n i c r a d i a l basis f u n c t i o n s a n d
used t h e scale i n v a r i a n t basis discussed i n S e c t i o n 34.4.
T h e classical a d d i t i v e Schwarz a l g o r i t h m is u s u a l l y discussed i n t h e context
o f p a r t i a l d i f f e r e n t i a l e q u a t i o n s , a n d i t is k n o w n t h a t one s h o u l d a d d a coarse level
c o r r e c t i o n i n o r d e r t o ensure convergence a n d t o f i l t e r o u t some o f t h e l o w - f r e q u e n c y
o s c i l l a t i o n s (see, e.g., [ S m i t h et al. ( 1 9 9 6 ) ] ) .
I n [Beatson et al. (2000)] a t w o - l e v e l a d d i t i v e a l g o r i t h m for i n t e r p o l a t i o n p r o b -
lems was presented. O n e b e g i n s b y s u b d i v i d i n g t h e set o f i n t e r p o l a t i o n p o i n t s X
i n t o M smaller sets X{, i = 1 , . . . , M, whose p a i r w i s e i n t e r s e c t i o n is n o n - e m p t y . The
p o i n t s t h a t b e l o n g t o one set Xi o n l y are c a l l e d inner points o f Xi. Those points
i n t h e i n t e r s e c t i o n o f m o r e t h a n one set need t o be assigned i n some w a y as i n n e r
p o i n t s t o o n l y one o f t h e subsets Xi so t h a t t h e c o l l e c t i o n o f a l l i n n e r p o i n t s y i e l d s
t h e e n t i r e set X. T h i s c o r r e s p o n d s t o t h e c o n c e p t o f overlapping domains. O n e also
needs t o choose a coarse g r i d y t h a t c o n t a i n s p o i n t s f r o m a l l o f t h e i n n e r p o i n t sets.
I n t h e setup phase o f t h e a l g o r i t h m t h e r a d i a l basis f u n c t i o n i n t e r p o l a t i o n m a t r i -
ces for t h e smaller p r o b l e m s o n each o f t h e subsets Xi, i = 1 , . . . , M, are c o m p u t e d
a n d f a c t o r e d . A t t h i s p o i n t one c a n use t h e h o m o g e n e o u s basis o f S e c t i o n 34.4 t o
ensure n u m e r i c a l s t a b i l i t y . N o w t h e a l g o r i t h m proceeds as follows:
Algorithm 35.2.
Input: D a t a / , p o i n t sets Xi and factored interpolation matrices Ai, i

1 , . . . , M, tolerance t o l
Initialize r = / , u = 0
W h i l e ||r|| > t o l do
F o r i 1 t o M (i.e., for each subset Xi) do
D e t e r m i n e t h e coefficient v e c t o r of the interpolant t o the residual

r\ X i on Xi.
end
Make c orthogonal to Tl _ .
m l
I n t e r p o l a t e t o r\ o n t h e coarse g r i d y u s i n g a n R B F e x p a n s i o n u . 2
U p d a t e u < u + u\ + u . 2
R e - e v a l u a t e t h e g l o b a l r e s i d u a l r = / u o n t h e w h o l e set X
end
I n [ B e a t s o n et al. (2000)] i t is p r o v e d t h a t a m u l t i p l i c a t i v e v e r s i o n o f t h i s algo-

r i t h m converges a t least l i n e a r l y . H o w e v e r , t h e a d d i t i v e v e r s i o n c a n be m o r e easily
implemented on a parallel computer.
I f s t r i c t l y p o s i t i v e d e f i n i t e kernels s u c h as G a u s s i a n s are used, t h e n i t is n o t
necessary t o m a k e t h e coefficients c o r t h o g o n a l t o p o l y n o m i a l s .
A s i n m a n y a l g o r i t h m s before, t h e e v a l u a t i o n o f t h e residuals needs t o be m a d e
"fast" u s i n g e i t h e r a fast m u l t i p o l e m e t h o d o r a v e r s i o n o f t h e fast F o u r i e r t r a n s f o r m .
I n t h e case o f v e r y l a r g e d a t a sets i t m a y be necessary t o use m o r e t h a n t w o
levels so t h a t one ends u p w i t h a multigrid algorithm.
T h e a u t h o r s o f [ B e a t s o n et al. (2000)] r e p o r t h a v i n g s o l v e d i n t e r p o l a t i o n p r o b -
lems w i t h several m i l l i o n s o f p o i n t s u s i n g t h e d o m a i n d e c o m p o s i t i o n a l g o r i t h m
above.
A n u m b e r o f o t h e r p a p e r s discussing d o m a i n d e c o m p o s i t i o n m e t h o d s for r a d i a l
basis f u n c t i o n s have a p p e a r e d i n t h e l i t e r a t u r e (see, e.g., [Dubai (1994); H o n and
W u (2000); I n g b e r et al. ( 2 0 0 4 ) ; L i a n d H o n ( 2 0 0 4 ) ; L i n g a n d K a n s a ( 2 0 0 4 ) ; W o n g
et al. ( 1 9 9 9 ) ] ) . H o w e v e r , m o s t o f these p a p e r s c o n t a i n l i t t l e t h e o r y , f o c u s i n g m o s t l y
on numerical experiments.
Chapter 36
Generalized Hermite Interpolation
I n 1975 R o l l a n d H a r d y m e n t i o n e d t h e p o s s i b i l i t y o f u s i n g m u l t i q u a d r i c basis func-

t i o n s for H e r m i t e i n t e r p o l a t i o n , i.e., i n t e r p o l a t i o n t o d a t a t h a t also c o n t a i n s d e r i v a -
t i v e i n f o r m a t i o n (see [ H a r d y (1975)] or t h e s u r v e y p a p e r [ H a r d y ( 1 9 9 0 ) ] ) . T h i s p r o b -
l e m , however, was n o t f u r t h e r i n v e s t i g a t e d i n t h e R B F l i t e r a t u r e u n t i l t h e p a p e r [ W u
(1992)] a p p e a r e d . Since t h e n , t h e i n t e r e s t i n t h i s t o p i c has increased s i g n i f i c a n t l y .
I n p a r t i c u l a r , since t h e r e is a close c o n n e c t i o n b e t w e e n t h e generalized H e r m i t e i n -
t e r p o l a t i o n a p p r o a c h a n d s y m m e t r i c c o l l o c a t i o n for e l l i p t i c p a r t i a l d i f f e r e n t i a l equa-
s
t i o n s (see C h a p t e r 3 8 ) . W u deals w i t h H e r m i t e - B i r k h o f f i n t e r p o l a t i o n i n M. a n d his
m e t h o d is l i m i t e d i n t h e sense t h a t one c a n have o n l y one i n t e r p o l a t i o n c o n d i t i o n
per d a t a p o i n t (i.e., some l i n e a r c o m b i n a t i o n o f f u n c t i o n value a n d d e r i v a t i v e s ) . I n
[Sun (1994a)] t h i s r e s t r i c t i o n is e l i m i n a t e d . S u n deals w i t h t h e E u c l i d e a n s e t t i n g a n d
gives results analogous t o t h e ( L a g r a n g e ) i n t e r p o l a t i o n r e s u l t s o f [ M i c c h e l l i ( 1 9 8 6 ) ] .
I n [ N a r c o w i c h a n d W a r d (1994a)] a n even m o r e g e n e r a l t h e o r y o f H e r m i t e i n t e r p o -
s
l a t i o n for c o n d i t i o n a l l y p o s i t i v e d e f i n i t e ( m a t r i x - v a l u e d ) kernels i n R is d e v e l o p e d .
H e r m i t e i n t e r p o l a t i o n w i t h c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s is also discussed
i n [iske (1995)]. A n u m b e r o f a u t h o r s have also c o n s i d e r e d t h e H e r m i t e i n t e r p o l a t i o n
s e t t i n g o n spheres (see, e.g., [Fasshauer ( 1 9 9 9 b ) ; Freeden (1982); Freeden (1987);
R o n a n d S u n (1996)]) or even g e n e r a l R i e m a n n i a n m a n i f o l d s [ D y n et al. (1999);
N a r c o w i c h (1995)].
36.1 T h e Generalized Hermite Interpolation Problem
W e n o w consider d a t a {x^, A j / } , i = 1 , . . . , N, X i G R , w h e r e A = { A i , . . . , XN} is

s
a l i n e a r l y i n d e p e n d e n t set o f c o n t i n u o u s l i n e a r f u n c t i o n a l s a n d / is some ( s m o o t h )
d a t a f u n c t i o n . For e x a m p l e , A^ c o u l d d e n o t e p o i n t e v a l u a t i o n at t h e p o i n t X i a n d
t h u s y i e l d a L a g r a n g e i n t e r p o l a t i o n c o n d i t i o n , or i t c o u l d d e n o t e e v a l u a t i o n o f some
d e r i v a t i v e at t h e p o i n t X \ . H o w e v e r , we a l l o w t h e set A t o c o n t a i n m o r e g e n e r a l
f u n c t i o n a l s such as, e.g., l o c a l i n t e g r a l s . T h i s k i n d o f p r o b l e m was r e c e n t l y s t u d i e d i n
[Beatson a n d L a n g t o n (2006)]. F u r t h e r m o r e , we stress t h a t t h e r e is n o a s s u m p t i o n
t h a t requires t h e d e r i v a t i v e s t o be i n consecutive o r d e r as is u s u a l l y t h e case for
333
p o l y n o m i a l or spline-type H e r m i t e i n t e r p o l a t i o n problems.
W e t r y t o find a n i n t e r p o l a n t o f t h e f o r m
N
^/(*) = $>^(ll*ll), ^ r , (36.i)
w i t h appropriate ( r a d i a l ) basis f u n c t i o n s ipj so t h a t Vf satisfies t h e generalized

interpolation conditions
KVf^Xif, i = 1,..., N.
T o keep t h e discussion t h a t f o l l o w s as t r a n s p a r e n t as possible w e n o w i n t r o d u c e

t h e n o t a t i o n I , . . . , J V for t h e centers o f t h e r a d i a l basis f u n c t i o n s . They will
u s u a l l y be selected t o c o i n c i d e w i t h t h e d a t a sites X = {x ,... x ,xjy}. However, the
f o l l o w i n g is clearer i f w e f o r m a l l y d i s t i n g u i s h b e t w e e n centers j a n d d a t a sites Xi.
A s we w i l l s h o w i n t h e n e x t s e c t i o n , i t is n a t u r a l t o l e t ^(Hxd) = A^</?(||cc | | )
w i t h t h e same f u n c t i o n a l s A j t h a t g e n e r a t e d t h e d a t a a n d ip one o f t h e u s u a l r a d i a l
basic f u n c t i o n s . H o w e v e r , t h e n o t a t i o n A^ i n d i c a t e s t h a t t h e f u n c t i o n a l A n o w acts
o n ip v i e w e d as a f u n c t i o n o f i t s second a r g u m e n t . W e w i l l n o t a d d a n y s u p e r s c r i p t
i f A acts o n a single v a r i a b l e f u n c t i o n o r o n t h e k e r n e l </? as a f u n c t i o n o f i t s f i r s t
variable. T h e r e f o r e , w e assume t h e g e n e r a l i z e d H e r m i t e i n t e r p o l a n t t o be o f t h e
form
N
s
Vf(x) = ^CjXfrQx - ||), x e M, (36.2)
3= 1
a n d r e q u i r e i t t o satisfy
KV f = Xif, i = l,...,N.
T h e l i n e a r s y s t e m Ac = f \ w h i c h arises i n t h i s case has m a t r i x entries
A =X Xfy,
ij i i,j = l , . . . , i V , (36.3)
T
a n d r i g h t - h a n d side f \ [Xif,..., Ajv/] .
I n t h e references m e n t i o n e d at t h e b e g i n n i n g o f t h i s c h a p t e r i t is s h o w n t h a t
A is n o n - s i n g u l a r for t h e same classes o f cp t h a t w e r e a d m i s s i b l e for s c a t t e r e d d a t a
i n t e r p o l a t i o n i n our earlier chapters.
N o t e t h a t w h e n w e are a s s e m b l i n g t h e i n t e r p o l a t i o n m a t r i x A t h e f u n c t i o n a l s A
act o n (p b o t h as a f u n c t i o n o f t h e first v a r i a b l e as w e l l as t h e second v a r i a b l e . T h i s
2 k k
i m p l i e s t h a t we need t o use C functions i n order t o interpolate C d a t a . T h i s is
t h e p r i c e we need t o p a y t o e n s u r e i n v e r t i b i l i t y o f A.
I t is i n t e r e s t i n g t o n o t e t h a t t h e effect o f t h e d e r i v a t i v e a c t i n g o n t h e second
v a r i a b l e ( i . e . , t h e c e n t e r ) o f <p ( w h i c h leads t o a s i g n change for d e r i v a t i v e s o f o d d
orders) was n o t t a k e n i n t o a c c o u n t i n t h e e a r l y p a p e r [ H a r d y ( 1 9 7 5 ) ] , a n d t h u s his
i n t e r p o l a t i o n m a t r i x is n o t s y m m e t r i c .
I t s h o u l d be p o i n t e d o u t t h a t t h e f o r m u l a t i o n i n (36.2) is v e r y g e n e r a l a n d goes
c o n s i d e r a b l y b e y o n d t h e s t a n d a r d n o t i o n o f H e r m i t e i n t e r p o l a t i o n ( w h i c h refers t o
36. Generalized Hermite Interpolation 335
i n t e r p o l a t i o n o f successive d e r i v a t i v e values o n l y ) . H e r e a n y k i n d o f linear func-

t i o n a l s are a l l o w e d as l o n g as t h e set A is l i n e a r l y i n d e p e n d e n t . For example, i n
C h a p t e r 38 we w i l l see h o w t h i s f o r m u l a t i o n c a n be a p p l i e d t o t h e s o l u t i o n o f p a r t i a l
differential equations.
One c o u l d also envision use o f a s i m p l e r R B F e x p a n s i o n o f t h e f o r m
Vf{x) = Y, M\\*-t\\)*
c
x
^ s
-
3= 1
However, i n t h i s case t h e i n t e r p o l a t i o n m a t r i x w i l l n o t be s y m m e t r i c a n d m u c h m o r e
difficult t o analyze t h e o r e t i c a l l y . I n fact, t h e a p p r o a c h j u s t suggested is f r e q u e n t l y
used for t h e s o l u t i o n o f e l l i p t i c p a r t i a l d i f f e r e n t i a l e q u a t i o n s (see t h e d e s c r i p t i o n
of K a n s a ' s m e t h o d i n C h a p t e r 3 8 ) , a n d i t is k n o w n t h a t for c e r t a i n c o n f i g u r a t i o n s
o f t h e c o l l o c a t i o n p o i n t s a n d c e r t a i n d i f f e r e n t i a l o p e r a t o r s t h e s y s t e m m a t r i x does
indeed become singular.
T h e q u e s t i o n o f w h e n t h e f u n c t i o n a l s i n A are l i n e a r l y i n d e p e n d e n t is n o t ad-
dressed i n m o s t papers o n t h e s u b j e c t . However, the b o o k [Wendland (2005a)]
contains t h e f o l l o w i n g reassuring t h e o r e m t h a t covers b o t h H e r m i t e i n t e r p o l a t i o n
a n d c o l l o c a t i o n solutions o f P D E s .
s 2k s
T h e o r e m 3 6 . 1 . Suppose that < G L i ( R ) f l C (M. ) is a strictly positive definitek-
a ( , )
ernel. If the functionals Xj = ( 5 j ; . o D , j = 1 , . . . , N, with multi-indices \a^\ < k
are pairwise distinct, meaning that ct^ ^ if Xj = for different
X j ^ i, then
S
they are also linearly independent over the native space A / $ ( R ) .
I n t h e t h e o r e m above t h e f u n c t i o n a l 5 . X denotes p o i n t e v a l u a t i o n at t h e p o i n t
X j , a n d t h e kernel $ is r e l a t e d t o cp as usual, i.e., <fr(cc, ) = cp(\\x | | ) . L i k e m o s t
results o n s t r i c t l y p o s i t i v e definite f u n c t i o n s , t h i s t h e o r e m c a n also be generalized
t o t h e s t r i c t l y c o n d i t i o n a l l y p o s i t i v e definite case.
36.2 M o t i v a t i o n for t h e S y m m e t r i c Formulation
I n t h i s section we i l l u s t r a t e w h y t h e f o r m u l a t i o n used i n (36.2) is n a t u r a l for t h e

Hermite interpolation problem. T h a t is, aside f r o m t h e fact t h a t t h e symmetric
i n t e r p o l a t i o n m a t r i x (36.3) is g u a r a n t e e d t o be i n v e r t i b l e for a l l c o m m o n l y used
R B F s , we w i l l show t h a t b y c h o o s i n g t h e basis f u n c t i o n s as i n (36.2) t h e m a t r i x
associated w i t h ( H e r m i t e ) i n t e r p o l a t i o n t o f u n c t i o n value a n d first d e r i v a t i v e value
at a p o i n t corresponds t o a l i m i t o f t h e m a t r i x for L a g r a n g e i n t e r p o l a t i o n t o clusters
of p o i n t s . W e w i l l also i l l u s t r a t e t h i s fact n u m e r i c a l l y i n t h e n e x t section.
I n [Franke et al. (1995)] t h e a u t h o r s i n v e s t i g a t e d a d a p t i v e least squares a p p r o x -
2
imation w i t h multiquadrics i n M b y means o f i n s e r t i n g k n o t s ( s i m i l a r t o o u r algo-
r i t h m o f C h a p t e r 2 1 ) . T h e a u t h o r s describe n u m e r i c a l e x p e r i m e n t s w h i c h suggest
t h a t ( L a g r a n g e ) m u l t i q u a d r i c basis f u n c t i o n s associated w i t h clusters o f centers i n
a d a p t i v e least squares a p p r o x i m a t i o n s h o u l d be r e p l a c e d b y a p p r o p r i a t e d i r e c t i o n a l
d e r i v a t i v e s o f one o f t h e basis f u n c t i o n s .
W e n o w present a t h e o r e t i c a l j u s t i f i c a t i o n for t h i s o b s e r v a t i o n based o n a n a n a l -
ysis o f a o n e - d i m e n s i o n a l e x a m p l e . A m o r e g e n e r a l a n a l y s i s i n v o l v i n g h i g h e r d e r i v a -
t i v e s a n d h i g h e r - d i m e n s i o n a l spaces w o u l d be o f t h e same flavor u s i n g t h e m u l t i v a r i -
ate T a y l o r t h e o r e m . W e discuss i n t e r p o l a t i o n t o f u n c t i o n values a n d first d e r i v a t i v e s
a t g i v e n p o i n t s o n t h e r e a l l i n e u s i n g r a d i a l basis f u n c t i o n s .
To show h o w one g e n e r a l s u b - b l o c k i n t h e H e r m i t e m a t r i x relates t o a n associated
b l o c k o f a L a g r a n g e m a t r i x , i t w i l l suffice t o a n a l y z e t h e s u b - b l o c k o f t h e L a g r a n g e
i n t e r p o l a t i o n m a t r i x c o r r e s p o n d i n g t o t w o p a i r s o f n e a r b y p o i n t s . L e t these p o i n t s
be Xi, Xi + Ax, and / + A for some indices i a n d j a n d some s m a l l distances
Ax a n d A . F u r t h e r m o r e , l e t t h e r a d i a l f u n c t i o n b e o f t h e f o r m ip <p(\x
x, K. W e also assume (p is d i f f e r e n t i a b l e a t t h e o r i g i n . I n the proof of the
f o l l o w i n g l e m m a we m a k e use o f t h e f o l l o w i n g i d e n t i t i e s , w h i c h are s t r a i g h t f o r w a r d
applications of the univariate Taylor theorem
<p($x + Ax) - fl) = <p(\x - |) + A s J^(|a; - |) + 2

0((Ax) ), (36.4)
<p{\x - ( + A O I ) = <p{\x - c;|) - A ^ { \ x - | ) + O ( ( A 0 ) - 2 (36.5)
T o keep t h e n o t a t i o n as s i m p l e as possible we w r i t e ^<p(\xi j|) t o denote
3$\x-- xi, j%<p(\xi-tj\) t o d e n o t e ^ ^ ( l ^ t - ^\)\i=j, a n d -^^(\xi

dxde - ^\)
2
d
to denote \)\x=Xi,,=j
L e m m a 3 6 . 1 . For the ID situation described above we have
^ | = detM H + O(Ax) + O(A0,
where M L is the part of the Lagrange matrix corresponding to the basis functions
centered at j and + A interpolating to values at Xi and Xi + Ax, i.e.,
I) (&+AOI)
_cp{$xi + Ax) - 0 1 ) <p(\(xi + Ax) - + AOI)
and MH is the associated Hermite block
M H
Proof. I f we use (36.4) t o m o d i f y t h e second r o w o f M L , and then subtract the

first r o w f r o m t h e second one, w e o b t a i n
cp(\xi-^$ v(l*i-(&+A0l)
det M L = Ax
&<p(\xi - 61) + O(Ax) <p(\Xi - (0 + A)|) + O(Ax)
T h i s t e c h n i q u e is c o m m o n l y used w h e n a n a l y z i n g s i g n p r o p e r t i e s o f H e r m i t e m a t r i -
ces (see, e.g., [ S c h u m a k e r ( 1 9 8 1 ) ] ) . N o w w e r e p e a t t h i s process w i t h (36.5) a n d t h e
36. Generalized Hermite Interpolation 337
second c o l u m n o f M L t o get
detM L = AzAd; x
fetpUxi - 0 | ) + O(Ax) -jgszvUxi - 0 | ) + O(Ax) + O(A0
a n d t h u s t h e s t a t e m e n t follows.
We now illustrate the Hermite interpolation approach w i t h a simple 2D example

using first-order p a r t i a l derivative functionals.
E x a m p l e 3 6 . 1 . L e t d a t a { x , , /(xi)}?^ a n d { x ^ ^(xi)}]jL n+1 w i t h x = (x,y)

2
R be g i v e n . T h u s
5xi
A = J'
\S o-g-,
Xi
i = 1, , n ,
i = n + l,...,N.
Then
N
J^CjX^cpdlx
7= 1
J
n N
Y.cjfiwx - fcii)+ E
j=n+l
n
dip
E^H*
J'=l
~ u)- E j'=n+l
A f t e r e n f o r c i n g t h e i n t e r p o l a t i o n c o n d i t i o n s t h e s y s t e m m a t r i x is g i v e n b y
A At
A =
A x Ax
with
A^ = ip(\\xi - 3\\J> i,j = 1, ,n,
(^)ij = ^(\\Xi-Zj\\) = - ^ ( \ \ X i - Z j \ \ ) , i l , . . . , n , j = n + 1,... ,N,
dip
{A )ij
x i = n + l,...,N, j = l, ,n,
dx
2
d i,3 = n + 1 , . . - , N.
N o t e t h a t t h e t w o b l o c k s A$ a n d A x are i d e n t i c a l p r o v i d e d t h e d a t a sites a n d
centers coincide since i n t h i s case t h e s i g n change d u e t o d i f f e r e n t i a t i o n w i t h respect
t o t h e second v a r i a b l e i n A^ is c a n c e l l e d b y t h e i n t e r c h a n g e o f t h e roles o f X i a n d
j when compared t o A . x H e r e one needs t o realize t h a t t h e p a r t i a l d e r i v a t i v e o f <p
w i t h respect t o t h e c o o r d i n a t e x w i l l a l w a y s c o n t a i n a l i n e a r f a c t o r i n x, i.e., (for
2 2
t h e 2 D e x a m p l e c o n s i d e r e d h e r e ) c/?(||ai||) = <p(r) = ip(y/'x + y ), so t h a t b y t h e
chain rule
|^>(ll*||) = |:V(r)^r(x,y)
d x
2
dr y/ 2x + y
= ivrf (36.6)
2 2
since r = ||cc|| = \/x + y. T h i s a r g u m e n t generalizes for a n y o d d - o r d e r d e r i v a t i v e .
N o t e t h a t t h e m a t r i x A is also s y m m e t r i c f o r e v e n - o r d e r d e r i v a t i v e s . F o r e x a m -
ple, one c a n easily v e r i f y t h a t
(
2 2 d
d / us 1
i d 2
, s V , A
2 2 2
dx r \ dr r dr
so t h a t n o w t h e i n t e r c h a n g e o f ccj a n d j does n o t cause a s i g n change. O n the

o t h e r h a n d , t w o d e r i v a t i v e s o f (p w i t h respect t o t h e second v a r i a b l e d o n o t l e a d
t o a s i g n change, e i t h e r .
A c a t a l o g o f R B F s a n d t h e i r d e r i v a t i v e s is p r o v i d e d i n A p p e n d i x D .
Chapter 37
R B F Hermite Interpolation in MATLAB
W e n o w i l l u s t r a t e t h e s y m m e t r i c a p p r o a c h t o H e r m i t e i n t e r p o l a t i o n w i t h a set o f
n u m e r i c a l e x p e r i m e n t s for f i r s t - o r d e r H e r m i t e i n t e r p o l a t i o n (i.e., to positional and
g r a d i e n t d a t a ) i n 2 D u s i n g t h e M A T L A B p r o g r a m R B F H e r m i t e _ 2 D . m l i s t e d b e l o w as
P r o g r a m 3 7 . 1 . Since d e r i v a t i v e s o f b o t h t h e R B F s a n d t h e test f u n c t i o n need t o be
i n c l u d e d i n t h e p r o g r a m w e use t h e f u n c t i o n
_ t a n h ( 9 ( y - x)) + 1
J y )
^ ~ tanh(9) + l
w h i c h has f a i r l y s i m p l e p a r t i a l d e r i v a t i v e s (see lines 9 - 1 0 o f t h e p r o g r a m ) t o generate
t h e d a t a . T h e R B F used i n t h i s set o f e x p e r i m e n t s is t h e m u l t i q u a d r i c w i t h shape
p a r a m e t e r e = 6.
W e c o m p a r e four different p r o b l e m s :
(1) L a g r a n g e i n t e r p o l a t i o n , i.e., i n t e r p o l a t i o n t o f u n c t i o n values o n l y , a t a set o f N

e q u a l l y spaced p o i n t s i n t h e u n i t square.
(2) L a g r a n g e i n t e r p o l a t i o n t o f u n c t i o n values a t SN c l u s t e r e d p o i n t s w i t h s e p a r a t i o n
distance q = O.lh, w h e r e h is t h e f i l l d i s t a n c e o f t h e set o f e q u a l l y spaced p o i n t s
(see t h e left p l o t i n F i g u r e 3 7 . 1 ) .
(3) T h e same as above, b u t w i t h q O.Ol/i (see t h e r i g h t p l o t i n F i g u r e 3 7 . 1 ) .
(4) H e r m i t e i n t e r p o l a t i o n t o f u n c t i o n value, a n d values o f b o t h f i r s t - o r d e r p a r t i a l
d e r i v a t i v e s a t t h e N e q u a l l y spaced p o i n t s used i n t h e first e x p e r i m e n t .
T h e s t a n d a r d L a g r a n g e i n t e r p o l a n t s were c o m p u t e d v i a P r o g r a m 2.1 w i t h t h e
r e q u i r e d m o d i f i c a t i o n o f t h e R B F a n d test f u n c t i o n d e f i n i t i o n s , i.e., l i n e 1 is r e p l a c e d
by
1 r b f = @(e,r) s q r t ( l + ( e * r ) . ~ 2 ) ; ep = 6 ;
a n d lines 2 - 6 are replaced b y t h e single l i n e
2 t e s t f u n c t i o n = @(x,y) ( t a n h ( 9 * ( y - x ) ) + l ) / ( t a n h ( 9 ) + l ) ;
T h e e x p e r i m e n t s w i t h L a g r a n g e i n t e r p o l a t i o n a t c l u s t e r e d d a t a sites w e r e ac-
c o m p l i s h e d b y t h e same p r o g r a m b y a d d i n g t h e f o l l o w i n g code b e t w e e n lines 8 a n d
9 i n Program 2.1:
339
q = 0.1/(sqrt(N)-l);
grid = linspace(0,1,sqrt(N));
s h i f t e d = l i n s p a c e ( q , 1 + q , s q r t ( N ) ) ; s h i f t e d ( e n d ) = 1-q;
[xcl.ycl] = meshgrid(shifted,grid);
[xc2,yc2] = meshgrid(grid,shifted);
dsites = [dsites; x c l ( : ) y c l ( : ) ; xc2(:) yc2(:)];
T h e r e s u l t i n g d a t a p o i n t sets for q = 0. l / ( s q r t ( N ) - l ) , i.e., q = h/10, a n d for

q = 0.01/(sqrt(N)-l) (or q = h/100) are s h o w n i n F i g u r e 37.1.
1
1
f*" -8" -8* " i
% 8o 8o 8o
0.6
y iio 8o 8o 8o
0.4
! 8o 8o 8o o
0.2
oft -8e-
0.2 0.4 0.6 0.8
x
Fig. 37.1 Clustered point sets with N = 25 basic data points. Cluster size h/10 (left) and cluster
size h/100 (right).
T h e p r o g r a m RBFHermite_2D .m m a i n t a i n s t h e same basic s t r u c t u r e as e a r l i e r

interpolation programs. N o w , h o w e v e r , w e need t o define d e r i v a t i v e s o f t h e R B F
of up to twice the order of the data. T h i s is d o n e for t h e M Q basic f u n c t i o n o n
lines 1-6. N o t e t h a t t h e second-order p a r t i a l s c o u l d be expressed e i t h e r as s t a t e d
i n P r o g r a m 37.1 or as
4 dxxrbf = @(e,r,dy) e ~ 2 * ( l + ( e * d y ) . ~ 2 ) . / ( l + ( e * r ) . ~ 2 ) . ~ ( 3 / 2 ) ;
6 dyyrbf = @(e,r,dx) e~2*(l+(e*dx).~2)./(l+(e*r).~2)."(3/2);
H e r e dx a n d dy d e n o t e n o n - r a d i a l difference t e r m s o f t h e x or y - c o m p o n e n t s , re-
s p e c t i v e l y (see, e.g., (36.6)). W e choose t h e f o r m e r r e p r e s e n t a t i o n since for m a n y
o t h e r basic f u n c t i o n s these second-order p a r t i a l s are m o r e n a t u r a l l y expressed i n
t e r m s o f t h e differences o f t h e v a r i a b l e o f d i f f e r e n t i a t i o n (i.e., dxxrbf is expressed
i n t e r m s o f x-differences, etc.).
Since t h e d e r i v a t i v e s o f t h e basic f u n c t i o n n o w also c o n t a i n t h e difference t e r m s
m e n t i o n e d above, we need a n o t h e r subroutine that computes m a t r i c e s o f differ-
ences o f p o i n t c o o r d i n a t e s . T h i s s u b r o u t i n e is c a l l e d D i f f e r e n c e M a t r i x . m (see
P r o g r a m 37.2), a n d i t is b u i l t a n a l o g o u s t o P r o g r a m 1.1 (DistanceMatrix.m). T h u s ,
o n lines 17-22 o f P r o g r a m 37.1 w e c o m p u t e n o t o n l y d i s t a n c e m a t r i c e s o f d a t a sites
a n d centers ( o r e v a l u a t i o n p o i n t s a n d c e n t e r s ) , b u t also t h e c o r r e s p o n d i n g difference
37. RBF Hermite Interpolation in MATLAB 341
matrices. These three matrices are then required when we evaluate the R B F and its
partials to obtain the building blocks of the interpolation and evaluation matrices
(see lines 25-35). Note the minus signs used with the blocks in columns 2 and 3 of
the block matrices IM and EM on lines 31 and 35. They reflect differentiation of the
basic function with respect to its second variable (c.f. (36.2) and (36.3)).
The data are generated by sampling the test function and its derivatives (see
lines 8-10, and line 23). Evaluation of the interpolant, error computation and
rendering are exactly the same as in earlier programs.
Program 37.1. RBFHermite_2D .m
'/. RBFHermite_2D
% S c r i p t t h a t performs f i r s t - o r d e r 2D RBF Hermite i n t e r p o l a t i o n
% C a l l s on: D i s t a n c e M a t r i x , D i f f e r e n c e M a t r i x
% Define RBF and i t s d e r i v a t i v e s
1 r b f = @(e,r) s q r t ( l + ( e * r ) . "2) ; 7, MQ RBF
2 dxrbf = @(e,r,dx) dx*e~2./sqrt ( l + ( e * r ) . ""2) ;
3 dyrbf = @(e,r,dy) d y * e ~ 2 . / s q r t ( l + ( e * r ) . ~ 2 ) ;
4a dxxrbf = @(e,r,dx) e ~ 2 * ( l + ( e * r ) . ~ 2 - ( e * d x ) . ~ 2 ) . / . . .
4b (l+(e*r).~2).~(3/2);
5 dxyrbf = @(e,r,dx,dy) - e " 4 * d x . * d y . / ( l + ( e * r ) . " 2 ) . " ( 3 / 2 ) ;
6a dyyrbf = @(e,r,dy) e~2*(1+(e*r)."2-(e*dy)."2)./...
6b (l+(e*r)."2).~(3/2);
7 ep = 6;
% Define t e s t f u n c t i o n and i t s d e r i v a t i v e s
8 t f = (x,y) ( t a n h ( 9 * ( y - x ) ) + l ) / ( t a n h ( 9 ) + l ) ;
9 tfDx = @(x,y) 9 * ( t a n h ( 9 * ( y - x ) ) . ~ 2 - l ) / ( t a n h ( 9 ) + l ) ;
10 tfDy = @(x,y) 9 * ( l - t a n h ( 9 * ( y - x ) ) . ~ 2 ) / ( t a n h ( 9 ) + l ) ;
5
11 N = 289; g r i d t y p e = u ' ;
12 neval = 40;
0
13 name = s p r i n t f ( 'Data2D_y d / s' ,N,gridtype) ; load(name)
o 0
14 c t r s = d s i t e s ;
'/, Create n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
% i n t h e u n i t square
16 e p o i n t s = [ x e ( : ) y e ( : ) ] ;
Compute t h e d i s t a n c e and d i f f e r e n c e m a t r i c e s f o r
'/, e v a l u a t i o n matrix
18 dx_eval = D i f f e r e n c e M a t r i x ( e p o i n t s ( : , 1 ) , c t r s ( : , 1 ) ) ;
19 dy_eval = D i f f e r e n c e m a t r i x ( e p o i n t s ( : , 2 ) , c t r s ( : , 2 ) ) ;
% Compute t h e d i s t a n c e and d i f f e r e n c e m a t r i c e s f o r
% interpolation matrix
21 dx_data = D i f f e r e n c e M a t r i x ( d s i t e s ( : , 1 ) , c t r s ( : , 1 ) ) ;
22 dy_data = D i f f e r e n c e M a t r i x ( d s i t e s ( : , 2 ) , c t r s ( : , 2 ) ) ;
23a r h s = [ t f ( d s i t e s ( : , 1 ) , d s i t e s ( : , 2 ) ) ; ...
23b t f D x ( d s i t e s ( : , 1 ) , d s i t e s ( : , 2 ) ) ; ...
23c tfDy(dsites(:,1),dsites(:,2))];
24 exact = t f ( e p o i n t s ( : , 1 ) , e p o i n t s ( : , 2 ) ) ;
/o Compute b l o c k s f o r i n t e r p o l a t i o n m a t r i x
25 IM = rbf(ep,DM_data) ;
26 DxIM = dxrbf(ep,DM_data,dx_data);
27 DylM = dyrbf(ep,DM_data,dy_data);
28 DxxIM = dxxrbf(ep,DM_data,dx_data);
29 DxylM = dxyrbf(ep,DM_data,dx_data,dy_data);
30 DyyIM = dyyrbf(ep,DM_data,dy_data);
/o Assemble symmetric i n t e r p o l a t i o n m a t r i x
31a IM = [IM -DxIM -DylM;
31b DxIM -DxxIM -DxylM;
31c DylM -DxylM -DyyIM];
% Compute b l o c k s f o r e v a l u a t i o n m a t r i x
33 DxEM = dxrbf(ep,DM_eval,dx_eval);
34 DyEM = dyrbf(ep,DM_eval,dy_eval);
/, Assemble e v a l u a t i o n m a t r i x
35 EM = [EM -DxEM -DyEM];
RBF Hermite i n t e r p o l a n t
36 Pf = EM * ( I M \ r h s ) ;
39 f p r i n t f ('RMS e r r o r : /oe\n', r m s _ e r r )
40 f p r i n t f ( 'Maximum e r r o r : y,e\n', maxerr)
41 fview = [-30,30] ; viewing angles f o r plot
P r o g r a m 3 7 . 2 . DifferenceMatrix.m
/, DM = D i f f e r e n c e M a t r i x ( d a t a c o o r d , c e n t e r c o o r d )
% Forms t h e d i f f e r e n c e m a t r i x of two s e t s of p o i n t s i n R
% (some f i x e d c o o r d i n a t e of p o i n t i n R ~ s ) , i . e . ,
% DM(j,k) = datacoord_j - centercoord_k .
1 f u n c t i o n DM = D i f f e r e n c e M a t r i x ( d a t a c o o r d , c e n t e r c o o r d )
r
37. RBF Hermite Interpolation in MATLAB 343
7. The n d g r i d command produces two MxN matrices:

7o d r , c o n s i s t i n g of N i d e n t i c a l columns
7o (each c o n t a i n i n g t h e M d a t a sites)
% c c , c o n s i s t i n g of M i d e n t i c a l rows
7o (each c o n t a i n i n g t h e N c e n t e r s )
2 [dr,cc] = ndgrid(datacoord(:),centercoord(:));
3 DM = d r - c c ;
I n T a b l e s 3 7 . 1 a n d 37.2 as w e l l as F i g u r e 37.2 w e d i s p l a y R M S - e r r o r s , - 2
c o n d i t i o n numbers o f t h e i n t e r p o l a t i o n matrices, a n d p l o t s o f t h e interpolants for

t h e e x p e r i m e n t s d e s c r i b e d above.
Several o b s e r v a t i o n s c a n b e m a d e . F i r s t , t h e l i m i t i n g r e l a t i o n b e t w e e n c l u s t e r e d
L a g r a n g e i n t e r p o l a n t s a n d H e r m i t e i n t e r p o l a n t s as discussed i n t h e p r e v i o u s s e c t i o n
is o b v i o u s . M o r e o v e r , i t is also o b v i o u s t h a t i n t e r p o l a t i o n t o f u n c t i o n a n d d e r i v a t i v e
d a t a a t a g i v e n p o i n t is m o r e a c c u r a t e t h a n i n t e r p o l a t i o n t o f u n c t i o n values alone.
Table 37.1 2D interpolation with clustered data vs. Hermite interpolation

(part 1).
Lagrange clustered , q = 0.1h
mesh RMS-error cond(A) RMS-error cond(^4)
3x3 1.620492e-001 6.078349e+001 8.471301e-002 9.052247e+003

5x5 6.148258e-002 9.464176e+002 2.733258e-002 3.073957e+005
9x9 8.521994e-003 6.523036e+004 2.678543e-003 8.811980e+007
17 x 17 2.246810e-004 9.017750e+007 3.138761e-005 3.555214e+012
33 x 33 2.017643e-006 4.799960e+013 2.925784e-007 6.474324e+020
Table 37.2 2D interpolation with clustered data vs. Hermite interpolation

(part 2).
clustered, q = O.Olh Hermite
mesh RMS-error cond(A) RMS-error cond(A)
3x3 9.084939e-002 8.580483e+005 9.128193e-002 1.326346e+002

5x5 2.792157e-002 2.829762e+007 2.794943e-002 2.292450e+003
9x9 2.687753e-003 8.325283e+009 2.688346e-003 2.185224e+005
17 x 17 3.147808e-005 3.426489e+014 3.148843e-005 2.486624e+009
33 x 33 8.941613e-006 8.943758e+020 5.731027e-009 6.261336e+018
The a d v a n t a g e o f t h e H e r m i t e i n t e r p o l a t i o n a p p r o a c h over t h e c l u s t e r e d L a -
g r a n g e a p p r o a c h is c l e a r l y e v i d e n t f o r t h e e x p e r i m e n t s w i t h N 33 x 33 = 1089
basic d a t a p o i n t s ( o r N = 3 2 6 7 c l u s t e r e d d a t a p o i n t s ) . I n t h i s case t h e ^ - c o n d i t i o n
2 0
n u m b e r o f A for t h e c l u s t e r e d i n t e r p o l a n t s is o n t h e o r d e r o f 1 0 , w h i l e i t is " o n l y "
6.261336e-)-018 for t h e H e r m i t e m a t r i x . T h i s difference, however, has a s i g n i f i c a n t
impact on the numerical stability, and the resulting RMS-errors. T h e H e r m i t e i n -

t e r p o l a n t is m o r e t h a n t h r e e o r d e r s o f m a g n i t u d e m o r e a c c u r a t e t h a n t h e Lagrange
i n t e r p o l a n t t o clusters w i t h q = / i / 1 0 0 (see t h e l a s t r o w o f T a b l e 3 7 . 2 ) .
z
0.5-
z
0.5
X 10
1.5
z
0.5-
z
0.5-
2
UJ
10.5
Fig. 37.2 F i t s for clustered interpolants with N 289 basic data points. Top left to bottom
right: Lagrange interpolant, interpolant with cluster size h/10, interpolant with cluster size h/100,
Hermite interpolant.
Chapter 38
Solving Elliptic Partial Differential

Equations via R B F Collocation
I n t h i s c h a p t e r we discuss h o w t h e t e c h n i q u e s used i n p r e v i o u s c h a p t e r s for L a g r a n g e

a n d H e r m i t e i n t e r p o l a t i o n c a n be a p p l i e d t o t h e n u m e r i c a l s o l u t i o n o f e l l i p t i c p a r t i a l
d i f f e r e n t i a l e q u a t i o n s . T h e r e s u l t i n g n u m e r i c a l m e t h o d w i l l be a collocation approach
based o n r a d i a l basis f u n c t i o n s . I n t h e P D E l i t e r a t u r e t h i s is also o f t e n r e f e r r e d t o

as a strong form solution.
T o m a k e t h e discussion t r a n s p a r e n t we w i l l i n i t i a l l y focus o n t h e case o f a t i m e

2
independent linear elliptic p a r t i a l differential equation i n R .
38.1 Kansa's Approach
A now very popular non-symmetric method for t h e s o l u t i o n o f e l l i p t i c P D E s w i t h

r a d i a l basis f u n c t i o n s was suggested b y E d K a n s a i n [ K a n s a ( 1 9 9 0 b ) ] . I n o r d e r t o be
able t o c l e a r l y p o i n t o u t t h e differences b e t w e e n K a n s a ' s m e t h o d a n d a s y m m e t r i c
a p p r o a c h p r o p o s e d i n [Fasshauer (1997)] w e r e c a l l some o f t h e basics o f s c a t t e r e d
s
d a t a i n t e r p o l a t i o n w i t h r a d i a l basis f u n c t i o n s i n R .
I n the scattered d a t a i n t e r p o l a t i o n context we are g i v e n d a t a {xi,fi}, i =
s
1,...,N, X i G R , w h e r e we c a n t h i n k o f t h e values fi being sampled from a
s
function / : R R . T h e g o a l is t o f i n d a n i n t e r p o l a n t o f t h e f o r m
N
v (x)
f = J2cMW* ~ J\\)>
X X
e R , s
(38.1)
such t h a t
V (xi)
f = fi, i = l,...,N.
T h e s o l u t i o n o f t h i s p r o b l e m leads t o a l i n e a r s y s t e m Ac = / w i t h t h e e n t r i e s o f A
given by
A^ = <p(\\xi - Xj\\), i,j = l,...,N. (38.2)
A s discussed earlier, t h e m a t r i x A is n o n - s i n g u l a r for a large class o f r a d i a l f u n c t i o n s

i n c l u d i n g (inverse) m u l t i q u a d r i c s , Gaussians, a n d t h e s t r i c t l y p o s i t i v e d e f i n i t e c o m -
p a c t l y s u p p o r t e d f u n c t i o n s o f W e n d l a n d , W u , G n e i t i n g or B u h m a n n . I n t h e case
345
o f s t r i c t l y c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s s u c h as p o l y h a r m o n i c splines t h e
p r o b l e m needs t o be a u g m e n t e d b y p o l y n o m i a l s .
W e now s w i t c h to the collocation s o l u t i o n o f p a r t i a l differential equations. As-
s
s u m e we are g i v e n a d o m a i n 17 C R , a n d a l i n e a r e l l i p t i c p a r t i a l d i f f e r e n t i a l e q u a t i o n
of t h e f o r m
Cu(x) = f(x), x i n 17, (38.3)
w i t h (for s i m p l i c i t y o f d e s c r i p t i o n ) D i r i c h l e t b o u n d a r y c o n d i t i o n s
u(x) = g(x), x o n <917. (38.4)
For K a n s a ' s c o l l o c a t i o n m e t h o d we t h e n choose t o represent t h e approximate

s o l u t i o n u b y a r a d i a l basis f u n c t i o n e x p a n s i o n a n a l o g o u s t o t h a t used for s c a t t e r e d
data interpolation, i.e.,
N
u(x) = J2 M\\x-tj\\)-
c
(38.5)
3= 1
As i n t h e p r e v i o u s c h a p t e r o n H e r m i t e i n t e r p o l a t i o n we n o w f o r m a l l y d i s t i n -
guish i n our n o t a t i o n between centers 3 {i,...,jv} a n
d collocation points
X = {xi,..., XN} C 17. W h i l e f o r m a l l y different, these p o i n t s w i l l o f t e n p h y s i c a l l y
coincide. A scenario w i t h 3 ^ X w i l l be e x p l o r e d i n C h a p t e r s 39 a n d 40. F o r t h e
f o l l o w i n g discussion we assume t h e s i m p l e s t possible s e t t i n g , i.e., 3 X a n d no
p o l y n o m i a l t e r m s are a d d e d t o t h e e x p a n s i o n ( 3 8 . 5 ) .
T h e c o l l o c a t i o n m a t r i x t h a t arises w h e n m a t c h i n g t h e d i f f e r e n t i a l e q u a t i o n (38.3)
a n d t h e b o u n d a r y c o n d i t i o n s (38.4) at t h e c o l l o c a t i o n p o i n t s X w i l l be o f t h e f o r m
(38.6)
w h e r e t h e t w o blocks are g e n e r a t e d as follows:
{Ac)ij = C<p(\\x -j\\)\ , x=Xi Xi G I , j G 3,

Aij = (f(\\xi - Xi G B, j G 3.
H e r e t h e set X o f c o l l o c a t i o n p o i n t s is s p l i t i n t o a set X o f i n t e r i o r p o i n t s , a n d a
set B o f b o u n d a r y p o i n t s . T h e p r o b l e m is w e l l - p o s e d i f t h e l i n e a r s y s t e m Ac = y,
w i t h y a v e c t o r c o n s i s t i n g o f entries f(xi), Xi G X , f o l l o w e d b y g(xi), Xi G B, has
a unique solution.
W e n o t e t h a t a change i n t h e b o u n d a r y c o n d i t i o n s (38.4) is as s i m p l e as m a k i n g
changes t o a few r o w s o f t h e m a t r i x A i n (38.6) as w e l l as o n the* r i g h t - h a n d side y.
W e also p o i n t o u t t h a t w h i l e t h i s is a r a t h e r general d e s c r i p t i o n o f a n u m e r i c a l
m e t h o d w i t h n o p a r t i c u l a r R B F i n m i n d , K a n s a specifically p r o p o s e d t o use m u l t i -
quadrics i n (38.5), and consequently this non-symmetric collocation approach often
appears i n t h e l i t e r a t u r e as t h e multiquadric method. I n the paper [Kansa (1990b)]
t h e a u t h o r describes t h r e e sets o f e x p e r i m e n t s u s i n g t h e m u l t i q u a d r i c m e t h o d a n d he
38. Solving Elliptic Partial Differential Equations via RBF Collocation 347
comments on the superior performance o f m u l t i q u a d r i c s i n terms of c o m p u t a t i o n a l

c o m p l e x i t y a n d a c c u r a c y w h e n c o m p a r e d t o f i n i t e difference m e t h o d s .
M o r e o v e r , K a n s a suggests t h e use o f v a r y i n g shape p a r a m e t e r s Ej, j = 1 , . . . , N.
W h i l e t h e t h e o r e t i c a l analysis o f t h e r e s u l t i n g m e t h o d is near i n t r a c t a b l e , K a n s a
shows t h a t t h i s t e c h n i q u e i m p r o v e s t h e a c c u r a c y a n d s t a b i l i t y o f t h e m e t h o d w h e n
c o m p a r e d t o u s i n g o n l y one c o n s t a n t v a l u e o f e (see [ K a n s a ( 1 9 9 0 b ) ] ) . Except
for one p a p e r b y B o z z i n i , L e n a r d u z z i a n d Schaback [ B o z z i n i et al. (2002)] ( w h i c h
addresses o n l y t h e i n t e r p o l a t i o n s e t t i n g ) t h e t h e o r e t i c a l aspects o f v a r y i n g shape
p a r a m e t e r s have n o t been discussed i n t h e l i t e r a t u r e .
A p r o b l e m w i t h K a n s a ' s m e t h o d is t h a t for a c o n s t a n t shape parameter
e the m a t r i x A may be singular for c e r t a i n c o n f i g u r a t i o n s o f t h e centers j.
O r i g i n a l l y , K a n s a assumed t h a t t h e n o n - s i n g u l a r i t y r e s u l t s e s t a b l i s h e d b y M i c c h e l l i
for i n t e r p o l a t i o n m a t r i c e s (see t h e discussion i n t h e earlier c h a p t e r s o f t h i s b o o k )
w o u l d c a r r y over t o t h e P D E case. H o w e v e r , as t h e n u m e r i c a l e x p e r i m e n t s o f [ H o n
a n d Schaback (2001)] show, t h i s is n o t so. T h i s fact is n o t r e a l l y s u r p r i s i n g since t h e
m a t r i x for t h e c o l l o c a t i o n p r o b l e m is c o m p o s e d o f r o w s t h a t are b u i l t f r o m different
f u n c t i o n s , w h i c h d e p e n d i n g o n t h e d i f f e r e n t i a l o p e r a t o r C m i g h t n o t even be
r a d i a l . T h e results for t h e n o n - s i n g u l a r i t y o f i n t e r p o l a t i o n m a t r i c e s , however, are
based o n t h e fact t h a t A is g e n e r a t e d b y a single f u n c t i o n (p.
Nevertheless, a n i n d i c a t i o n o f t h e success o f K a n s a ' s m e t h o d are t h e e a r l y
papers [Dubai (1992); D u b a i ( 1 9 9 4 ) ; G o l b e r g et al. (1996); K a n s a (1992);
M o r i d i s a n d K a n s a (1994)] a n d m a n y m o r e since. Since t h e n u m e r i c a l e x p e r i m e n t s
of H o n a n d Schaback show t h a t K a n s a ' s m e t h o d c a n n o t be w e l l - p o s e d for a r b i t r a r y
center l o c a t i o n s , i t is n o w a n o p e n q u e s t i o n t o f i n d sufficient c o n d i t i o n s o n t h e center
l o c a t i o n s t h a t guarantee i n v e r t i b i l i t y o f t h e K a n s a m a t r i x . O n e possible a p p r o a c h
b u i l t o n t h e basic ideas o f t h e g r e e d y a l g o r i t h m o f C h a p t e r 33 is t o a d a p -
t i v e l y select " g o o d " centers f r o m a l a r g e set o f possible c a n d i d a t e s . F o l l o w i n g t h i s
s t r a t e g y i t is possible t o ensure i n v e r t i b i l i t y o f t h e c o l l o c a t i o n m a t r i x t h r o u g h o u t
t h e i t e r a t i v e a l g o r i t h m . T h i s a p p r o a c h is d e s c r i b e d i n t h e recent p a p e r [ L i n g et al.
(2006)].
Before we discuss a n a l t e r n a t e a p p r o a c h (based o n t h e s y m m e t r i c H e r m i t e i n -
t e r p o l a t i o n m e t h o d ) w h i c h does ensure well-posedness o f t h e r e s u l t i n g c o l l o c a t i o n
m a t r i x we w o u l d l i k e t o p o i n t o u t t h a t i n [ M o r i d i s a n d K a n s a (1994)] t h e a u t h o r s
suggest h o w K a n s a ' s m e t h o d c a n be a p p l i e d t o o t h e r t y p e s o f p a r t i a l d i f f e r e n t i a l
e q u a t i o n p r o b l e m s such as n o n - l i n e a r e l l i p t i c P D E s , systems o f e l l i p t i c P D E s , a n d
t i m e - d e p e n d e n t p a r a b o l i c or h y p e r b o l i c P D E s . W e w i l l also see i n t h e n e x t c h a p t e r
t h a t K a n s a ' s m e t h o d is w e l l - s u i t e d for e l l i p t i c p r o b l e m s w i t h v a r i a b l e coefficients.
W e w i l l come b a c k t o t h e use o f K a n s a ' s m e t h o d for t i m e - d e p e n d e n t p r o b l e m s i n
C h a p t e r 42.
38.2 A n Hermite-based Approach
T h e f o l l o w i n g s y m m e t r i c c o l l o c a t i o n m e t h o d is based o n t h e generalized H e r m i t e
i n t e r p o l a t i o n m e t h o d d e t a i l e d i n C h a p t e r 36. A s s u m e w e are g i v e n t h e same l i n e a r
e l l i p t i c P D E (38.3) w i t h D i r i c h l e t b o u n d a r y c o n d i t i o n s (38.4) as i n t h e previous
s e c t i o n o n K a n s a ' s m e t h o d . I n o r d e r t o be able t o a p p l y t h e r e s u l t s f r o m g e n e r a l i z e d
H e r m i t e i n t e r p o l a t i o n t h a t w i l l ensure t h e n o n - s i n g u l a r i t y o f t h e c o l l o c a t i o n m a t r i x
we propose t h e f o l l o w i n g e x p a n s i o n for t h e u n k n o w n f u n c t i o n u:
N T N
Hx)=J2c ^(\\ -$\\)k=Z +
j X j E cMWx-ZA)- (38-7)
3= 1 j=Nx +l
H e r e Nj denotes t h e n u m b e r o f nodes i n t h e i n t e r i o r o f O, a n d is t h e d i f f e r e n t i a l
o p e r a t o r used i n t h e d i f f e r e n t i a l e q u a t i o n ( 3 8 . 3 ) , b u t a c t i n g o n ip v i e w e d as a f u n c t i o n
of t h e second a r g u m e n t , i.e., dp is e q u a l t o C^-cp u p t o a possible difference i n s i g n .
T h u s , t h e l i n e a r f u n c t i o n a l s A i n (36.2) are g i v e n b y Xj = <5^. o , j = 1 , . . . ,Nx,
a n d Xj = 5 , $j j = N x + 1, . , N.
After enforcing the collocation conditions
Cu(Xi) = f(xi), Xi G X ,
u(xi) = g(xi), Xi G B,
we end u p w i t h a c o l l o c a t i o n m a t r i x A t h a t is o f t h e f o r m
Ar
j A c
A (38.8)
Ac< A
H e r e t h e four b l o c k s are g e n e r a t e d as follows:
OiccOy = C&<pi\\x -SIDU^,*^, Xi,j X,

(A )ij
c = dp(\\x - &||)|x=xi, Xi 6 X , j 6,
(Acshj = ^(p(\\xi-$\\)\ , 6=ij Xi,&B, & GX,
A^ = (p(\\xi - Xi,jeB.
N o t e t h a t we have i d e n t i f i e d t h e t w o sets X = X U B o f c o l l o c a t i o n p o i n t s a n d E o f
centers.
T h e m a t r i x A o f (38.8) is o f t h e same t y p e as t h e generalized H e r m i t e i n t e r p o l a -
t i o n m a t r i c e s ( 3 6 . 3 ) , a n d therefore n o n - s i n g u l a r as l o n g as ip is chosen a p p r o p r i a t e l y .
T h u s , v i e w e d u s i n g t h e n e w e x p a n s i o n (38.7) for w, t h e c o l l o c a t i o n a p p r o a c h is cer-
t a i n l y well-posed. A n o t h e r p o i n t i n favor o f t h e H e r m i t e - b a s e d a p p r o a c h is t h a t
t h e m a t r i x (38.8) is s y m m e t r i c as o p p o s e d t o t h e c o m p l e t e l y u n s t r u c t u r e d m a t r i x
(38.6) o f t h e same size used i n t h e n o n - s y m m e t r i c a p p r o a c h . T h i s p r o p e r t y is o f
v a l u e w h e n t r y i n g t o devise a n efficient i m p l e m e n t a t i o n o f t h e c o l l o c a t i o n m e t h o d .
A l s o n o t e t h a t a l t h o u g h A n o w consists o f f o u r b l o c k s , i t s t i l l is o f t h e same size,
n a m e l y N x N, as t h e c o l l o c a t i o n m a t r i x (38.6) o b t a i n e d for K a n s a ' s approach.
r
However, t h e s y m m e t r i c c o l l o c a t i o n m a t r i x is m o r e c o m p l i c a t e d t o assemble, i t re-

quires s m o o t h e r basis f u n c t i o n s t h a n t h e n o n - s y m m e t r i c K a n s a m e t h o d , a n d i t does
n o t l e n d itself v e r y n i c e l y t o t h e s o l u t i o n o f n o n - l i n e a r p r o b l e m s .
One a t t e m p t t o o b t a i n a n efficient i m p l e m e n t a t i o n o f t h e H e r m i t e - b a s e d c o l l o -
c a t i o n m e t h o d is a v a r i a t i o n o f t h e greedy a l g o r i t h m described i n S e c t i o n 3 3 . 1 . W e
refer t h e reader t o t h e o r i g i n a l p a p e r [ H o n et al. (2003)] for details.
38.3 E r r o r B o u n d s for S y m m e t r i c C o l l o c a t i o n
A convergence analysis for t h e s y m m e t r i c c o l l o c a t i o n m e t h o d was p r o v i d e d i n

[Franke a n d Schaback (1998a); F r a n k e a n d Schaback ( 1 9 9 8 b ) ] . T h e e r r o r e s t i m a t e s
established i n those papers r e q u i r e t h e s o l u t i o n o f t h e P D E t o be v e r y s m o o t h .
Therefore, one s h o u l d be able t o use meshfree r a d i a l basis f u n c t i o n c o l l o c a t i o n t e c h -
niques especially w e l l for ( h i g h - d i m e n s i o n a l ) P D E p r o b l e m s w i t h s m o o t h s o l u t i o n s
on possibly irregular domains. Due t o the k n o w n counterexamples from [ H o n and
Schaback (2001)] for t h e n o n - s y m m e t r i c m e t h o d , a convergence analysis is s t i l l lack-
i n g for t h a t m e t h o d . H o w e v e r , for a n a d a p t i v e v e r s i o n o f t h e n o n - s y m m e t r i c m e t h o d
Schaback r e c e n t l y a n a l y z e d t h e convergence i n [Schaback (2006a)].
I n [ W e n d l a n d (2005a)] one c a n f i n d t h e f o l l o w i n g convergence result for t h e
symmetric collocation method:
s
Theorem 38.1. Let fl C R be a polygonal and open region. Let ^ 0 be a
2 k 2
second-order linear elliptic differential operator with coefficients in C ( ~ ^ (fl) that
2 / c s
either vanish on fl or have no zero there. Suppose that 3> G C ( M ) is a strictly
positive definite function. Suppose further that the boundary value problem
u f in fl,
u g on dfl
has a unique solution u 6 jV<&(fl) for given f G C(fl) and g G C(dfl). Let u be the
approximate collocation solution of the form (38.7) based on $ = <p(|| | | ) . Then
fe 2
l l - w | U ( f ) < C7i - |MU/-*(fi)
0 0 2
for all sufficiently small h, where h is the larger of the fill distances in the interior
and on the boundary of fl, respectively.
T h e p r o o f uses t h e same techniques as i n C h a p t e r 14 a n d takes a d v a n t a g e o f a

"splitting theorem" t h a t permits s p l i t t i n g the error into a boundary error and an
e r r o r i n t h e i n t e r i o r . A s a consequence o f t h e p r o o f W e n d l a n d suggests t h a t the
c o l l o c a t i o n p o i n t s a n d centers be chosen so t h a t t h e f i l l distance o n t h e b o u n d a r y
is smaller t h a n i n t h e i n t e r i o r since t h e a p p r o x i m a t i o n orders differ b y a f a c t o r
(for d i f f e r e n t i a l o p e r a t o r s o f o r d e r ) . M o r e precisely, he suggests d i s t r i b u t i n g t h e
p o i n t s so t h a t
Some n u m e r i c a l evidence for convergence r a t e s o f t h e s y m m e t r i c c o l l o c a t i o n

m e t h o d is g i v e n b y t h e e x a m p l e s i n t h e n e x t c h a p t e r , a n d i n t h e p a p e r s [ J u m a r h o n
et al. ( 2 0 0 0 ) ; P o w e r a n d B a r r a c o ( 2 0 0 2 ) ] .
38.4 O t h e r Issues
Since t h e m e t h o d s d e s c r i b e d a b o v e w e r e b o t h o r i g i n a l l y used w i t h g l o b a l l y s u p -
p o r t e d basis f u n c t i o n s , t h e same concerns a b o u t s t a b i l i t y a n d n u m e r i c a l efficiency
a p p l y as for i n t e r p o l a t i o n p r o b l e m s . T h e t w o r e c e n t p a p e r s [ L i n g a n d K a n s a ( 2 0 0 4 ) ;
L i n g a n d K a n s a (2005)] address these issues. I n p a r t i c u l a r , t h e a u t h o r s d e v e l o p a
p r e c o n d i t i o n e r i n t h e s p i r i t o f t h e one d e s c r i b e d i n S e c t i o n 34.3, a n d d e s c r i b e t h e i r
experience w i t h a d o m a i n d e c o m p o s i t i o n a l g o r i t h m .
R e c e n t l y , M i r a n d a [ M i r a n d a (2004)] has s h o w n t h a t K a n s a ' s m e t h o d w i l l be
w e l l - p o s e d i f i t is c o m b i n e d w i t h so-called R-functions. T h i s i d e a was also u s e d
b y H o l l i g a n d his c o - w o r k e r s i n t h e i r d e v e l o p m e n t o f web-splines (see, e.g., [Hollig
(2003)]).
O t h e r recent p a p e r s i n v e s t i g a t i n g v a r i o u s aspects o f r a d i a l basis f u n c t i o n c o l l o c a -
t i o n are, e.g., [ C h e n g et al. ( 2 0 0 3 ) ; Fedoseyev et al. ( 2 0 0 2 ) ; K a n s a a n d H o n ( 2 0 0 0 ) ;
Larsson and Fornberg (2003); L e i t a o (2001); M a i - D u y a n d T r a n - C o n g (2001a);
Y o u n g et al. ( 2 0 0 4 ) ] .
F o r e x a m p l e , i n t h e p a p e r [Fedoseyev et al. (2002)] t h e a u t h o r s suggest t h a t
t h e c o l l o c a t i o n p o i n t s o n t h e b o u n d a r y s h o u l d also be used t o satisfy t h e P D E . T h e
m o t i v a t i o n for t h i s m o d i f i c a t i o n is t h e w e l l - k n o w n fact t h a t b o t h for i n t e r p o l a t i o n
a n d c o l l o c a t i o n w i t h r a d i a l basis f u n c t i o n s t h e e r r o r is largest n e a r t h e b o u n d a r y . I n
order t o prevent the collocation m a t r i x f r o m becoming t r i v i a l l y singular (by using
d u p l i c a t e c o l u m n s , i.e., basis f u n c t i o n s ) i t is suggested i n [Fedoseyev et al. (2002)]
t h a t t h e c o r r e s p o n d i n g centers l i e o u t s i d e t h e d o m a i n S7 ( t h u s c r e a t i n g a d d i t i o n a l
basis f u n c t i o n s ) . I n v a r i o u s n u m e r i c a l e x p e r i m e n t s t h i s s t r a t e g y is s h o w n t o i m p r o v e
the accuracy o f Kansa's n o n - s y m m e t r i c m e t h o d . We implement this approach i n
t h e n e x t c h a p t e r . H o w e v e r , i t s h o u l d be n o t e d t h a t t h e r e is once m o r e n o t h e o r e t -
i c a l f o u n d a t i o n for t h i s m o d i f i c a t i o n o f e i t h e r t h e n o n - s y m m e t r i c o r t h e s y m m e t r i c
method.
L a r s s o n a n d F o r n b e r g c o m p a r e K a n s a ' s basic c o l l o c a t i o n m e t h o d , t h e m o d i f i c a -
t i o n j u s t described, a n d t h e Hermite-based s y m m e t r i c approach m e n t i o n e d earlier
(see [Larsson a n d F o r n b e r g ( 2 0 0 3 ) ] ) . U s i n g m u l t i q u a d r i c basis f u n c t i o n s i n a s t a n -
d a r d i m p l e m e n t a t i o n t h e y c o n c l u d e t h a t t h e s y m m e t r i c m e t h o d is t h e m o s t accu-
rate, followed b y the n o n - s y m m e t r i c m e t h o d w i t h b o u n d a r y collocation. T h e reason
for t h i s is t h e b e t t e r c o n d i t i o n i n g o f t h e s y s t e m f o r t h e s y m m e t r i c m e t h o d . Lars-
son a n d F o r n b e r g also discuss a n i m p l e m e n t a t i o n o f t h e t h r e e m e t h o d s u s i n g t h e
complex Contour-Pade i n t e g r a t i o n m e t h o d mentioned i n Section 16.1. W i t h this
t e c h n i q u e s t a b i l i t y p r o b l e m s are o v e r c o m e , a n d i t t u r n s o u t t h a t b o t h t h e s y m m e t -
ric a n d t h e n o n - s y m m e t r i c m e t h o d p e r f o r m w i t h c o m p a r a b l e accuracy. Boundary
r
c o l l o c a t i o n o f t h e P D E y i e l d s a n i m p r o v e m e n t o n l y i f these c o n d i t i o n s are used as

a d d i t i o n a l e q u a t i o n s , i.e., b y i n c r e a s i n g t h e p r o b l e m size. I t s h o u l d also be n o t e d
t h a t o f t e n t h e m o s t a c c u r a t e results were achieved w i t h values o f t h e m u l t i q u a d r i c
shape p a r a m e t e r e t h a t w o u l d lead t o severe i l l - c o n d i t i o n i n g u s i n g a s t a n d a r d i m -
p l e m e n t a t i o n , a n d therefore these results c o u l d be achieved o n l y u s i n g t h e c o m p l e x
i n t e g r a t i o n m e t h o d . M o r e o v e r , i n [ L a r s s o n a n d F o r n b e r g (2003)] r a d i a l basis func-
t i o n c o l l o c a t i o n is deemed t o be far s u p e r i o r i n a c c u r a c y t o s t a n d a r d second-order
f i n i t e differences o r even a s t a n d a r d F o u r i e r - C h e b y s h e v p s e u d o s p e c t r a l m e t h o d .
L e i t a o applies t h e s y m m e t r i c c o l l o c a t i o n m e t h o d t o a f o u r t h - o r d e r Kirchhoff
p l a t e b e n d i n g p r o b l e m (see [ L e i t a o ( 2 0 0 1 ) ] ) a n d emphasizes t h e s i m p l i c i t y o f t h e
i m p l e m e n t a t i o n o f t h e r a d i a l basis f u n c t i o n c o l l o c a t i o n m e t h o d . M a i - D u y a n d T r a n -
C o n g suggest a c o l l o c a t i o n m e t h o d for w h i c h t h e basis f u n c t i o n s are t a k e n t o be
a n t i - d e r i v a t i v e s o f t h e u s u a l r a d i a l basis f u n c t i o n s (see [ M a i - D u y a n d T r a n - C o n g
(2001a)]). A n d , finally, i n [ Y o u n g et al. (2004)] t h e a u t h o r s discuss t h e s o l u t i o n o f
2 D a n d 3 D Stokes' systems b y a self-consistent i t e r a t i v e a p p r o a c h based o n K a n s a ' s
non-symmetric method.
l;.
Chapter 39
Non-Symmetric R B F Collocation
in MATLAB
I n t h i s a n d t h e n e x t t w o c h a p t e r s we present a n u m b e r o f M A T L A B i m p l e m e n t a t i o n s
for s t a n d a r d L a p l a c e / P o s s i o n p r o b l e m s , p r o b l e m s w i t h v a r i a b l e coefficients, a n d
p r o b l e m s w i t h m i x e d or piecewise defined b o u n d a r y c o n d i t i o n s . T h e n o n - s y m m e t r i c
K a n s a m e t h o d is discussed i n t h i s c h a p t e r . W e p r o v i d e a f a i r l y d e t a i l e d p r e s e n t a t i o n
since t h e M A T L A B code changes r a t h e r s i g n i f i c a n t l y f r o m one p r o b l e m t o a n o t h e r .
M o s t o f t h e f o l l o w i n g t e s t examples are s i m i l a r t o those s t u d i e d i n [ L i et al.
(2003)]. W e r e s t r i c t ourselves t o t w o - d i m e n s i o n a l e l l i p t i c p r o b l e m s whose a n a l y t i c
s o l u t i o n is r e a d i l y available a n d t h e r e f o r e can easily be verified. W e w i l l refer t o a
2
point x in R as (x,y).
39.1 Kansa's Non-Symmetric Collocation M e t h o d
E x a m p l e 3 9 . 1 . Consider t h e f o l l o w i n g Poisson p r o b l e m w i t h D i r i c h l e t b o u n d a r y
conditions:
(39.1)
u(x,y) = sin(Tnr), (x,y)GTi,

u(x,y) = 0, (x,y)eT ,
2
where T\ = {(x,y) : 0 < x < 1, y = 0 } a n d T 2 = dl\Ti. A s c a n easily be verified,

t h e exact s o l u t i o n is g i v e n b y
A M A T L A B p r o g r a m for t h e n o n - s y m m e t r i c c o l l o c a t i o n s o l u t i o n o f t h i s p r o b l e m
u s i n g inverse m u l t i q u a d r i c R B F s is p r o v i d e d as P r o g r a m 3 9 . 1 . W h i l e t h i s p r o g r a m
s t i l l is o f t h e same general s t r u c t u r e as earlier i n t e r p o l a t i o n p r o g r a m s we n o w r e q u i r e
n o t o n l y a d e f i n i t i o n o f t h e basic f u n c t i o n , b u t also o f i t s L a p l a c i a n (see l i n e 2 ) . O n
lines 3 a n d 4 we define t h e exact s o l u t i o n a n d i t s L a p l a c i a n for t h i s test p r o b l e m .
N o t e t h a t w h e n we define t h e r i g h t - h a n d side o f t h e p r o b l e m , i n s t e a d o f b r e a k i n g t h e
b o u n d a r y c o n d i t i o n d o w n i n t o t w o pieces as g i v e n i n t h e p r o b l e m d e f i n i t i o n above
353
Li
3
we s i m p l y evaluate t h e k n o w n s o l u t i o n o n t h e b o u n d a r y (see l i n e 26 o f t h e c o d e ) . O f
course, t h i s is n o t possible i n g e n e r a l since t h e s o l u t i o n w i l l n o t b e k n o w n . I n t h a t
case one w o u l d have t o replace l i n e 26 b y t h e s l i g h t l y m o r e c o m p l i c a t e d expression
rhs = [Lu(intdata(:,1),intdata(:,2));...
sin(pi*bdydata(l:sn-1,1)); zeros(3*(sn-1),1)];
I n o r d e r t o s t a y as close as p o s s i b l e t o t h e code used i n e a r l i e r p r o g r a m s we

l o a d t h e ( i n t e r i o r ) c o l l o c a t i o n p o i n t s f r o m d a t a files. F o r e x a m p l e , o n l i n e 7 we
2
read N = 289 u n i f o r m l y spaced p o i n t s i n [0, l ] f r o m t h e file D a t a 2 D _ 2 8 9 u i n t o
t h e v a r i a b l e d s i t e s . A s a l w a y s , t h e centers for t h e basis f u n c t i o n s associated with
i n t e r i o r p o i n t s are t a k e n t o be t h e same as t h e c o l l o c a t i o n (i.e., d a t a ) sites.
H o w e v e r , as e x p l a i n e d i n t h e p r e v i o u s c h a p t e r , we n o w also r e q u i r e c o l l o c a t i o n
p o i n t s a n d centers t o f i t t h e b o u n d a r y c o n d i t i o n s . T h e r e are several a p p r o a c h e s we
could take to accomplish this:
W e c o u l d use those c o l l o c a t i o n p o i n t s r e a d f r o m file t h a t lie o n t h e b o u n d a r y as

b o u n d a r y collocation points ( a n d centers). T h i s means identifying those points
i n t h e a r r a y d s i t e s . T h i s a p p r o a c h w o u l d b e t h e closest i n s p i r i t t o t h e t h e o r y
discussed i n t h e p r e v i o u s c h a p t e r . I n M A T L A B one c o u l d easily code t h i s w i t h
the commands
indx = f i n d ( d s i t e s ( : , 1 ) = = 0 I d s i t e s ( : , 1 ) = = 1 | ...
dsites(:,2)==0 I dsites(:,2)==1);
bdydata = d s i t e s ( i n d x , : ) ;
intdata = dsites(setdiff([1:N],indx), :) ;
bdyctrs = bdydata;
H o w e v e r , we d o n o t f o l l o w t h i s a p p r o a c h here.
W e c a n create a d d i t i o n a l c o l l o c a t i o n p o i n t s for t h e b o u n d a r y c o n d i t i o n s . T h e s e
p o i n t s c a n lie a n y w h e r e o n t h e b o u n d a r y . W e t a k e t h e m t o be e q u a l l y spaced
(see lines 9 - 1 1 ) . N o t e t h a t we arrange the b o u n d a r y points i n a counter-
clockwise m a n n e r s t a r t i n g f r o m t h e o r i g i n . N o w w e have several choices for
t h e b o u n d a r y centers:
W e c a n l e t t h e b o u n d a r y centers c o i n c i d e w i t h t h e b o u n d a r y c o l l o c a t i o n
points. However, this approach w i l l lead t o a singular collocation m a t r i x
for u n i f o r m i n t e r i o r p o i n t s (since t h a t set a l r e a d y c o n t a i n s p o i n t s o n t h e
b o u n d a r y , a n d t h e r e f o r e d u p l i c a t e c o l u m n s are c r e a t e d ) . N o t e , however,
t h a t t h i s a p p r o a c h w o r k s fine i f we t a k e t h e i n t e r i o r c o l l o c a t i o n p o i n t s t o
be H a l t o n p o i n t s (since t h o s e p o i n t s d o n o t lie o n t h e b o u n d a r y o f t h e u n i t
s q u a r e ) . T h i s a p p r o a c h c a n be r e a l i z e d b y r e p l a c i n g lines 1 2 - 1 4 b y
bdyctrs = bdydata;
W e c a n create a d d i t i o n a l b o u n d a r y centers outside t h e d o m a i n (see lines 12

15). W e f o l l o w t h i s a p p r o a c h i n m o s t o f o u r e x p e r i m e n t s since i t seems
39. Non-Symmetric RBF Collocation in MATLAB 355
to provide a slightly more accurate solution. Placing boundary centers

away from the boundary has been recommended recently by a number of
authors. Note that this approach takes us into the realm of R B F methods
for which the centers differ from the data sites (or collocation points), and
we stated earlier that not much is known theoretically about this setting
(i.e., invertibility of system matrices or error bounds). It is an open problem
how to find the best location for the boundary centers. We take them a
small distance perpendicularly from the boundary collocation points (see
Figure 39.1).
P r o g r a m 3 9 . 1 . KansaLaplace_2D .m
/ KansaLaplace_2D
% S c r i p t t h a t performs Kansa c o l l o c a t i o n f o r 2D L a p l a c e equation
% C a l l s on: DistanceMatrix
% IMQ RBF and i t s L a p l a c i a n
1 r b f = @(e,r) 1 . / s q r t ( l + ( e * r ) . ~ 2 ) ; ep = 3;
2 Lrbf = @(e,r) e ~ 2 * ( ( e * r ) . " 2 - 2 ) . / ( l + ( e * r ) . ~ 2 ) . " ( 5 / 2 ) ;
% Exact s o l u t i o n and i t s L a p l a c i a n f o r t e s t problem
3 u = @(x,y) s i n ( p i * x ) . * c o s ( p i * y / 2 ) ;
4 Lu = @(x,y) - 1 . 2 5 * p i ~ 2 * s i n ( p i * x ) . * c o s ( p i * y / 2 ) ;
% Number and type of c o l l o c a t i o n p o i n t s
5 N = 289; gridtype = 'u';
6 n e v a l = 40;
% Load ( i n t e r i o r ) c o l l o c a t i o n p o i n t s
, 0 0
7 name = s p r i n t f ( Data2D_ / d / s' ,N .gridtype) ; load (name) ;
0 0
8 intdata = dsites;
% A d d i t i o n a l ( e q u a l l y spaced) boundary c o l l o c a t i o n p o i n t s
9 sn = s q r t ( N ) ; b d y l i n = l i n s p a c e ( 0 , 1 , s n ) ' ;
10 bdyO = z e r o s ( s n - 1 , 1 ) ; bdyl = o n e s ( s n - 1 , 1 ) ;
11a bdydata = [ b d y l i n ( l : e n d - 1 ) bdyO; bdyl b d y l i n d :end-1) ; . . .
lib f l i p u d ( b d y l i n ( 2 : e n d ) ) bdyl; bdyO f l i p u d ( b d y l i n ( 2 : e n d ) ) ] ;
% Create a d d i t i o n a l boundary c e n t e r s OUTSIDE the domain
12 h = l / ( s n - l ) ; b d y l i n = ( h : h : l - h ) ' ;
13 bdyO = -h*ones(sn-2,1); bdyl = ( l + h ) * o n e s ( s n - 2 , 1 ) ;
14a b d y c t r s = [-h -h; b d y l i n bdyO; 1+h -h; bdyl b d y l i n ; . . .
14b 1+h 1+h; f l i p u d ( b d y l i n ) bdyl; -h 1+h; bdyO f l i p u d ( b d y l i n ) ] ;
15 c t r s .= [ i n t d a t a ; b d y c t r s ] ;
% Create neval-by-neval e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
17 epoints = [ x e ( : ) y e ( : ) ] ;
20 exact = u ( e p o i n t s ( : , 1 ) , e p o i n t s ( : , 2 ) ) ;
% Compute b l o c k s f o r c o l l o c a t i o n m a t r i x
21 DM_intdata = D i s t a n c e M a t r i x ( i n t d a t a , c t r s ) ;
22 LCM = Lrbf(ep,DM_intdata);
23 DM_bdydata = D i s t a n c e M a t r i x ( b d y d a t a , c t r s ) ;
24 BCM = rbf(ep,DM_bdydata);
25 CM = [LCM; BCM] ;
'/. Create r i g h t - h a n d s i d e
26a r h s = [ L u ( i n t d a t a ( : , 1 ) , i n t d a t a ( : , 2 ) ) ; ...
26b u(bdydata(:,1),bdydata(:,2))];
/ Compute RBF s o l u t i o n
/ Compute maximum e r r o r on e v a l u a t i o n g r i d
0
30 f p r i n t f ('RMS e r r o r : '/ e\n', r m s _ e r r )
0
31 f p r i n t f ('Maximum e r r o r : 7e\n' , maxerr)

% P l o t c o l l o c a t i o n p o i n t s and c e n t e r s
1
32 hold on; p l o t ( i n t d a t a ( : , 1 ) , i n t d a t a ( : , 2 ) , ' b o ) ;
33 p l o t ( b d y d a t a ( : , 1 ) , b d y d a t a ( : , 2 ) , ' r x ' ) ;
34 p l o t ( b d y c t r s ( : , 1 ) , b d y c t r s ( : , 2 ) , ' g x ' ) ; h o l d o f f
35 fview = [-30,30]; % v i e w i n g angles f o r p l o t
37 P l o t E r r o r 2 D ( x e , y e , P f , e x a c t , m a x e r r , n e v a l , f v i e w ) ;
I n Tables 39.1 a n d 39.2 we l i s t R M S - e r r o r s a n d c o n d i t i o n n u m b e r s for t h e n o n -

s y m m e t r i c c o l l o c a t i o n s o l u t i o n o f t h e P D E p r o b l e m (39.1). I n T a b l e 39.1 a n d t h e
r i g h t p a r t o f T a b l e 39.2 we present r e s u l t s for c o l l o c a t i o n w i t h inverse m u l t i q u a d r i c
R B F s u s i n g a shape p a r a m e t e r o f e = 3, N = 289 i n t e r i o r , a n d a n a d d i t i o n a l 64
b o u n d a r y c o l l o c a t i o n p o i n t s . I n T a b l e 39.1 t h e i n t e r i o r p o i n t s are i r r e g u l a r l y spaced
H a l t o n p o i n t s , w h i l e i n T a b l e 39.2 w e use u n i f o r m l y spaced i n t e r i o r p o i n t s . The
b o u n d a r y centers are p l a c e d o u t s i d e t h e d o m a i n for t h e results i n T a b l e 39.2 (see
t h e e x p l a n a t i o n above a n d t h e left p a r t o f F i g u r e 39.1). I n T a b l e 39.1 we c o m p a r e
t h e effect o f p l a c i n g t h e b o u n d a r y centers d i r e c t l y o n t h e b o u n d a r y ( c o i n c i d e n t w i t h
t h e b o u n d a r y c o l l o c a t i o n p o i n t s ) as o p p o s e d t o p l a c e m e n t o u t s i d e t h e d o m a i n as i n
F i g u r e 39.1.
T h e left p a r t o f T a b l e 39.2 c o m p a r e s t h e use o f Gaussians ( w i t h t h e same shape
p a r a m e t e r e = 3) t o inverse m u l t i q u a d r i c s . F o r Gaussians we replace lines 1 a n d 2
o f P r o g r a m 39.1 b y
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; ep = 3;
Table 39.1 Non-symmetric collocation solution of Example 39.1 with I M Q s , e = 3

and interior Halton points.
centers on boundary centers outside

N
(interior points) RMS-error cond(A) RMS-error cond(A)
9 5.642192e-002 5.276474e+002 6.029293e-002 4.399608e+002

25 1.039322e-002 3.418858e+003 4.187975e-003 2.259698e+003
81 2.386062e-003 1.726995e+006 4.895870e-004 3.650369e+005
289 4.904715e-005 1.706884e+010 2.668524e-005 5.328110e+009
1089 3.676576e-008 1.446865e+018 1.946954e-008 5.015917e+017
2 Lrbf = @(e,r) 4 * e ~ 2 * e x p ( - ( e * r ) ~ 2 ) . * ( ( e * r ) . " 2 - 1 ) ;
Table 39.2 Non-symmetric collocation solution of Example 39.1 with Gaussians and
I M Q s , = 3 and uniform interior points and boundary centers outside the domain.
Gaussian IMQ
3 x3 1.981675e-001 1.258837e+003 1.526456e-001 2.794516e+002

5 x5 7.199931e-003 4.136193e+003 6.096534e-003 2.409431e+003
9 x9 1.947108e-004 2.529708e+010 8.071271e-004 8.771630e+005
17 x 17 4.174290e-008 5.335000e+019 3.219110e-005 5.981238e+010
33 x 33 1.408750e-005 7.106505e+020 1.552047e-007 1.706638e+020
Several observations c a n be m a d e b y l o o k i n g at Tables 39.1 a n d 39.2. T h e use o f

H a l t o n p o i n t s i n s t e a d o f u n i f o r m p o i n t s seems t o be beneficial since b o t h t h e e r r o r s
a n d t h e c o n d i t i o n n u m b e r s are s m a l l e r (c.f. t h e r i g h t p a r t o f T a b l e 39.1 vs. t h e r i g h t
p a r t o f T a b l e 3 9 . 2 ) . P l a c e m e n t o f t h e b o u n d a r y centers o u t s i d e t h e d o m a i n seems
t o be advantageous since a g a i n b o t h t h e e r r o r s a n d t h e c o n d i t i o n n u m b e r s decrease
(c.f. T a b l e 3 9 . 1 ) . A l s o , t h e last r o w o f T a b l e 39.2 seems t o i n d i c a t e t h a t Gaussians
are m o r e p r o n e t o i l l - c o n d i t i o n i n g t h a n inverse m u l t i q u a d r i c s .
O f course, these are r a t h e r s u p e r f i c i a l o b s e r v a t i o n s based o n o n l y a few n u m e r i c a l
experiments. For m a n y o f these c l a i m s t h e r e is n o t h e o r e t i c a l f o u n d a t i o n , a n d
m a n y m o r e e x p e r i m e n t s w o u l d be needed t o m a k e a m o r e conclusive s t a t e m e n t (for
e x a m p l e , no a t t e m p t was m a d e here t o f i n d t h e best a p p r o x i m a t i o n s , i.e., o p t i m i z e
t h e value o f t h e shape p a r a m e t e r ) . A l s o , one c o u l d e x p e r i m e n t w i t h different values
of t h e shape p a r a m e t e r o n t h e b o u n d a r y a n d i n t h e i n t e r i o r (as suggested, e.g., i n
[ K a n s a a n d C a r l s o n (1992)]).
T h e c o l l o c a t i o n p o i n t s a n d centers used here ( a n d i n m o s t o f t h e f o l l o w i n g ex-
amples) are d i s p l a y e d i n t h e left p l o t o f F i g u r e 3 9 . 1 , w h i l e t h e r i g h t p l o t c o n t a i n s
a s o l u t i o n for N 289 i n t e r i o r H a l t o n p o i n t s c o r r e s p o n d i n g t o r o w 4 i n t h e r i g h t
part of Table 39.1.
X 10
1.2
1
o oo o oo o oo o o o o o o
0.8 o oo o oo o oo o o o o o o
o oo o oo o oo o o o o o o 0.5-
0.6 o oo o oo o oo oo o o o o
z
o oo o oo o oo o o o o o o 9
o oo o oo o oo oo o o o o
0.4 o oo o oo o oo o o o o o e UJ
0.2 o oo o oo o oo o o o o o o
o oo o oo o oo oo o o o o
0
f i l l
-0.2
-0.2 0.2 0.4 0.6 0.8 1.2
Fig. 39.1 Collocation points (interior: blue circles, boundary: red crosses) and centers (interior:
blue circles, boundary: green crosses) (left) and non-symmetric R B F collocation solution (right)
for Example 39.1 using I M Q s with e 3 and N = 289 interior points.
E x a m p l e 3 9 . 2 . Consider t h e f o l l o w i n g e l l i p t i c e q u a t i o n w i t h v a r i a b l e coefficients
a n d homogeneous D i r i c h l e t b o u n d a r y c o n d i t i o n s :
d ( d \ d ( d \
2
( a(x,y) u(x,y)\ + ^ ( b(x, y ) u(x, y) j = f{x,y), (x,y) e fl = [0, l ] ,
u(x, y) = 0, (x, y) G T = dfl,
where
x y 2 2
f{x, y) = - 1 6 z ( l - x){3 - 2y)e ~ + 3 2 y ( l - y)(Sx + y - x - 2 ) ,
a n d t h e coefficients are g i v e n b y
2 2 x y
a(x, y) = 2 - x - y , b(x, y) = e~.
A s can easily be verified, t h e e x a c t s o l u t i o n for t h i s p r o b l e m is g i v e n b y
u(x,y) = 1 6 x ( l - x)y{l - y).
T h e c o r r e s p o n d i n g M A T L A B p r o g r a m is l i s t e d as P r o g r a m 39.2. T h e d e f i n i t i o n
section o f t h i s p r o g r a m (lines 1-9) is m u c h longer t h a n before since we need t o
w o r k w i t h first a n d second-order p a r t i a l d e r i v a t i v e s o f t h e basic f u n c t i o n . A l s o , t h e
coefficients a a n d b a n d t h e i r p a r t i a l s are r e q u i r e d .
W h i l e m o s t o f t h e r e m a i n d e r o f t h e p r o g r a m is i d e n t i c a l t o t h e p r e v i o u s one,
t h e assembly o f t h e c o l l o c a t i o n m a t r i x (lines 2 6 - 3 2 ) is m u c h m o r e i n v o l v e d since
we need t o a p p l y t h e d i f f e r e n t i a l o p e r a t o r t o t h e basis f u n c t i o n s (see l i n e 30 for
t h e c o m p u t a t i o n o f t h e b l o c k LCM w h i c h c o r r e s p o n d s t o t h e b l o c k Ac i n o u r earlier
discussion ( 3 8 . 6 ) ) .
P r o g r a m 3 9 . 2 . K a n s a E l l i p t i c V C J 2 D .m
% KansaEllipticVC_2D
% S c r i p t t h a t performs K a n s a c o l l o c a t i o n f o r 2D e l l i p t i c PDE
% with v a r i a b l e c o e f f i c i e n t s
% C a l l s on: DistanceMatrix, DifferenceMatrix
% IMQ RBF and i t s d e r i v a t i v e s
1 r b f = @(e,r) 1 . / s q r t ( l + ( e * r ) . ~ 2 ) ; ep = 3;
2 dxrbf = @(e,r,dx) - d x * e ~ 2 . / ( l + ( e * r ) . ~ 2 ) . " ( 3 / 2 ) ;
3 dyrbf = @(e,r,dy) - d y * e ~ 2 . / ( l + ( e * r ) . ~ 2 ) . " ( 3 / 2 ) ;
4a dxxrbf = @(e,r,dx) e ~ 2 * ( 3 * ( e * d x ) . ~ 2 - l - ( e * r ) . ~ 2 ) . / . . .
4b (l+(e*r).~2)."(5/2);
5a dyyrbf = @(e,r,dy) e ~ 2 * ( 3 * ( e * d y ) . " 2 - 1 - ( e * r ) . ~ 2 ) . / . . .
5b (l+(e*r).~2)."(5/2);
/ T e s t problem input ( r i g h t - h a n d s i d e , c o e f f i c i e n t s )
0
6 u = @(x,y) 1 6 * x . * ( l - x ) . * y . * ( l - y ) ;
7a Lu = @(x,y) - 1 6 * x . * e x p ( x - y ) . * ( l - x ) . * ( 3 - 2 * y ) + . . .
7b 32*y.*(1-y).*(3*x."2+y.~2-x-2);
8 a = @(x,y) 2-x."2-y."2; ax = @(x,y) -2*x;
9 b = @(x,y) e x p ( x - y ) ; by = @(x,y)-exp(x-y);
10 N = 289; gridtype = 'h';
11 n e v a l = 40;
0
12 name = s p r i n t f ( 'Data2D_yod / s' ,N, g r i d t y p e ) ; l o a d (name) ;
0
13 i n t d a t a = d s i t e s ;
% A d d i t i o n a l boundary c o l l o c a t i o n p o i n t s
5
14 sn = s q r t ( N ) ; b d y l i n = l i n s p a c e ( 0 , 1 , s n ) ;
16a bdydata = [ b d y l i n d : end-1) bdyO; bdyl b d y l i n ( l : end-1) ;.. .
16b f l i p u d ( b d y l i n ( 2 : e n d ) ) b d y l ; bdyO f l i p u d ( b d y l i n ( 2 : e n d ) ) ] ;
% Create a d d i t i o n a l boundary c e n t e r s OUTSIDE the domain
18 bdyO = -h*ones(sn-2,1); bdyl = ( l + h ) * o n e s ( s n - 2 , 1 ) ;
19a bdyctrs = [-h -h; b d y l i n bdyO; 1+h -h; bdyl b d y l i n ; . . .
19b 1+h 1+h; f l i p u d ( b d y l i n ) b d y l ; -h 1+h; bdyO f l i p u d ( b d y l i n ) ] ;
20 c t r s = [ i n t d a t a ; b d y c t r s ] ;
/ i n the u n i t square
0
22 epoints = [ x e ( : ) y e ( : ) ] ;
% Compute blocks f o r c o l l o c a t i o n matrix
26 DM_intdata = D i s t a n c e M a t r i x ( i n t d a t a , c t r s ) ;
27 DM_bdydata = DistanceMatrix(bdydata,ctrs);
28 dx_intdata = D i f f e r e n c e m a t r i x ( i n t d a t a ( : , 1 ) , c t r s ( : , 1 ) ) ;
29 dy_intdata = D i f f e r e n c e m a t r i x ( i n t d a t a ( : , 2 ) , c t r s ( : , 2 ) ) ;
30a LCM = d i a g ( a x ( i n t d a t a ( : , 1 ) ) ) * ...
30b dxrbf(ep,DM_intdata,dx_intdata) + ...
30c d i a g ( a ( i n t d a t a ( : , 1 ) , i n t d a t a ( : , 2 ) ) ) * ...
30d dxxrbf(ep,DM_intdata,dx_intdata) + ...
30e d i a g ( b y ( i n t d a t a ( : , 1 ) , i n t d a t a ( : , 2 ) ) ) * ...
30f dyrbf(ep,DM_intdata,dy_intdata) + ...
30g d i a g ( b ( i n t d a t a ( : , 1 ) , i n t d a t a ( : , 2 ) ) ) * ...
30h dyyrbf(ep,DM_intdata,dy_intdata);
31 BCM = rbf(ep,DM_bdydata);
32 CM = [LCM; BCM];
7, C r e a t e r i g h t - h a n d s i d e
33 rhs = [Lu(intdata(:,1),intdata(:,2)); zeros(4*(sn-1),1)];
7, RBF s o l u t i o n
37 f p r i n t f ('RMS e r r o r : 7,e\n', r m s . e r r )
38 f p r i n t f ( 'Maximum e r r o r : 7 e\n' , maxerr)
0
7 P l o t c o l l o c a t i o n p o i n t s and c e n t e r s
39 hold on; p l o t ( i n t d a t a ( : , 1 ) , i n t d a t a ( : , 2 ) , ' b o ' ) ;
40 plot(bdydata(:,1),bdydata(:,2),'rx');
41 plot(bdyctrs(:,1).bdyctrs(:,2),'gx'); hold off
42 fview = [-30,30]; % v i e w i n g angles f o r p l o t
I n T a b l e 39.3 w e c o m p a r e t h e s o l u t i o n o b t a i n e d w i t h Gaussians a n d inverse

m u l t i q u a d r i c s based o n i n t e r i o r H a l t o n p o i n t s . T h e b o u n d a r y centers are t a k e n t o
lie o u t s i d e t h e d o m a i n as i n F i g u r e 3 9 . 1 . A g a i n , t h e s o l u t i o n w i t h inverse multi-
q u a d r i c s is s l i g h t l y b e t t e r c o n d i t i o n e d . F o r Gaussians w e need t o replace lines 1-5
o f P r o g r a m 39.2 b y
1 r b f = @(e,r) e x p ( - ( e * r ) . ~ 2 ) ; ep = 3;
2 dxrbf = @(e,r,dx) - 2 * d x * e ~ 2 . * e x p ( - ( e * r ) . " 2 ) ;
3 dyrbf = @(e,r,dy) - 2 * d y * e ~ 2 . * e x p ( - ( e * r ) . ~ 2 ) ;
4 dxxrbf = @(e,r,dx) 2 * e ~ 2 * ( 2 * ( e * d x ) . ~ 2 - l ) . * e x p ( - ( e * r ) . ~ 2 ) ;
5 dyyrbf = @(e,r,dy) 2*e~2*(2*(e*dy)."2-1).*exp(-(e*r).~2);
T h e t o p p a r t o f F i g u r e 39.2 c o n t a i n s p l o t s o f t h e a p p r o x i m a t e s o l u t i o n a n d
m a x i m u m e r r o r for t h e inverse m u l t i q u a d r i c s o l u t i o n based o n N = 289 i n t e r i o r
Table 39.3 Solution of Example 39.2 with Gaussians and I M Q s , e = 3 and interior
Halton points.
Gaussian IMQ
9 6.852103e-002 8.874341e+003 1.123770e-001 6.954910e+002

25 1.091888e-002 4.898291e+003 1.123575e-002 3.302471e+003
81 1.854386e-004 1.286993e+009 1.370992e-003 4.992219e+005
289 8.445637e-007 7.031011e+019 8.105109e-005 7.527456e+009
1089 2.559824e-005 4.553162e-f020 7.041415e-008 7.785955e+017
and 64 b o u n d a r y p o i n t s .
E x a m p l e 39.3. Consider t h e Poisson p r o b l e m w i t h m i x e d b o u n d a r y c o n d i t i o n s

2
V u(x, y) = -5.4x, (x, y)Efl= 2
[0, l ] ,
u(x,y) = 0, (x,y)er UT , 1 3
u{x,y) = 0.1, (x,y)eT , 2
u(x,y) = l, (x,y)eT ,4
where
r i = i(x,y) 0 < x < 1, y = 0},

r 2 = {(x,y) : x = 1, 0 < y < 1},
r 3
= {(x,y) : 0 < x < 1, y = 1},
r 4 = {{x,y) : x = 0, 0 < y < ! }
For t h i s p r o b l e m t h e exact s o l u t i o n is g i v e n b y
u(x,y) = 1 - 0.9a; . 3
N o t e t h a t t h e n o r m a l d e r i v a t i v e o n t h e edges T i a n d T3 is g i v e n b y ^ and
^1 respectively. T h e r e f o r e , for t h e M A T L A B p r o g r a m w e r e q u i r e t h e y - p a r t i a l o f
t h e basic f u n c t i o n i n a d d i t i o n t o i t s d e f i n i t i o n a n d i t s L a p l a c i a n (see lines 1-3 o f
P r o g r a m 39.3). A g a i n , t h e m a i n difference i n t h e code is i n t h e assembly o f t h e
c o l l o c a t i o n m a t r i x o n lines 22-30. N o t e t h a t t h i s t i m e we need t o d e a l c a r e f u l l y
w i t h t h e b o u n d a r y c o n d i t i o n s a n d r i g h t - h a n d side (see lines 26-29 a n d 30). I t is
i m p o r t a n t t h a t t h e o r i e n t a t i o n o f t h e b o u n d a r y p o i n t s is consistent.
P r o g r a m 3 9 . 3 . KansaLaplaceMixedBC_2D .m
/. KansaLaplaceMixedBC_2D
"/, S c r i p t t h a t performs Kansa c o l l o c a t i o n f o r 2D L a p l a c e e q u a t i o n
*/, w i t h mixed BCs
IMQ RBF and i t s L a p l a c i a n

1 r b f = @(e,r) 1 . / s q r t ( l + ( e * r ) . ~ 2 ) ; ep = 3;
2 dyrbf = <3(e,r,dy) -dy*e~2. / ( l + ( e * r ) . "2) . ~ (3/2) ;
3 L r b f = @(e,r) e " 2 * ( ( e * r ) . " 2 - 2 ) . / ( l + ( e * r ) . ~ 2 ) . ~ ( 5 / 2 ) ;
% E x a c t s o l u t i o n and i t s L a p l a c i a n f o r t e s t problem
4 u = @(x,y) l-0.9*x.~3+0*y;
5 Lu = @(x,y) -5.4*x+0*y;
6 N = 289; g r i d t y p e = 'h';
7 n e v a l = 40;
8 name = s p r i n t f ( D a t a 2 D _ y o d y o S ' , N , g r i d t y p e ) ; l o a d (name) ;
,
9 intdata = dsites;
5
10 s n = s q r t ( N ) ; b d y l i n = l i n s p a c e ( 0 , 1 , s n ) ;
12a bdydata = [ b d y l i n d : end-1) bdyO; bdyl b d y l i n d : end-1) ; ...
12b f l i p u d ( b d y l i n ( 2 : e n d ) ) b d y l ; bdyO f l i p u d ( b d y l i n ( 2 : e n d ) ) ]
% Create a d d i t i o n a l boundary c e n t e r s OUTSIDE t h e domain
J
13 h = l / ( s n - l ) ; b d y l i n = ( h : h : l - h ) ;
14 bdyO = - h * o n e s ( s n - 2 , l ) ; bdyl = (1+h)*ones(sn-2,1);
15a b d y c t r s = [-h -h; b d y l i n bdyO; 1+h -h; bdyl b d y l i n ; ...
15b 1+h 1+h; f l i p u d ( b d y l i n ) b d y l ; -h 1+h; bdyO f l i p u d ( b d y l i n
/ Create n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
0
18 epoints = [ x e ( : ) y e ( : ) ] ;
22 DM_intdata = D i s t a n c e M a t r i x d n t d a t a , c t r s ) ;
23 DM_bdydata = D i s t a n c e M a t r i x ( b d y d a t a , c t r s ) ;
24 dy_bdydata = D i f f e r e n c e m a t r i x ( b d y d a t a ( : , 2 ) , c t r s ( : , 2 ) ) ;
25 LCM = Lrbf(ep,DM_intdata);
26 BCM1 = -dyrbf(ep,DM_bdydata(l:sn-1,:),dy_bdydata(l:sn-1,:));
27 BCM2 = rbf(ep,DM_bdydata(sn:2*sn-2,:));
28a BCM3 = dyrbf(ep,DM_bdydata(2*sn-l:3*sn-3,:),...
28b dy_bdydata(2*sn-l:3*sn-3,:));
29 BCM4 = rbf(ep,DM_bdydata(3*sn-2rend,:));
30 CM = [LCM; BCM1; BCM2; BCM3; BCM4];

7, Create right-hand s i d e
31a rhs = [ L u ( i n t d a t a ( : , 1 ) , i n t d a t a ( : , 2 ) ) ; z e r o s ( s n - 1 , 1 ) ; ...
31b 0.1*ones(sn-l,l); zeros(sn-1,1); ones(sn-l,1)];
/. RBF s o l u t i o n
7. Compute maximum e r r o r on e v a l u a t i o n g r i d
35 f p r i n t f ('RMS e r r o r : /e\n', r m s _ e r r )
0
36 fprintf('Maximum e r r o r : 7.e\n', maxerr)

7, P l o t c o l l o c a t i o n p o i n t s and c e n t e r s
38 plot(bdydata(:,1),bdydata(:,2),'rx');
39 p l o t ( b d y c t r s ( : , 1 ) , b d y c t r s ( : , 2 ) , ' g x ' ) ; h o l d off
40 fview = [-30,30]; 7 viewing angles f o r p l o t
I n T a b l e 39.4 we a g a i n c o m p a r e t h e use o f Gaussians a n d inverse m u l t i q u a d r i c s

on a set o f N 9 , 2 5 , 8 1 , 2 8 9 a n d 1089 i n t e r i o r H a l t o n p o i n t s ( w i t h a d d i t i o n a l
b o u n d a r y centers o u t s i d e t h e d o m a i n ) . A s i n t h e p r e v i o u s e x p e r i m e n t s t h e G a u s s i a n
s o l u t i o n is s l i g h t l y i n f e r i o r i n t e r m s o f s t a b i l i t y for t h e same value o f t h e shape
parameter.
Table 39.4 Non-symmetric collocation solution of Example 39.3 with Gaussians and
I M Q s , e = 3 and interior Halton points.
Gaussian IMQ
(interior points) RMS-error cond(A) RMS-error cond(,4)
9 3.423330e-001 5.430073e+003 7.937403e-002 2.782348e+002

25 1.065826e-002 1.605086e+003 5.605445e-003 1.680888e+003
81 5.382387e-004 3.684159e+008 1.487160e-003 2.611650e+005
289 6.181855e-006 1.452124e+019 1.822077e-004 3.775455e+009
1089 2.060470e-006 1.628262e+021 1.822221e-007 3.155751e+017
I n t h e b o t t o m p a r t o f F i g u r e 39.2 we show t h e inverse m u l t i q u a d r i c s o l u t i o n for

AT = 289 i n t e r i o r p o i n t s a l o n g w i t h i t s m a x i m u m e r r o r . N o t e t h a t (even t h o u g h t h e
p r o b l e m has a s y m m e t r i c s o l u t i o n ) t h e a p p r o x i m a t e s o l u t i o n is n o t q u i t e s y m m e t r i c
(as d e m o n s t r a t e d b y t h e e r r o r p l o t , c.f. also t h e t o p p a r t o f F i g u r e 3 9 . 2 ) .
y 0 o x y 0 o
Fig. 39.2 Top: Non-symmetric collocation solution (left) and error plot (right) for Example 39.2
using I M Q s with e = 3 and N = 289 interior Halton points. Bottom: Approximate solution (left)
and error plot (right) for Example 39.3 using I M Q s with e 3 and AT = 289 interior Halton points.
I n [ L i et al. (2003)] t h e a u t h o r s r e p o r t t h a t t h e n o n - s y m m e t r i c c o l l o c a t i o n
s o l u t i o n for t h i s p r o b l e m w i t h m u l t i q u a d r i c R B F s is several o r d e r s o f m a g n i t u d e
m o r e a c c u r a t e t h a n a s o l u t i o n w i t h piecewise l i n e a r f i n i t e elements u s i n g t h e same
n u m b e r o f nodes.
Chapter 40
Symmetric R B F Collocation in MATLAB
I n t h i s c h a p t e r we discuss t h e i m p l e m e n t a t i o n o f t h e H e r m i t e - b a s e d symmetric
c o l l o c a t i o n m e t h o d . A g a i n , o u r discussion is f a i r l y d e t a i l e d w i t h c o m p l e t e M A T L A B
code. A s i n t h e p r e v i o u s c h a p t e r w e r e s t r i c t ourselves t o t w o - d i m e n s i o n a l e l l i p t i c
p r o b l e m s whose a n a l y t i c s o l u t i o n is r e a d i l y a v a i l a b l e a n d t h e r e f o r e c a n easily be
2
verified. W e w i l l refer t o a p o i n t x i n R as (x,y).
40.1 Symmetric Collocation M e t h o d
For p r o b l e m s i n v o l v i n g t h e L a p l a c i a n we n o w r e q u i r e also t h e d i f f e r e n t i a l o p e r a t o r
2 2 2 2 2 2 2 2
d d d d d d d d
2 2 2 2 2 2 2 2
<9 dx dr) dx d dy drj dy
4 4 4
d d n d \
4
+ 2 - 2 2
+ 4
dx dx y dy J '
w h e r e t h e s i m p l i f i c a t i o n i n t h e last l i n e is j u s t i f i e d since w e are w o r k i n g w i t h even-

o r d e r d e r i v a t i v e s . For e x a m p l e , u s i n g t h e c h a i n r u l e w i t h r = \\x | | we get for
2
v a r i o u s r a d i a l basis f u n c t i o n s i n M :
V 2 2 -( r)
v e
2
= 1 6 4 ( 2 _ 4 ( r )2 + ( r ) 4 ) ~(er)^
e G a u s s i a i l j ( 4 0 .1)
9 9 1 3 (3(er) 4 4
- 2 4 ( e r ) + 8) 2
n r t t N
2 (40 2)
V
^ V
7 T T F F =
(iU^ ' -
2 7 / 2
(l + (er) )
M o r e examples o f R B F s a n d t h e i r d e r i v a t i v e s are c o l l e c t e d i n A p p e n d i x D .
Example 40.1. W e use t h e same P D E a n d b o u n d a r y c o n d i t i o n as i n E x a m -

ple 3 9 . 1 . A M A T L A B p r o g r a m for s y m m e t r i c H e r m i t e - b a s e d c o l l o c a t i o n is g i v e n as
P r o g r a m 4 0 . 1 . N o t e t h a t t h i s p r o g r a m is q u i t e a b i t m o r e c o m p l i c a t e d t h a n the
365
corresponding one for the non-symmetric collocation method (c.f. Program 39.1).
The evaluation matrix EM now consists of two blocks (similar to the collocation ma-
trix for the non-symmetric case, see lines 19-23), whereas the collocation matrix is
assembled from four blocks (c.f. lines 25-33). Note that we now also require one of
the iterated Laplacians of the basic function as listed in (40.1)-(40.3).
P r o g r a m 40.1. HermiteLaplace_2D.m
% HermiteLaplace_2D
% S c r i p t t h a t performs Hermite c o l l o c a t i o n f o r 2D L a p l a c e equation
7, C a l l s on: D i s t a n c e M a t r i x
7o IMQ RBF and i t s L a p l a c i a n and double L a p l a c i a n
1 r b f = <3(e,r) 1 . / s q r t ( l + ( e * r ) . ~ 2 ) ; ep = 3;
2 Lrbf = @(e,r) e~2* ( ( e * r ) . "2-2) . / ( l + ( e * r ) . "2) . " (5/2) ;
3a L2rbf = @(e,r) 3*e"4*(3*(e*r)."4-24*(e*r)."2+8)./...
3b (l+(e*r)."2)."(9/2);
7. Exact s o l u t i o n and i t s L a p l a c i a n f o r t e s t problem
7. Number and type of c o l l o c a t i o n p o i n t s
6 N = 289; g r i d t y p e = 'u';
7 n e v a l = 40;
7o Load ( i n t e r i o r ) c o l l o c a t i o n p o i n t s
8 name = s p r i n t f ( 'Data2D_7od7oS' ,N,gridtype) ; load (name) ;
9 intdata = dsites;
7o A d d i t i o n a l ( e q u a l l y spaced) boundary c o l l o c a t i o n p o i n t s
10 sn = s q r t ( N ) ; b d y l i n = l i n s p a c e ( 0 , 1 , s n ) ' ;
11 bdyO = z e r o s ( s n - 1 , 1 ) ; b d y l = o n e s ( s n - 1 , 1 ) ;
12b f l i p u d ( b d y l i n ( 2 : e n d ) ) bdyl; bdyO f l i p u d ( b d y l i n ( 2 : e n d ) ) ] ;
7. C r e a t e a d d i t i o n a l boundary c e n t e r s OUTSIDE the domain
14 bdyO = -h*ones(sn-2,1); bdyl = (1+h)*ones(sn-2,1);
15a b d y c t r s = [-h -h; b d y l i n bdyO; 1+h -h; b d y l b d y l i n ; ...
15b 1+h 1+h; f l i p u d ( b d y l i n ) bdyl; -h 1+h; bdyO f l i p u d ( b d y l i n ) ] ;
16 i n t c t r s = i n t d a t a ;
7. Create n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
17 g r i d = linspace(0,1neval); [xe.ye] = m e s h g r i d ( g r i d ) ;
18 e p o i n t s = [ x e ( : ) y e ( : ) ] ;
7o Compute e v a l u a t i o n m a t r i x
19 DM_inteval = D i s t a n c e M a t r i x ( e p o i n t s , i n t c t r s ) ;
20 LEM = L r b f ( e p , D M _ i n t e v a l ) ;
21 DM_bdyeval = DistanceMatrix(epoints,bdyctrs);
^ .Li.
40. Symmetric RBF Collocation in MATLAB 367
22 BEM = rbf(ep,DM_bdyeval);
23 EM = [LEM BEM];
25 DM_IIdata = D i s t a n c e M a t r i x d n t d a t a , i n t c t r s ) ;
26 LLCM = L2rbf(ep,DM_IIdata);
27 DM_IBdata = D i s t a n c e M a t r i x d n t d a t a , b d y c t r s ) ;
28 LBCM = Lrbf(ep,DM_IBdata);
29 DM_BIdata = D i s t a n c e M a t r i x ( b d y d a t a , i n t c t r s ) ;
30 BLCM = Lrbf(ep,DM_BIdata);
31 DM_BBdata = D i s t a n c e M a t r i x ( b d y d a t a , b d y c t r s ) ;
32 BBCM = rbf(ep,DM_BBdata);
33 CM = [LLCM LBCM; BLCM BBCM];
% Create right-hand s i d e
34b sin(pi*bdydata(l:sn-1,1)); zeros(3*(sn-1),1)];
% Compute RBF s o l u t i o n
37 rms.err = norm(Pf-exact)/neval;
38 fprintf('RMS e r r o r : %e\n', r m s . e r r )
39. f p r i n t f ('Maximum e r r o r : /e\n' , maxerr)
/ P l o t c o l l o c a t i o n p o i n t s and c e n t e r s
0

42 p l o t ( b d y c t r s ( : , 1 ) , b d y c t r s ( : , 2 ) , ' g x ' ) ; hold o f f
43 fview = [-30,30]; % viewing a n g l e s f o r p l o t
As above we deal with the boundary by allowing the use of different collocation
points and centers along the boundary. This causes the collocation matrix to be
non-symmetric, and therefore the theoretical foundation of Chapter 38 no longer
applies, i.e., it is not clear that in this case the matrix is invertible. In order to
work with a "safe" symmetric (and guaranteed invertible) matrix one should replace
lines 13-15 with
b d y c t r s = bdydata;
Note that, contrary to the non-symmetric Kansa approach, we can do this for both
uniform and non-uniform interior points. In this case it is also possible to simplify
the assembly of the collocation matrix. We can remove lines 29-30 and replace
line 33 by
CM = [LLCM LBCM; LBCM' BBCM];
T h e same set o f e x p e r i m e n t s as for t h e n o n - s y m m e t r i c K a n s a m e t h o d (see

T a b l e s 3 9 . 1 a n d 3 9 . 2 ) are d i s p l a y e d i n T a b l e s 4 0 . 1 a n d 4 0 . 2 for t h e s y m m e t r i c
Hermite-based method.
Table 40.1 Symmetric collocation solution of Example 40.1 with I M Q s , e = 3 and

Halton points.
centers on boundary centers outside

JV
9 1.869505e-001 9.055720e+003 2.438041e-001 3.549895e+004

25 7.698471e-002 8.506782e+004 9.429580e-002 1.162027e+005
81 4.839682e-003 1.338599e+007 5.070833e-003 1.017388e+007
289 4.480250e-005 9.991615e+010 3.448546e-005 7.180249e+010
1089 2.481407e-008 2.820823e+018 1.907000e-008 2.262777e+018
We note that, as for t h e n o n - s y m m e t r i c c o l l o c a t i o n m e t h o d , inverse m u l t i -

q u a d r i c s w i t h i n t e r i o r H a l t o n p o i n t s a n d e x t e r i o r b o u n d a r y centers seems t o per-
f o r m o v e r a l l s l i g h t l y b e t t e r t h a n t h e o t h e r choices (i.e., Gaussians, i n t e r i o r u n i f o r m
p o i n t s , or b o u n d a r y centers o n t h e b o u n d a r y ) .
Table 40.2 Symmetric collocation solution of Example 40.1 with Gaussians and I M Q s ,
e = 3 and uniform points with boundary centers outside the domain.
Gaussian IMQ
N
(interior points) RMS-error cond(>l) RMS-error cond(yl)
3x3 4.088188e-001 1.196486e+005 2.806897e-001 3.105155e+004

5x5 7.704584e-003 1.359899e+005 1.583948e-001 1.216534e+005
9x9 2.272289e-004 2.453107e+010 8.650782e-004 2.016503e+007
17 x 17 5.271776e-008 4.338406e+021 3.962654e-005 6.051588e+011
33 x 33 5.805757e-007 1.438258e+022 1.870210e-007 2.324115e+020
I t is r e m a r k a b l e , h o w e v e r , h o w s m a l l t h e difference i n p e r f o r m a n c e b e t w e e n t h e
s y m m e t r i c a n d n o n - s y m m e t r i c a p p r o a c h is. T h i s c a n be c o n c l u d e d b y c o m p a r i n g
t h e t a b l e s i n E x a m p l e 3 9 . 1 w i t h t h o s e i n E x a m p l e 4 0 . 1 . A l s o , F i g u r e 4 0 . 1 shows
e r r o r p l o t s for t h e t w o m e t h o d s u s i n g t h e same set o f p a r a m e t e r s , i.e., inverse
multiquadrics w i t h e = 3, N = 289 interior H a l t o n points and 64 boundary points
w i t h t h e b o u n d a r y centers p l a c e d o u t s i d e t h e d o m a i n as i n F i g u r e 3 9 . 1 .
T h e e x a m p l e above shows v e r y h i g h convergence r a t e s as p r e d i c t e d b y t h e es-

t i m a t e i n T h e o r e m 3 8 . 1 w h e n u s i n g i n f i n i t e l y s m o o t h inverse m u l t i q u a d r i c s o n a
p r o b l e m t h a t has a s m o o t h s o l u t i o n .
E x a m p l e 4 0 . 2 . A M A T L A B i m p l e m e n t a t i o n o f t h e v a r i a b l e coefficient p r o b l e m o f
E x a m p l e 3 9 . 2 , w h i l e t h e o r e t i c a l l y possible, is v e r y c u m b e r s o m e u s i n g t h e s y m m e t r i c
X10 x10
11.2
10.8
|0.6.fc
10.4
10.2
Fig. 40.1 Error plots for the collocation solution of Example 39.1 (Example 40.1) using I M Q s
with e 3 and N 289 interior Halton points; boundary centers outside domain. Kansa's method
(left) and symmetric method (right).
c o l l o c a t i o n m e t h o d . For e x a m p l e , since t h e d i f f e r e n t i a l o p e r a t o r C is g i v e n b y
t h e basic e x p a n s i o n for t h e R B F s o l u t i o n is
N B N
3= 1 3=NB + 1
with
T h i s , however i m p l i e s t h a t t h e b l o c k Acc$. o f t h e s y m m e t r i c c o l l o c a t i o n m a t r i x has

entries c o m p u t e d w i t h t h e d i f f e r e n t i a l o p e r a t o r
tk ( "' S |( ' ^;j

a( y) + 6(x y)
4
a d A
+
= a(x, y)a(, + (a(x, y)b(, r,) + b(x, y)a(, r))) 9 g 2 2
b(x, y)b(, rf) 7-7 + -K

a(, rj) - a(x, y) 1 3
+
dy4
dx dx
3
'db(x,y) , dbjtj, y)\ d ,
2
, dy d-n J dx dy
'da(x,y)
dx
'db(x,y) > w ^(Â <9 da(x,y)da(f r))
3
: d 2
V) - Kx, y) ) + 2
, dy dr) J dy dx
&
di dx
( da{x,y) db^rj) db(x,y) da(^r))\ d 2
db(x,y) dbfoy) d 2
2
V dx dr) dy d J dxdy dy dr, dy
H e r e we expressed d e r i v a t i v e s w i t h respect t o t h e second v a r i a b l e = ( , rj) o f

t h e basic f u n c t i o n i n t e r m s o f those w i t h respect t o t h e first v a r i a b l e x (x,y)
r e m e m b e r i n g t h a t every d i f f e r e n t i a t i o n i n t r o d u c e s a s i g n change (c.f. t h e discussion
at the end of Chapter 36).
E x a m p l e 40.3. I n s t e a d o f r e p e a t i n g t h e c a l c u l a t i o n s for E x a m p l e 39.3, w e p r e s e n t

a different p r o b l e m w i t h piecewise defined b o u n d a r y c o n d i t i o n s .
2 2
V u(x,y) = 0, (x,y) eft = (-l,l) ,
u(x,y) = 0, (x,y) T x UT 3 UT , 5
u(x, y) = - sin(37ry), (x, y) <E T , 2

5
4
u(x,y) = sin (7ra;), (x,y) G T , 4
where
ri = {(x,y) - l < x < l , y = - l } ,

r 2 = {{x,y) x = l, - l < y < l } ,
r 3 = {(x,y) 0 < x < l , y=l},
r 4 = {{x,y) - l < x < 0 , y=l},
r 5
= {(x,y) x = - l , 0 < y < 1}.
For t h i s p r o b l e m we do n o t have a n exact s o l u t i o n a v a i l a b l e . H o w e v e r , t h i s p r o b l e m

is t a k e n f r o m [ T r e f e t h e n (2000)] a n d w e use t h e p s e u d o s p e c t r a l s o l u t i o n f r o m t h e r e
for c o m p a r i s o n . W e w i l l r e v i s i t t h i s p r o b l e m l a t e r w h e n we discuss R B F - P S m e t h o d s
i n Chapter 42.
P r o g r a m 40.2. HermiteLaplaceMixedBCTref_2D .m
% HermiteLaplaceMixedBCTref_2D
7, Note: Prog 36 i n T r e f e t h e n (2000), e x a c t s o l u t i o n not provided
/ C a l l s on: D i s t a n c e M a t r i x
0
% IMQ RBF and i t s L a p l a c i a n

1 r b f = @(e,r) 1 . / s q r t ( l + ( e * r ) . " 2 ) ; ep = 3;
2 L r b f = @(e,r) e ~ 2 * ( ( e * r ) . " 2 - 2 ) . / ( l + ( e * r ) . ~ 2 ) . ~ ( 5 / 2 ) ;
3a L2rbf = @(e,r) 3 * e ~ 4 * ( 3 * ( e * r ) . " 4 - 2 4 * ( e * r ) . " 2 + 8 ) . / . . .
3b (l+(e*r)."2)."(9/2);
% L a p l a c i a n f o r t e s t problem
4 Lu = @(x,y) z e r o s ( s i z e ( x ) ) ;
5 N = 289; g r i d t y p e = 'u';
6 n e v a l = 41;
7, Load ( i n t e r i o r ) c o l l o c a t i o n p o i n t s
7 name = s p r i n t f ( 'Data2D_7od7 s' ,N, g r i d t y p e ) ; l o a d (name) ;
0
8 intdata = 2*dsites-l;
9 sn = sqrt(N); bdylin = l i n s p a c e ( - 1 , 1 , s n ) ' ;
10 bdyl = ones(sn-1,1);
11a bdydata = [ b d y l i n d : end-1) -bdyl; bdyl b d y l i n d : end-1) ;
lib f l i p u d ( b d y l i n ( 2 : e n d ) ) bdyl; -bdyl f l i p u d ( b d y l i n ( 2 : e n d ) ) ] ;
/ Create a d d i t i o n a l boundary c e n t e r s OUTSIDE t h e domain
0
12 h = 2 / ( s n - l ) ; b d y l i n = (-l+h:h:1-h)';
13 bdyO = repmat(-1-h,sn-2,1); bdyl = repmat(1+h,sn-2,1);
14a bdyctrs = [-1-h -1-h; b d y l i n bdyO; 1+h -1-h; bdyl b d y l i n ; ...
14b 1+h 1+h; f l i p u d ( b d y l i n ) bdyl; -1-h 1+h; bdyO f l i p u d ( b d y l i n ) ] ;
15 i n t c t r s = intdata;
16 g r i d = l i n s p a c e ( - 1 , 1 , n e v a l ) ; [xe,ye] = m e s h g r i d ( g r i d ) ;
17 epoints = [ x e ( : ) y e ( : ) ] ;
% Compute e v a l u a t i o n matrix
18 DM_inteval = D i s t a n c e M a t r i x ( e p o i n t s , i n t c t r s ) ;
19 LEM = Lrbf(ep,DM_inteval);
20 DM_bdyeval = D i s t a n c e M a t r i x ( e p o i n t s , b d y c t r s ) ;
22 EM = [LEM BEM] ;
7, Compute b l o c k s f o r c o l l o c a t i o n matrix
23 DM_IIdata = D i s t a n c e M a t r i x d n t d a t a , i n t c t r s ) ;
25 DM_IBdata = D i s t a n c e M a t r i x d n t d a t a , b d y c t r s ) ;
27 DM_BIdata = D i s t a n c e M a t r i x ( b d y d a t a , i n t c t r s ) ;
29 DM_BBdata = D i s t a n c e M a t r i x ( b d y d a t a , b d y c t r s ) ;
7o Create right-hand s i d e
32a r h s = [ L u ( i n t d a t a ( : , 1 ) , i n t d a t a ( : , 2 ) ) ; z e r o s ( s n - 1 , 1 ) ; ...
32b 0.2*sin(3*pi*bdydata(sn:2*sn-2,2)); zeros((sn-l)/2,1);...
32c sin(pi*bdydata((5*sn-3)/2:3*sn-3,l)).~4; zeros(sn-l,1)];
7 Compute RBF s o l u t i o n
34 surf(xe,ye,reshape(Pf,neval,neval));
35 view(-20,45), a x i s ( [ - l 1 - 1 1 -.2 1 ] ) ;
36 t e x t ( 0 , . 8 , . 5 , s p r i n t f ( > u ( 0 , 0 ) = 7.12 . lOf ' ,Pf (841) ) )
T h e d e f i n i t i o n o f t h e b o u n d a r y c o n d i t i o n s i n t h e M A T L A B code for P r o g r a m 40.2

is s i m i l a r t o t h a t for P r o g r a m 39.3. H o w e v e r , n o w w e are w o r k i n g o n t h e s q u a r e
2 2
[1, l ] i n s t e a d o f [0, l ] , a n d t h e r e f o r e s l i g h t a d j u s t m e n t s are r e q u i r e d . F o r e x a m -
ple, t h e c o l l o c a t i o n p o i n t s w e l o a d f r o m file are n o w t r a n s f o r m e d o n l i n e 8. Also,
t h e b o u n d a r y centers have t o be offset f r o m a d i f f e r e n t b o u n d a r y (see lines 1 2 - 1 4 ) .
Fig. 40.2 Plots for solution of Example 40.3 using pseudospectral method with 361 points from
[Trefethen (2000)] (left), and with symmetric collocation using I M Q s with e = 3 and N = 289
uniform interior points; 64 additional boundary centers outside domain.
W e n o t e t h a t t h e q u a l i t y o f t h e t w o s o l u t i o n s d i s p l a y e d i n F i g u r e 40.2 is q u i t e
s i m i l a r . T h e t o t a l n u m b e r o f p o i n t s u s e d for t h e P S s o l u t i o n is 3 6 1 , w h i l e 353 p o i n t s
(289 i n t e r i o r p l u s 64 b o u n d a r y ) are used for t h e R B F s o l u t i o n .
40.2 Summarizing R e m a r k s on the S y m m e t r i c and Non-

Symmetric Collocation Methods
A l l i n a l l , t h e n o n - s y m m e t r i c ( K a n s a ) m e t h o d seems t o p e r f o r m j u s t a l i t t l e b i t
b e t t e r t h a n t h e s y m m e t r i c ( H e r m i t e ) m e t h o d ( c o m p a r e T a b l e s 3 9 . 1 a n d 39.2 w i t h
T a b l e s 4 0 . 1 a n d 4 0 . 2 ) . F o r t h e s a m e v a l u e o f t h e shape p a r a m e t e r e t h e e r r o r s as w e l l
as t h e c o n d i t i o n n u m b e r s are s l i g h t l y s m a l l e r . T h i s does n o t agree w i t h t h e f i n d i n g s
i n [ L a r s s o n a n d F o r n b e r g (2003)] w h e r e t h e a u t h o r s c o n c l u d e d t h a t t h e s y m m e t r i c
m e t h o d is m o r e a c c u r a t e (see also o u r d i s c u s s i o n a t t h e e n d o f C h a p t e r 3 8 ) .
An a d v a n t a g e o f t h e H e r m i t e a p p r o a c h over K a n s a ' s m e t h o d is t h a t t h e c o l -
l o c a t i o n m a t r i c e s r e s u l t i n g f r o m t h e H e r m i t e a p p r o a c h are s y m m e t r i c i f a l l o f t h e
centers c o i n c i d e w i t h t h e c o l l o c a t i o n p o i n t s . T h e r e f o r e t h e a m o u n t o f c o m p u t a t i o n
c a n be r e d u c e d c o n s i d e r a b l y b y u s i n g a solver for s y m m e t r i c s y s t e m s . Since K a n s a ' s
m e t h o d r e q u i r e s fewer d e r i v a t i v e s o f t h e basic f u n c t i o n i t has t h e a d d e d advan-
tages o f b e i n g s i m p l e r t o i m p l e m e n t a n d a p p l i c a b l e t o p r o b l e m s w i t h less s m o o t h
solutions. M o r e o v e r , as w e saw i n E x a m p l e s 39.2 a n d 40.2, t h e non-symmetric
m e t h o d is m u c h s i m p l e r for p r o b l e m s w i t h n o n - c o n s t a n t coefficients. Further-
r
m o r e , i t is n o t clear h o w t o d e a l w i t h n o n - l i n e a r p r o b l e m s u s i n g t h e s y m m e t r i c
m e t h o d . F o r a t r e a t m e n t o f n o n - l i n e a r P D E s based o n t h e n o n - s y m m e t r i c c o l l o c a -
t i o n m e t h o d w i t h i n a n o p e r a t o r N e w t o n f r a m e w o r k see [ B e r n a l a n d K i n d e l a n ( 2 0 0 6 ) ;
Fasshauer (2001a)].
A n o t h e r c o n t r a p o s i t i o n o f t h e t w o m e t h o d s w i l l be p r e s e n t e d i n t h e c o n t e x t o f
p s e u d o s p e c t r a l m e t h o d s i n C h a p t e r 42.
B o t h o f t h e m e t h o d s d e s c r i b e d i n t h i s s e c t i o n have b e e n i m p l e m e n t e d for m a n y
different a p p l i c a t i o n s . Comparisons of the two methods were r e p o r t e d i n , e.g.,
[Fasshauer (1997); L a r s s o n a n d F o r n b e r g ( 2 0 0 3 ) ; P o w e r a n d B a r r a c o ( 2 0 0 2 ) ] .
I
'30
Chapter 41
Collocation with C S R B F s in MATLAB
I n this t h i r d chapter describing the M A T L A B i m p l e m e n t a t i o n of R B F collocation

m e t h o d s we l o o k at h o w c o m p a c t l y s u p p o r t e d f u n c t i o n s c a n be used i n b o t h a d i r e c t
approach and w i t h i n a multilevel framework. A s i n the previous t w o chapters we
2
present o n l y t w o - d i m e n s i o n a l e l l i p t i c p r o b l e m s a n d w i l l refer t o a p o i n t x i n R as
41.1 Collocation with Compactly Supported RBFs
W h i l e K a n s a i n i t i a l l y p r o p o s e d t h e n o n - s y m m e t r i c c o l l o c a t i o n m e t h o d for m u l t i -
q u a d r i c s , t h e general m e t h o d applies t o a n y k i n d o f R B F i n c l u d i n g those w i t h
c o m p a c t s u p p o r t . T h e same goes for t h e s y m m e t r i c m e t h o d . W e n o w present M A T -
6
L A B code for t h e s y m m e t r i c c o l l o c a t i o n m e t h o d based o n W e n d l a n d ' s C function
3 2
>3,3(0 (1 r ) ^ . ( 3 2 r + 2 5 r + 8r + 1). I t s L a p l a c i a n a n d b i h a r m o n i c d e r i v a t i v e s
are g i v e n b y
6 3 2
V V , 3 ( r ) = 4 4 ( 1 - r) (88r
3 + + 3r - 6 r - 1),
4 3 2
V V , ( r ) = 1 0 5 6 ( 1 - r ) _ ( 2 9 7 r - 2 1 2 r + 16r + 4 ) .
3 3
N o t e , however, t h a t i n o r d e r for us t o be able t o t a k e a d v a n t a g e o f t h e s u b r o u t i n e

D i s t a n c e M a t r i x C S R B F . m w e p r o v i d e d earlier i n P r o g r a m 12.1 we need t o represent
t h e basic f u n c t i o n a n d i t s d e r i v a t i v e s i n t h e s h i f t e d f o r m
8 2 3
3 ) 3 ( r ) = r ( 6 6 - 154r + 1 2 1 r - 32r ),
2 6 2 3
V ^ 3 ) 3 ( r ) = 4 4 r ( 8 4 - 264r + 2 6 7 r - 88r ),
4 4 2 3
V < , (r) - 3 3 1056r (105 - 483r + 679r - 297r ),
as i m p l e m e n t e d o n lines 1-3 o f P r o g r a m 4 1 . 1 . W h i l e w e t e c h n i c a l l y do n o t need
t o i n c l u d e a scale factor e i n t h e M A T L A B code for t h e basic f u n c t i o n (since t h e
s u p p o r t size is a l r e a d y used t o d e t e r m i n e t h e m a t r i x entries i n P r o g r a m 12.1), t h e
d e r i v a t i v e s o f t h e basic f u n c t i o n s t i l l r e q u i r e t h e scale factor w h i c h a p p e a r s as a
consequence o f t h e c h a i n r u l e (see lines 2 a n d 3 ) .
A n o t h e r p o i n t t o reconsider i n t h e c o m p a c t s u p p o r t s e t t i n g is t h e p l a c e m e n t o f
t h e b o u n d a r y centers. W h i l e we saw for g l o b a l l y s u p p o r t e d basic f u n c t i o n s t h a t
375
i t was a c t u a l l y beneficial t o place some centers o u t s i d e t h e d o m a i n , t h i s no l o n g e r

m a k e s m u c h sense i f we decide t o use c o m p a c t l y s u p p o r t e d functions. Clearly,
a n y basic f u n c t i o n whose s u p p o r t r a d i u s is s m a l l e r t h a n t h e d i s t a n c e o f i t s center
from the boundary of the domain w i l l not contribute to the solution of the problem.
T h e r e f o r e , w e n o w use i n t e r i o r H a l t o n p o i n t s a u g m e n t e d b y e q u a l l y spaced b o u n d a r y
p o i n t s for b o t h t h e c o l l o c a t i o n p o i n t s a n d t h e centers. T h i s change is reflected on
lines 14 a n d 15 o f P r o g r a m 41.1. O t h e r w i s e , P r o g r a m 41.1 is essentially i d e n t i c a l t o
P r o g r a m 40.1. H o w e v e r , for t h e convenience o f t h e reader we d e c i d e d t o p r i n t t h e
e n t i r e p r o g r a m for t h e c o m p a c t l y s u p p o r t e d case, also.
P r o g r a m 41.1. HermiteLaplace_2D_CSRBF .m
7. HermiteLaplace_2D_CSRBF
7, w i t h s p a r s e m a t r i c e s
7. C a l l s on: DistanceMatrixCSRBF
7o Wendland C6 RBF, i t s L a p l a c i a n and double L a p l a c i a n
1 r b f = @(e,r) r . ~ 8 . * ( 6 6 * s p o n e s ( r ) - 1 5 4 * r + 1 2 1 * r . ~ 2 - 3 2 * r . ~ 3 ) ;
2 L r b f = @(e,r) 44*e~2*r."6.*(84*spones(r)-264*r+267*r.~2-88*r."3);
3a L2rbf = @(e,r) 1056*e~4*r."4.*...
3b (105*spones(r)-483*r+679*r.~2-297*r.~3);
4 ep = 0.25;
7o E x a c t s o l u t i o n and i t s L a p l a c i a n f o r t e s t problem
7o Number and type of c o l l o c a t i o n p o i n t s
7 N = 289; g r i d t y p e = 'h> ;
8 n e v a l = 40;
7o Load ( i n t e r i o r ) c o l l o c a t i o n p o i n t s
9 name = s p r i n t f ( 'Data2D_7od7oS' ,N .gridtype) ; l o a d (name) ;
10 i n t d a t a = d s i t e s ;
7. A d d i t i o n a l ( e q u a l l y spaced) boundary c o l l o c a t i o n p o i n t s
11 s n = s q r t ( N ) ; b d y l i n = l i n s p a c e ( 0 , 1 , s n ) ' ;
13b f l i p u d ( b d y l i n ( 2 : e n d ) ) b d y l ; bdyO f l i p u d ( b d y l i n ( 2 : e n d ) ) ] ;
7o L e t c e n t e r s c o i n c i d e w i t h ALL d a t a s i t e s
14 b d y c t r s = bdydata;
7o C r e a t e n e v a l - b y - n e v a l e q u a l l y spaced e v a l u a t i o n l o c a t i o n s
17 e p o i n t s = [ x e ( : ) y e ( : ) ] ;
41- Collocation with CSRBFs in MATLAB 377
'/, Compute e v a l u a t i o n m a t r i x
18 DM_inteval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , i n t d a t a , e p ) ;
19 DM_bdyeval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , b d y c t r s , e p ) ;
20 LEM = L r b f ( e p , D M _ i n t e v a l ) ;
22 EM = [LEM BEM];
% Compute blocks f o r c o l l o c a t i o n m a t r i x
24 DM_IIdata = D i s t a n c e M a t r i x C S R B F ( i n t d a t a , i n t d a t a , e p ) ;
25 DM_IBdata = D i s t a n c e M a t r i x C S R B F ( i n t d a t a , b d y c t r s , e p ) ;
26 DM_BIdata = DistanceMatrixCSRBF(bdydata,intdata,ep);
27 DM_BBdata = DistanceMatrixCSRBF(bdydata,bdyctrs,ep);
7. Create right-hand s i d e
% Compute RBF s o l u t i o n
37 fprintf('RMS e r r o r : %e\n', r m s _ e r r )
% P l o t c o l l o c a t i o n p o i n t s and c e n t e r s
41 p l o t ( b d y c t r s ( : , 1 ) , b d y c t r s ( : , 2 ) , ' g x ' ) ; hold o f f
42 fview = [-30,30]; '/. v i e w i n g angles f o r p l o t
If we want to replace the symmetric collocation method in Program 41.1 by the

non-symmetric one, then lines 18-32 need to be replaced by
DM_eval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , c t r s , e p ) ;
EM = rbf(ep,DM_eval);
exact = u ( e p o i n t s ( : , 1 ) , e p o i n t s ( : , 2 ) ) ;
% Compute b l o c k s f o r c o l l o c a t i o n matrix
DM_intdata = D i s t a n c e M a t r i x C S R B F ( i n t d a t a , c t r s , e p ) ;
DM_bdydata = D i s t a n c e M a t r i x C S R B F ( b d y d a t a , c t r s , e p ) ;
LCM = Lrbf(ep,DM_intdata);
BCM = rbf(ep,DM_bdydata);
CM = [LCM; BCM];
Example 4 1 . 1 . W e use t h e t e s t p r o b l e m o f E x a m p l e s 3 9 . 1 a n d 4 0 . 1 . However,

t h i s t i m e w e c o m p a r e a s t a t i o n a r y a p p r o x i m a t i o n scheme t o a n o n - s t a t i o n a r y one
for b o t h t h e n o n - s y m m e t r i c and the symmetric collocation method. I n the station-
a r y s e t t i n g we t a k e a n i n i t i a l p a r a m e t e r v a l u e o f e = 0.25 (i.e., v e r y w i d e basis
functions t h a t cover t h e e n t i r e d o m a i n ) , a n d t h e n d o u b l e i t s v a l u e for e v e r y suc-
cessive e x p e r i m e n t . T h i s keeps t h e "bandwidth" of the collocation m a t r i x fixed
a n d r e s u l t s i n a n efficient a p p r o x i m a t i o n m e t h o d . H o w e v e r , as for s c a t t e r e d d a t a
i n t e r p o l a t i o n (c.f. T a b l e 12.1) w e s h o u l d n o t e x p e c t c o n v e r g e n c e for h > 0 i n t h i s
s t a t i o n a r y s e t t i n g . T h i s c a n be seen c l e a r l y i n t h e left p a r t o f T a b l e s 4 1 . 1 a n d 4 1 . 2 .
I n fact, we c a n o b s e r v e t h a t for t h e c o l l o c a t i o n a p p r o a c h t h i n g s are e v e n w o r s e t h a n
for i n t e r p o l a t i o n . N o t o n l y is t h e r e n o c o n v e r g e n c e for h > 0, t h e e r r o r s a c t u a l l y
increase. T h i s is e s p e c i a l l y p r o n o u n c e d f o r t h e s y m m e t r i c method.
Table 41.1 Non-symmetric collocation solution of Example 40.1 with C S R B F s , sta-

tionary e (initial value e = 0.25) and non-stationary e 0.25. TV interior Halton
points, 4(\/]V 1) equally spaced boundary centers coinciding with boundary colloca-
tion points.
3x3 6.077025e-003 1.495127e+005 6.077025e-003 1.495127e+005

5x5 2.352498e-003 6.274058e+005 3.810928e-004 2.720833e+007
9x9 1.947271e-003 6.192333e+006 4.430301e-005 2.851716e+010
17 x 17 1.326745e-003 6.113189e+007 2.200286e-006 2.293235e+013
33 x 33 5.703309e-003 1.824487e+008 9.986944e-008 8.537419e+015
Table 41.2 Symmetric collocation solution of Example 40.1 with C S R B F s , stationary

e (initial value e = 0.25) and non-stationary e = 0.25. N interior Halton points,
4(\/]V 1) equally spaced boundary centers coinciding with boundary collocation
points.
3x3 5.866837e-003 4.448249e+004 5.866837e-003 4.448249e+004

5x5 3.190108e-003 2.454493e+006 4.757992e-004 3.582872e-(-007
9x9 8.381144e-003 8.693440e+007 3.825086e-005 2.360746e+010
17 x 17 1.115179e-001 2.162078e+009 2.099321e-006 1.392838e+013
33 x 33 3.696962e-001 3.164425e+010 8.680882e-008 6.127122e+015
41. Collocation with CSRBFs in MATLAB 379
I n t h e n o n - s t a t i o n a r y s e t t i n g we observe convergence whose r a t e is r e m a r k a b l y

s i m i l a r for b o t h approaches (c.f. t h e r i g h t p a r t o f T a b l e s 4 1 . 1 a n d 4 1 . 2 ) . H o w e v e r ,
t h e c o l l o c a t i o n m a t r i c e s are n o w c o m p l e t e l y dense, a n d t h e r e f o r e t h i s a p p r o a c h
as i n t h e case o f i n t e r p o l a t i o n defies t h e use o f c o m p a c t l y s u p p o r t e d f u n c t i o n s .
I t is also i n t e r e s t i n g t o n o t e t h a t we c a n see t h a t t h e a c c u r a c y o b t a i n e d w i t h
6
the C W e n d l a n d f u n c t i o n s ( i n " g l o b a l m o d e " ) is s i m i l a r t o t h a t o f t h e g l o b a l l y
s u p p o r t e d C Gaussians a n d inverse m u l t i q u a d r i c s used i n E x a m p l e 3 9 . 1 a n
i n d i c a t i o n t h a t t h e s o l u t i o n t o t h e P D E does n o t l i e i n t h e n a t i v e space o f t h e
Gaussians or inverse m u l t i q u a d r i c s .
On t h e o t h e r h a n d , i f t h e basic f u n c t i o n s are chosen t o o l o c a l ( t o keep the
m e t h o d efficient), t h e n t h e b o u n d a r y i n f o r m a t i o n c a n n o t p e n e t r a t e t o t h e inside o f
t h e p r o b l e m . T h i s is essentially w h a t h a p p e n s i n t h e s t a t i o n a r y s e t t i n g . F i g u r e 4 1 . 1
shows fits w i t h e = 2 ( s m a l l s u p p o r t ) a n d w i t h e = 0.25 ( l a r g e s u p p o r t ) for N = 289
i n t e r i o r H a l t o n p o i n t s plus 64 e q u a l l y spaced b o u n d a r y p o i n t s c o r r e s p o n d i n g t o t h e
f o u r t h r o w i n Tables 4 1 . 1 a n d 41.2. A s i n d i c a t e d above, t h e centers are chosen
t o c o i n c i d e w i t h t h e c o l l o c a t i o n p o i n t s . T h e c o l l o c a t i o n m a t r i x i n t h e e = 2 case
is sparse a n d has o n l y 1 1 % n o n z e r o entries w h e n u s i n g t h e s y m m e t r i c m e t h o d
a n d 4 3 % for t h e n o n - s y m m e t r i c m e t h o d . T h e r a t h e r s i g n i f i c a n t difference i n t h e
s p a r s i t y p a t t e r n s o f t h e s y m m e t r i c a n d n o n - s y m m e t r i c m e t h o d s is due t o t h e fact
t h a t t h e entries for t h e s y m m e t r i c m a t r i x are g i v e n b y h i g h e r - o r d e r d e r i v a t i v e s o f
t h e basic f u n c t i o n t h a n t h o s e i n t h e n o n - s y m m e t r i c case. W h i l e t h e d e r i v a t i v e s o f
ip t h e o r e t i c a l l y r e t a i n t h e same s u p p o r t as ip, n u m e r i c a l l y t h e size o f t h e s u p p o r t
appears t o s h r i n k w i t h increasing differentiation. T h u s the resulting a p p r o x i m a t i o n
i n t h e s y m m e t r i c s t a t i o n a r y case is m u c h p o o r e r because t h e basis f u n c t i o n s a n d
t h e i r d e r i v a t i v e s are t o o l o c a l a n d t h e b o u n d a r y i n f o r m a t i o n i n p r e v e n t e d from
t r a v e l i n g across t h e d o m a i n . S i m i l a r o b s e r v a t i o n s w e r e r e p o r t e d i n [ J u m a r h o n et
al. ( 2 0 0 0 ) ; Fasshauer ( 1 9 9 9 d ) ] .
I n [Fasshauer (1999d)] use o f a d i a g o n a l ( J a c o b i ) p r e c o n d i t i o n e r was p r o p o s e d

t o speed u p t h e convergence o f t h e c o n j u g a t e g r a d i e n t m e t h o d used t h e r e t o solve
t h e l i n e a r s y s t e m . H o w e v e r , t h e accuracy o f t h e m e t h o d does n o t benefit f r o m t h i s
measure a n d therefore we d o n o t p u r s u e t h e i d e a a n y f u r t h e r .
T h e e x p e r i m e n t s above, as w e l l as those r e p o r t e d i n [ J u m a r h o n et al. (2000)]
using Wendland's C 4
c o m p a c t l y s u p p o r t e d R B F ^3,2, i n d i c a t e t h a t t h e e r r o r b o u n d s
6
o f T h e o r e m 3 8 . 1 m a y be t o o p e s s i m i s t i c . F o r e l l i p t i c p r o b l e m s a n d C basis f u n c t i o n s
t h e t h e o r e m p r e d i c t s a n e r r o r o n t h e o r d e r o f 0(h) while the numerical experiments
s
i n T a b l e 41.2 suggest a n o r d e r o f a b o u t 0(h ) for t h e n o n - s t a t i o n a r y s e t t i n g .
E v e n m o r e t h a n i n t h e i n t e r p o l a t i o n s e t t i n g , for t h e n u m e r i c a l s o l u t i o n o f P D E s
w i t h c o m p a c t l y s u p p o r t e d R B F s we need t o use a m u l t i l e v e l a p p r o a c h t o have t h e
p o t e n t i a l t o c o m b i n e efficiency w i t h accuracy. A coarse s o l u t i o n w i t h w i d e f u n c t i o n s
w i l l have t o p r o v i d e a n i n i t i a l fit t h a t c a p t u r e s t h e m a i n features o f t h e s o l u t i o n ,
a n d t h e n m o r e refined r e s i d u a l u p d a t e s can i m p r o v e t h i s i n i t i a l s o l u t i o n l o c a l l y .
x 10~ 3
x 10'
y o o x y o o
Fig. 41.1 Plots for solution of Example 40.1 with non-symmetric (top) and symmetric (bottom)
collocation using C S R B F s with e = 2 (left) and e 0.25 (right) and N = 289 interior Halton
points; boundary centers coincide with equally spaced boundary collocation points.
41.2 Multilevel R B F Collocation
W e e n d t h e discussion o f t h e c o l l o c a t i o n a p p r o a c h b y l o o k i n g a t a m u l t i l e v e l i m p l e -
mentation of R B F collocation w i t h compactly supported functions.
T h e m o s t s i g n i f i c a n t difference b e t w e e n t h e use o f c o m p a c t l y s u p p o r t e d RBFs
for s c a t t e r e d d a t a i n t e r p o l a t i o n a n d for t h e n u m e r i c a l s o l u t i o n o f P D E s b y c o l l o c a -
t i o n appears w h e n we t u r n t o t h e m u l t i l e v e l a p p r o a c h . R e c a l l t h a t t h e use o f t h e
m u l t i l e v e l m e t h o d is m o t i v a t e d b y o u r desire t o o b t a i n a c o n v e r g e n t scheme w h i l e a t
t h e same t i m e k e e p i n g t h e b a n d w i d t h f i x e d , a n d t h u s t h e c o m p u t a t i o n a l c o m p l e x i t y
at O(N).
H e r e is a n a d a p t a t i o n o f t h e s t a t i o n a r y m u l t i l e v e l a l g o r i t h m o f C h a p t e r 32 t o
t h e case o f a c o l l o c a t i o n s o l u t i o n o f t h e l i n e a r p r o b l e m Lu = /:
Algorithm 41.1. Stationary Multilevel Collocation
(1) L e t u 0 = 0
(2) For k = 1,2, ...,K do
(a) F i n d Uk G Sx k s u c h t h a t Cu k = / uk-i o n g r i d X\-

(b) U p d a t e u k < u -i
k + u k
Here Sx k is t h e space o f f u n c t i o n s used for e x p a n s i o n (38.5) for t h e non-

s y m m e t r i c m e t h o d o r (38.7) for t h e s y m m e t r i c m e t h o d o n g r i d X . k Note that
t h e o p e r a t o r C encodes b o t h t h e d i f f e r e n t i a l e q u a t i o n a n d t h e b o u n d a r y c o n d i t i o n .
W h e r e a s we n o t e d i n C h a p t e r 32 t h a t t h e r e is s t r o n g n u m e r i c a l ( a n d l i m i t e d t h e -
o r e t i c a l ) evidence t h a t t h e s t a t i o n a r y m u l t i l e v e l i n t e r p o l a t i o n a l g o r i t h m converges
at least l i n e a r l y , t h e f o l l o w i n g e x a m p l e shows t h a t we cannot in general expect the
stationary multilevel collocation algorithm to converge at all.
E x a m p l e 41.2. Once m o r e we t a k e t h e same test p r o b l e m as i n E x a m p l e s 3 9 . 1 ,

4 0 . 1 , a n d 4 1 . 1 . A s for t h e n u m e r i c a l e x p e r i m e n t s i n t h e p r e v i o u s e x a m p l e , we l e t
t h e b o u n d a r y centers coincide w i t h t h e b o u n d a r y c o l l o c a t i o n p o i n t s (see l i n e 25 o f
P r o g r a m 41.2 b e l o w ) .
A n i m p o r t a n t difference b e t w e e n t h e m u l t i l e v e l i n t e r p o l a t i o n code o f C h a p t e r 32
a n d t h e code presented here for t h e c o l l o c a t i o n s o l u t i o n o f P D E s lies i n t h e c o m -
p u t a t i o n o f t h e residuals. N o t e t h a t i n t h e i n t e r p o l a t i o n s e t t i n g t h e r e s i d u a l is o f
t h e f o r m / Vf, w h i l e for P D E s we have / Cu -\. k T h u s , t h e evaluation matrix
i n t h e i n t e r p o l a t i o n s e t t i n g is f o r m e d d i r e c t l y f r o m t h e basis f u n c t i o n s , w h i l e i n t h e
c o l l o c a t i o n s e t t i n g ( i n b o t h t h e s y m m e t r i c a n d n o n - s y m m e t r i c case) t h e e v a l u a t i o n
m a t r i x for t h e residuals is f o r m e d u s i n g t h e derivatives o f t h e basis f u n c t i o n s . T h e s e
differences can be seen b y c o m p a r i n g lines 2 3 - 3 3 o f P r o g r a m 32.1 w i t h lines 3 9 - 5 6
o f P r o g r a m 41.2 b e l o w .
P r o g r a m 4 1 . 2 . ML_HermiteLaplaceCSRBF2D.m
7. M L _ H e r m i t e L a p l a c e C S R B F 2 D
% Script that performs s y m m e t r i c m u l t i l e v e l RBF c o l l o c a t i o n
7, u s i n g s p a r s e matrices
7. C a l l s o n : D i s t a n c e M a t r i x C S R B F
7o W e n d l a n d C6 RBF, i t s L a p l a c i a n a n d d o u b l e Laplacian
1 r b f = @(e,r) r.~8.*(66*spones(r)-154*r+121*r.~2-32*r."3);
2 Lrbf = @(e,r) 4 4 * e ~ 2 * r . ""6 . * ( 8 4 * s p o n e s ( r ) - 2 6 4 * r + 2 6 7 * r . " 2 - 8 8 * r . ~ 3 ) ;
3a L 2 r b f = @(e,r) 1056*e~4*r."4.*...
3b (105*spones(r)-483*r+679*r.~2-297*r.~3);
7o E x a c t s o l u t i o n a n d i t s Laplacian for test problem
6 K = 6; n e v a l = 40; g r i d t y p e = 'h';
7 ep = 0 . 5 * 2 . " [ 0 : K - 1 ] ;
7o C r e a t e n e v a l - b y - n e v a l e q u a l l y s p a c e d e v a l u a t i o n locations
8 grid = linspace(0,1,neval); [xe,ye] = meshgrid(grid);
,Lx
% Compute exact s o l u t i o n
11 Rf_old = zeros(17,l);
12 f o r k=l:K
13 Nl = ( 2 ' k + l ) ~ 2 ; N2 = ( 2 " ( k + l ) + l ) ' 2 ;
14 namel = s p r i n t f ( 'Data2D_/d/s' ,N1,gridtype) ;
0
0
15 name2 = s p r i n t f ( Data2D_ /od / s' ,N2, g r i d t y p e ) ;
, 0
0
16 load(name2)
/ A d d i t i o n a l boundary p o i n t s f o r r e s i d u a l e v a l u a t i o n
0
17 sn = sqrt(N2); bdylin = l i n s p a c e ( 0 , 1 , s n ) ' ;

19a bdyres = [ b d y l i n d : end-1) bdyO; bdyl b d y l i n d : end-1) ; ...
19b f l i p u d ( b d y l i n ( 2 : e n d ) ) b d y l ; bdyO f l i p u d ( b d y l i n ( 2 : e n d ) ) ]
20 intres = dsites;
21 load(name1); i n t d a t a = d s i t e s ;
/ A d d i t i o n a l boundary p o i n t s
22 s n = s q r t ( N l ) ; b d y l i n = l i n s p a c e ( 0 , 1 , s n ) ';
24b f l i p u d ( b d y l i n ( 2 : e n d ) ) bdyl; bdyO f l i p u d ( b d y l i n ( 2 : e n d ) ) ]
25 b d y c t r s { k } = bdydata;
26 intctrs{k> = intdata;
/ Compute new r i g h t - h a n d s i d e (= r e s i d u a l )
27a Tf = [ L u ( i n t d a t a ( : , 1 ) , i n t d a t a ( : , 2 ) ) ; ...
28 r h s = Tf - R f _ o l d ;
29 DM_IIdata = D i s t a n c e M a t r i x C S R B F d n t d a t a , i n t c t r s { k > , ep(k) ) ;
30 DM_IBdata = D i s t a n c e M a t r i x C S R B F ( i n t d a t a , b d y c t r s { k } , e p ( k ) ) ;
31 DM.BIdata = D i s t a n c e M a t r i x C S R B F ( b d y d a t a , i n t c t r s { k } , e p ( k ) ) ;
32 DM_BBdata = DistanceMatrixCSRBF(bdydata,bdyctrs{k},ep(k));
33 LLCM = L 2 r b f ( e p ( k ) , D M _ I I d a t a ) ;
34 LBCM = Lrbf(ep(k),DM_IBdata);
35 BLCM = Lrbf(ep(k),DM_BIdata);
36 BBCM = rbf(ep(k),DM_BBdata);
% Compute c o e f f i c i e n t s f o r RBF s o l u t i o n of d e t a i l l e v e l
38 c o e f t k } = CM\rhs;
39 i f (k < K)
/ based on t h e d i s t a n c e s between t h e next f i n e r
0
/ p o i n t s ( r e s p o i n t s ) and c e n t e r s
0
40 forj=l:k
41- Collocation with CSRBFs in MATLAB 383
41 DM_IIres = DistanceMatrixCSRBF(intres, intctrs-Cj}, e p ( j ) )

42 DM.IBres = DistanceMatrixCSRBF(intres,bdyctrs{j},ep(j))
43 DM_BIres = DistanceMatrixCSRBF(bdyres,intctrs{j},ep(j))
44 DM.BBres = DistanceMatrixCSRBF(bdyres,bdyctrs{j},ep(j))
45 LLRM = L 2 r b f ( e p ( j ) , D M _ I I r e s ) ;
46 LBRM = L r b f ( e p ( j ) , D M _ I B r e s ) ;
47 BLRM = L r b f ( e p ( j ) , D M _ B I r e s ) ;
48 BBRM = r b f ( e p ( j ) , D M _ B B r e s ) ;
49 RM-Cj} = [LLRM LBRM; BLRM BBRM];
50 end
% E v a l u a t e RBF approximation (sum of a l l p r e v i o u s f i t s ,
/, but e v a l u a t e d on c u r r e n t g r i d )
51 Rf = zeros(N2+4*sqrt(N2)-4,1);
52 for j=l:k
53 Rf = Rf + R M { j } * c o e f { j } ;
54 end
55 R f _ o l d = Rf;
56 end
'/ Compute e v a l u a t i o n m a t r i x
57 DM_inteval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , i n t c t r s { k } , e p ( k ) ) ;
58 DM_bdyeval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , b d y c t r s { k } , e p ( k ) ) ;
59 LEM = L r b f ( e p ( k ) , D M _ i n t e v a l ) ;
60 BEM = rbf(ep(k),DM_bdyeval);
61 EM = [LEM BEM] ;
% E v a l u a t e RBF approximation
62 Pf = EM*coef{k};
63 i f (k > 1)
64 Pf = P f _ o l d + P f ;
65 end
66 Pf_old = Pf;
68 rms.err = n o r m ( P f - e x a c t ) / n e v a l ;
69 f p r i n t f ('RMS e r r o r : /.e\n', r m s . e r r )
71 i f (k > 1)
72 max_rate = l o g ( m a x e r r _ o l d / m a x e r r ) / l o g ( 2 ) ;
73 rms_rate = l o g ( r m s _ e r r _ o l d / r m s _ e r r ) / l o g ( 2 ) ;
74 f p r i n t f CRMS r a t e : % f \n', rms_rate)
75 f p r i n t f ( ' M a x e r r o r r a t e : % f \ n ' , max_rate)
76 end
77 maxerr_old = maxerr; r m s _ e r r _ o l d = rms.err;
'/, P l o t c o l l o c a t i o n s o l u t i o n
78 fview = [-30,30];
81 end
For n o n - s y m m e t r i c c o l l o c a t i o n w e c a n delete l i n e 3 o f P r o g r a m 41.2 a n d need t o

replace lines 29-37 b y
DM_intdata = D i s t a n c e M a t r i x C S R B F ( i n t d a t a , c t r s { k } , e p ( k ) ) ;
DM_bdydata = D i s t a n c e M a t r i x C S R B F ( b d y d a t a , c t r s { k } , e p ( k ) ) ;
LCM = L r b f ( e p ( k ) , D M _ i n t d a t a ) ;
BCM = rbf(ep(k),DM_bdydata);
CM = [LCM; BCM];
lines 41-49 b y
DM_intres = D i s t a n c e M a t r i x C S R B F ( i n t r e s , c t r s { j } , e p ( j ) ) ;
DM_bdyres = D i s t a n c e M a t r i x C S R B F ( b d y r e s , c t r s - C j } , e p ( j ) ) ;
LRM = L r b f ( e p ( j ) , D M _ i n t r e s ) ;
BRM = rbf(ep(j),DM_bdyres);
RM{j} = [LRM; BRM];
and lines 57-61 b y
DM_eval = D i s t a n c e M a t r i x C S R B F ( e p o i n t s , c t r s f k } , e p ( k ) ) ;
EM = rbf(ep(k),DM_eval);
Table 41.3 Stationary symmetric and non-symmetric multilevel collocation solu-

tions of Example 39.1 using C S R B F s with initial scale parameter e = 0.5. Interior
Halton points, additional centers on boundary.
symmetric non-symmetric
(interior points) RMS-error rate RMS-error rate % nonzero
3x3 2.491453e-002 1.512255e-002 100

5x5 1.011182e-002 1.3009 3.785466e-003 1.9982 89.53
9x9 9.016652e-003 0.1654 7.873870e-004 2.2653 40.03
17 x 17 8.995221e-003 0.0034 2.470405e-004 1.6723 13.55
33 x 33 8.994046e-003 0.0002 1.070175e-004 1.2069 4.00
65 x 65 8.993892e-003 0.0000 8.334939e-005 0.3606 1.12
129 x 129 8.993969e-003 -0.0000 7.863637e-005 0.0840 0.29
We n o t e t h a t t h e non-convergence b e h a v i o r c a n be o b s e r v e d for b o t h t h e s y m -
metric and the non-symmetric approach. However, w i t h the non-symmetric ap-
p r o a c h t h e convergence ceases a t a s i g n i f i c a n t l y l a t e r stage. T h e e x p l a n a t i o n for t h e
different convergence b e h a v i o r o f t h e t w o m e t h o d s is t h e same as t h a t p r e s e n t e d a t
t h e end o f t h e p r e v i o u s e x a m p l e . T h e h i g h e r d e r i v a t i v e s r e q u i r e d for t h e s y m m e t r i c
m e t h o d are n u m e r i c a l l y o f a m o r e l o c a l i z e d n a t u r e even t h o u g h here t h e spar-
s i t y p a t t e r n s o f t h e s y m m e t r i c a n d n o n - s y m m e t r i c m a t r i c e s are i d e n t i c a l (c.f. the
r i g h t - m o s t c o l u m n i n Tables 41.3 a n d 4 1 . 4 ) .
Table 41.4 Stationary symmetric and non-symmetric multilevel collocation solu-

tions of Example 39.1 using C S R B F s with initial scale parameter e = 0.25. Interior
Halton points, additional centers on boundary.
symmetric non-symmetric
(interior points) RMS-error rate RMS-error rate % nonzero
3x3 5.866837e-003 6.077025e-003 100

5x5 7.287305e-004 3.0091 5.751941e-004 3.4012 100
9x9 1.193468e-004 2.6102 1.168520e-004 2.2994 91.90
17 x 17 3.188905e-005 1.9040 1.482129&-005 2.9789 42.69
33 x 33 2.530241e-005 0.3338 2.550329e-006 2.5389 14.34
65 x 65 2.487657e-005 0.0245 6.067264e-007 2.0716 4.17
129 x 129 2.485184e-005 0.0014 7.348170e-008 3.0456 1.13
A l s o , t h e a c c u r a c y t h a t c a n be o b t a i n e d w i t h t h e m u l t i l e v e l a l g o r i t h m w h i l e
better t h a n using the s t a t i o n a r y approach w i t h o u t residual i t e r a t i o n i n E x a m p l e 41.1
is c o n s i d e r a b l y p o o r e r t h a n w h a t w e were able t o o b t a i n w i t h g l o b a l l y s u p p o r t e d
f u n c t i o n s (c.f. E x a m p l e s 3 9 . 1 , 4 0 . 1 , or t h e n o n - s t a t i o n a r y p a r t o f E x a m p l e 4 1 . 1 ) .
T h e same s a t u r a t i o n p h e n o m e n o n was o b s e r v e d b y W e n d l a n d i n t h e c o n t e x t
of a m u l t i l e v e l G a l e r k i n a l g o r i t h m for c o m p a c t l y s u p p o r t e d R B F s (see [ W e n d l a n d
(1999b)] as w e l l as o u r discussion i n C h a p t e r 4 4 ) .
I t has been suggested t h a t t h e convergence b e h a v i o r o f t h e m u l t i l e v e l c o l l o c a -
t i o n a l g o r i t h m m a y be l i n k e d t o t h e p h e n o m e n o n o f a p p r o x i m a t e a p p r o x i m a t i o n .
H o w e v e r , so far n o c o n n e c t i o n has been e s t a b l i s h e d .
A s was s h o w n i n [Fasshauer ( 1 9 9 9 d ) ] a possible r e m e d y for t h e non-convergence
p r o b l e m is . O n e m i g h t also e x p e c t t h a t a s l i g h t l y different s c a l i n g o f t h e s u p p o r t
sizes o f t h e basis f u n c t i o n s (such t h a t t h e b a n d w i d t h o f t h e m a t r i x is a l l o w e d t o i n -
crease s l o w l y f r o m one i t e r a t i o n t o t h e n e x t , i.e., m o v i n g t o w a r d t h e n o n - s t a t i o n a r y
s e t t i n g ) w i l l lead t o b e t t e r r e s u l t s . I n [Fasshauer (1999d)] i t was s h o w n t h a t t h i s is
i n fact t r u e . H o w e v e r , s m o o t h i n g f u r t h e r i m p r o v e d t h e convergence. A discussion
of t h e idea o f p o s t - c o n d i t i o n i n g v i a s m o o t h i n g is b e y o n d t h e scope o f t h i s t e x t . W e
refer t h e reader t o t h e p a p e r [Fasshauer a n d J e r o m e (1999)].
Chapter 42
Using Radial Basis Functions in

Pseudospectral Mode
P s e u d o s p e c t r a l ( P S ) m e t h o d s are k n o w n as h i g h l y a c c u r a t e solvers for p a r t i a l dif-

ferential equations. T h e basic idea (see, e.g., [ F o r n b e r g (1998); T r e f e t h e n (2000)])
is t o use a set o f v e r y s m o o t h a n d g l o b a l basis f u n c t i o n s B , 3 j = 1 , . . . , N, such as
p o l y n o m i a l s t o represent t h e a p p r o x i m a t e s o l u t i o n o f t h e P D E v i a
u(x) = ^CjBjix), x e R . (42.1)

3= 1
Since m o s t o f o u r discussion w i l l focus o n a r e p r e s e n t a t i o n of the spatial part of

t h e s o l u t i o n w e w i l l a t first i g n o r e t h e t i m e v a r i a b l e t h a t m a y be a p a r t o f t h e t h e
f o r m u l a s for u. W e w i l l l a t e r e m p l o y s t a n d a r d t i m e - s t e p p i n g procedures t o deal
w i t h t h e t e m p o r a l p a r t o f t h e s o l u t i o n . M o r e o v e r , since s t a n d a r d p s e u d o s p e c t r a l
m e t h o d s are designed for t h e u n i v a r i a t e case we i n i t i a l l y l i m i t ourselves t o single-
v a r i a b l e f u n c t i o n s . L a t e r we w i l l generalize t h e discussion t o m u l t i v a r i a t e ( s p a t i a l )
p r o b l e m s b y u s i n g r a d i a l basis f u n c t i o n s .
A n i m p o r t a n t feature o f p s e u d o s p e c t r a l m e t h o d s is t h e fact t h a t one u s u a l l y
is c o n t e n t w i t h o b t a i n i n g a n a p p r o x i m a t i o n t o t h e s o l u t i o n o n a discrete set o f
g r i d p o i n t s Xi, i 1 , . . . , AT. O n e o f several w a y s t o i m p l e m e n t t h e p s e u d o s p e c t r a l
m e t h o d is v i a so-called differentiation matrices, i.e., one finds a m a t r i x D such t h a t
a t t h e c o l l o c a t i o n p o i n t s Xi w e have
u' = Du, (42.2)
w h e r e u = [u(xi),..., U(XN)] T
is t h e v e c t o r o f values o f t h e a p p r o x i m a t e s o l u t i o n
u a t t h e c o l l o c a t i o n p o i n t s . F r e q u e n t l y , o r t h o g o n a l p o l y n o m i a l s such as C h e b y s h e v
p o l y n o m i a l s are used as basis f u n c t i o n s , a n d t h e c o l l o c a t i o n p o i n t s are c o r r e s p o n d i n g
C h e b y s h e v p o i n t s . I n t h i s case t h e entries o f t h e d i f f e r e n t i a t i o n m a t r i x are e x p l i c i t l y
k n o w n (see, e.g., [Trefethen ( 2 0 0 0 ) ] ) .
Example 42.1. I n order t o get a n idea o f h o w t h e d i f f e r e n t i a t i o n m a t r i x is used t o

solve a p a r t i a l d i f f e r e n t i a l e q u a t i o n we consider t h e f o l l o w i n g s i m p l e o n e - d i m e n s i o n a l
387
t r a n s p o r t e q u a t i o n (c.f. t h e n u m e r i c a l e x p e r i m e n t s i n S e c t i o n 43.1.1 b e l o w ) :
u {x,t)
t + cu (x,t) x = 0, x > - 1 , t > 0 , (42.3)
u ( - l , * ) = 0,
u(x,0) = / ( x ) .
I n o r d e r t o solve t h i s p r o b l e m w e d i s c r e t i z e t h e s p a t i a l d o m a i n w i t h t h e c o l l o c a t i o n
p o i n t s Xi, i = 1 , . . . , N, so t h a t for a n y fixed time t n we h a v e t h e v e c t o r =
T
[u(xi,t ),...
n ,u(xN,t )] n o f values o f t h e a p p r o x i m a t e s o l u t i o n . I n o r d e r t o m a r c h
i n t i m e w e use a s t a n d a r d f o r w a r d difference a p p r o x i m a t i o n o f t h e t i m e d e r i v a t i v e ,
i.e.,
U(X tn+l) U(X tn

Mx,t ) '
n - ' \
t (42.4)
w h e r e At = t n + i t. n I n our vectorized n o t a t i o n at t h e collocation p o i n t s applica-

t i o n o f t h e d i f f e r e n t i a t i o n m a t r i x t o express t h e s p a t i a l d e r i v a t i v e a l o n g w i t h (42.4)
for t h e t i m e d e r i v a t i v e leads t o
n + 1 ) ( n )
u^ = u - cAtDu^
for t h e s o l u t i o n o f ( 4 2 . 3 ) . T h u s , t h e r e is no n e e d as w i t h t h e R B F c o l l o c a t i o n
m e t h o d s s t u d i e d earlier t o c o m p u t e t h e e x p a n s i o n coefficients Cj i n t h e represen-
t a t i o n (42.1) o f t h e a p p r o x i m a t e s o l u t i o n . A l s o , n o l i n e a r s y s t e m s are s o l v e d d u r i n g
t h e t i m e - m a r c h i n g phase o f t h e code. T h e d e t e r m i n a t i o n o f t h e d i f f e r e n t i a t i o n m a -
t r i x w i l l , however, i n v o l v e s o l u t i o n o f a l i n e a r s y s t e m i n t h e R B F f r a m e w o r k .
W e are i n t e r e s t e d i n u s i n g i n f i n i t e l y s m o o t h r a d i a l basis f u n c t i o n s i n t h e pseu-

d o s p e c t r a l e x p a n s i o n ( 4 2 . 1 ) , i.e., Bj(x) = (p(\\x Xj\\), w h e r e </? is one o f o u r s t r i c t l y
p o s i t i v e d e f i n i t e basic f u n c t i o n s such as a G a u s s i a n or a n inverse m u l t i q u a d r i c . W e
w i l l also e x p e r i m e n t w i t h t h e use o f f u n c t i o n s h a v i n g o n l y l i m i t e d s m o o t h n e s s s u c h
as t h e g l o b a l l y s u p p o r t e d M a t e r n f u n c t i o n s o r W e n d l a n d f u n c t i o n s w i t h a large s u p -
p o r t . W i t h some a d d i t i o n a l n o t a t i o n a l effort a l l t h a t f o l l o w s c a n also be f o r m u l a t e d
for c o n d i t i o n a l l y p o s i t i v e d e f i n i t e f u n c t i o n s s u c h as m u l t i q u a d r i c s .
42.1 Differentiation Matrices
I n o r d e r t o u n d e r s t a n d h o w t o find a d i f f e r e n t i a t i o n m a t r i x consider t h e expansion

(42.1) a n d let Bj, j = 1 , . . . , N, be a n a r b i t r a r y l i n e a r l y i n d e p e n d e n t set o f s m o o t h
f u n c t i o n s t h a t w i l l serve as t h e basis for o u r a p p r o x i m a t i o n space.
I f we e v a l u a t e (42.1) a t t h e c o l l o c a t i o n p o i n t s Xi, i 1 , . . . , N, t h e n w e get
N
x
H i) = '^2 j j( i)i c B x
i=l,...,N,
3=1
or i n matrix-vector n o t a t i o n
u = Ac, (42.5)
42. Using Radial Basis Functions in Pseudospectral Mode 389
w h e r e c = [ c i , . . . , C J V ] is t h e coefficient vector, t h e evaluation

T
matrix A has entries
A^ = Bj(xi), a n d u is as before.
B y l i n e a r i t y we c a n also use t h e e x p a n s i o n (42.1) t o c o m p u t e t h e d e r i v a t i v e o f
u b y d i f f e r e n t i a t i n g t h e basis f u n c t i o n s
N
d d
I f w e a g a i n e v a l u a t e at t h e c o l l o c a t i o n p o i n t s Xi, t h e n we get i n m a t r i x - v e c t o r
notation
u' = A c, x (42.6)
w h e r e u a n d c are as i n (42.5) above, a n d t h e derivative matrix A x has entries

-^Bj(xi), o r , i n t h e case o f r a d i a l basis f u n c t i o n s , ^ V?(||^ I!) 1^=^^-
I n o r d e r t o o b t a i n t h e d i f f e r e n t i a t i o n m a t r i x D we need t o ensure i n v e r t i b i l i t y o f
t h e e v a l u a t i o n m a t r i x A. T h i s depends b o t h o n t h e basis f u n c t i o n s chosen as w e l l
as t h e l o c a t i o n o f t h e c o l l o c a t i o n p o i n t s Xi. F o r u n i v a r i a t e p o l y n o m i a l s i t is w e l l -
k n o w n t h a t t h e e v a l u a t i o n m a t r i x is i n v e r t i b l e for a n y set o f d i s t i n c t c o l l o c a t i o n
points. I n p a r t i c u l a r , i f t h e p o l y n o m i a l s are w r i t t e n i n c a r d i n a l (or Lagrange)
f o r m , t h e n t h e e v a l u a t i o n m a t r i x is t h e i d e n t i t y m a t r i x . I f we use s t r i c t l y p o s i t i v e
d e f i n i t e r a d i a l basis f u n c t i o n s , t h e n t h e m a t r i x A is i n v e r t i b l e for a n y set o f d i s t i n c t
s
c o l l o c a t i o n p o i n t s (also n o n - u n i f o r m l y spaced p o i n t s a n d i n R , s > 1) a c c o r d i n g
t o o u r discussion i n C h a p t e r 3. C a r d i n a l r a d i a l basis f u n c t i o n s , o n t h e o t h e r h a n d ,
are r a t h e r d i f f i c u l t t o o b t a i n . F o r t h e special case o f u n i f o r m o n e - d i m e n s i o n a l g r i d s
e x p l i c i t f o r m u l a s can be f o u n d i n [ P l a t t e a n d D r i s c o l l (2005)]. A general discussion
o f t h e c a r d i n a l r e p r e s e n t a t i o n o f R B F s is g i v e n i n C h a p t e r 14. I n t h e f o l l o w i n g we
w i l l n o t insist o n h a v i n g a c a r d i n a l representation.
Now t h a t we have discussed t h e i n v e r t i b i l i t y o f A, we can use (42.5) t o f o r m a l l y
1
solve for t h e coefficient v e c t o r c = A~ u, a n d w i t h t h i s (42.6) y i e l d s
x
u' = A A~ u,
x
so t h a t t h e differentiation matrix D c o r r e s p o n d i n g t o (42.2) is g i v e n b y

l
D = A A~ .
x
F o r m o r e c o m p l e x l i n e a r d i f f e r e n t i a l o p e r a t o r s w i t h c o n s t a n t coefficients we
p r o c e e d i n a n analogous fashion t o o b t a i n a d i s c r e t i z e d d i f f e r e n t i a l o p e r a t o r (differ-
entiation matrix)
L = A A~\ C (42.7)
w h e r e t h e m a t r i x Ac has entries (Ac)ij = Bj(xi). I n t h e case o f r a d i a l basis

f u n c t i o n s these entries are o f t h e f o r m (Ac)ij = </?(||x ^ | | ) | .
a = a ..-
I n the context of pseudospectral methods the differentiation matrices D or
L c a n n o w be used t o solve a l l k i n d s o f P D E s ( t i m e - d e p e n d e n t as w e l l as t i m e -
independent). S o m e t i m e s o n l y m u l t i p l i c a t i o n b y L is r e q u i r e d , e.g., for m a n y t i m e -
s t e p p i n g a l g o r i t h m s such as t h e e x a m p l e g i v e n a t t h e b e g i n n i n g o f t h e c h a p t e r . For
o t h e r p r o b l e m s one needs t o be able t o i n v e r t L . I n t h e s t a n d a r d PS case i t is

k n o w n t h a t t h e Chebyshev d i f f e r e n t i a t i o n m a t r i x has a n i V - f o l d zero eigenvalue (see
[ C a n u t o et al. (1988)], p . 70), a n d t h u s is n o t i n v e r t i b l e b y itself. H o w e v e r , once
b o u n d a r y c o n d i t i o n s are t a k e n i n t o c o n s i d e r a t i o n t h e s i t u a t i o n changes (see, e.g.,
[Trefethen (2000)], p . 6 7 ) .
E x a m p l e 4 2 . 2 . T o o b t a i n a l i t t l e m o r e i n s i g h t i n t o t h e special p r o p e r t i e s o f r a -
d i a l basis f u n c t i o n s let us p r e t e n d t o solve t h e ill-posed linear elliptic P D E of the
f o r m Cu = f b y i g n o r i n g b o u n d a r y c o n d i t i o n s . A n a p p r o x i m a t e s o l u t i o n at the
c o l l o c a t i o n p o i n t s Xi m i g h t be o b t a i n e d b y s o l v i n g t h e discrete linear s y s t e m
Lu = /,
where / contains t h e values o f / at t h e c o l l o c a t i o n p o i n t s a n d L is as above. In

o t h e r w o r d s , t h e s o l u t i o n at t h e c o l l o c a t i o n p o i n t s is g i v e n (see (42.7)) b y
u = L- f = 1
A(Acr f, 1
a n d we see t h a t i n v e r t i b i l i t y o f L ( a n d therefore Ac) is r e q u i r e d t o proceed.

As mentioned above, t h e d i f f e r e n t i a t i o n m a t r i x for p s e u d o s p e c t r a l m e t h o d s
based o n Chebyshev p o l y n o m i a l s is s i n g u l a r . T h i s is o n l y n a t u r a l since t h e p r o b l e m
of r e c o n s t r u c t i n g a n u n k n o w n f u n c t i o n f r o m t h e values o f i t s d e r i v a t i v e s alone is
ill-posed.
However, i f we use r a d i a l basis f u n c t i o n s t h e results o n generalized Hermite
i n t e r p o l a t i o n c i t e d i n C h a p t e r 36 ensure t h a t t h e m a t r i x Ac is i n v e r t i b l e p r o v i d e d a
s t r i c t l y p o s i t i v e d e f i n i t e basic f u n c t i o n is used a n d t h e d i f f e r e n t i a l o p e r a t o r is e l l i p t i c .
Therefore, t h e basic d i f f e r e n t i a t i o n m a t r i x L for R B F - b a s e d p s e u d o s p e c t r a l m e t h o d s
is i n v e r t i b l e .
T h e o b s e r v a t i o n j u s t m a d e suggests t h a t R B F m e t h o d s are s o m e t i m e s " t o o g o o d
t o be t r u e " . T h e y m a y deliver a " s o l u t i o n " even for i l l - p o s e d p r o b l e m s . T h i s is a
consequence o f t h e o p t i m a l i t y p r i n c i p l e s o f C h a p t e r 18, i.e., as t h e m i n i m i z e r o f t h e
n a t i v e space n o r m R B F m e t h o d s possess a b u i l t - i n regularization capability. This
i n t e r e s t i n g feature o f R B F s has r e c e n t l y been used t o solve i l l - p o s e d p r o b l e m s (see,
e.g., [Cheng a n d C a b r a l ( 2 0 0 5 ) ] ) .
42.2 PDEs w i t h Boundary Conditions via Pseudospectral Methods
F i r s t we discuss h o w t h e linear e l l i p t i c P D E p r o b l e m
Cu = / in Q
w i t h Dirichlet boundary condition
u = g on T = dfl
can be solved u s i n g p s e u d o s p e c t r a l m e t h o d s . S o m e t i m e s one c a n f i n d basis f u n c t i o n s

t h a t a l r e a d y satisfy t h e b o u n d a r y c o n d i t i o n s (especially for p e r i o d i c problems).
However, i f t h e basis f u n c t i o n s d o n o t satisfy t h e b o u n d a r y c o n d i t i o n s w e c a n f o l l o w

a v e r y s i m p l e p r o c e d u r e (see, e.g., P r o g r a m 36 o f [ T r e f e t h e n ( 2 0 0 0 ) ] ) . J u s t t a k e t h e
d i f f e r e n t i a t i o n m a t r i x L based o n a l l c o l l o c a t i o n p o i n t s Xi, a n d t h e n replace those
rows o f L corresponding t o collocation at b o u n d a r y points w i t h u n i t vectors t h a t
have a one i n t h e p o s i t i o n c o r r e s p o n d i n g t o t h e d i a g o n a l o f L . T h u s , t h e c o n d i t i o n
u = g w i l l be e x p l i c i t l y enforced a t t h i s p o i n t as s o o n as w e set t h e r i g h t - h a n d side
t o t h e c o r r e s p o n d i n g value o f g.
B y r e o r d e r i n g t h e rows a n d c o l u m n s o f t h e r e s u l t i n g m a t r i x w e o b t a i n a b l o c k
matrix of the form
M P
Lr = (42.8)
0 I
w h e r e t h e non-zero square b l o c k s M a n d I are o f size ( N N B ) X ( N N Q ) and

iVg x N B , respectively. Here N& denotes t h e n u m b e r o f g r i d p o i n t s o n t h e b o u n d a r y
r.
On the grid of collocation points the solution of the P D E w i t h boundary condi-
t i o n s is t h e n o b t a i n e d b y s o l v i n g t h e b l o c k l i n e a r s y s t e m
7 (42.9)
.9
w h e r e t h e vectors / a n d g collect t h e values o f / a n d g a t t h e respective c o l l o c a -
t i o n p o i n t s , a n d t h e v e c t o r u o f g r i d values o f t h e a p p r o x i m a t e s o l u t i o n has been
r e o r d e r e d a l o n g w i t h t h e c o l u m n s o f t h e m a t r i x so t h a t i t c a n be d e c o m p o s e d i n t o
u = [UZ,UB] -
T
Here ux collects t h e values i n t h e i n t e r i o r o f t h e d o m a i n fl a n d u g
collects t h e values o n t h e b o u n d a r y .
S o l v i n g (42.9) for u& = g a n d s u b s t i t u t i n g t h i s b a c k i n t o t h e t o p p a r t w e o b t a i n
u T = M-\f-Pg),
or, i n t h e case o f homogeneous b o u n d a r y c o n d i t i o n s ,

1
ux = M~ f.
We n o w see t h a t a l l t h a t r e a l l y m a t t e r s is w h e t h e r t h e m a t r i x M is i n v e r t i b l e . I n t h e
case o f Chebyshev p o l y n o m i a l basis f u n c t i o n s a n d t h e s e c o n d - d e r i v a t i v e o p e r a t o r
c o u p l e d w i t h different t y p e s o f b o u n d a r y c o n d i t i o n s t h i s q u e s t i o n has been answered
a f f i r m a t i v e l y b y G o t t l i e b a n d L u s t m a n (see [ G o t t l i e b a n d L u s t m a n (1983)], o r , e.g.,
S e c t i o n 11.4 o f [ C a n u t o et al. (1988)]). P r o g r a m 15 o f [ T r e f e t h e n (2000)] also
p r o v i d e s a discussion a n d a n i l l u s t r a t i o n o f one such p r o b l e m . W e w i l l l o o k a t t h e
m a t r i x M i n the R B F context i n t h e next section.
42.3 A Non-Symmetric RBF-based Pseudospectral Method
Once b o u n d a r y c o n d i t i o n s are a d d e d t o t h e P D E Cu = / , t h e n e i t h e r o f t h e t w o
c o l l o c a t i o n approaches discussed i n C h a p t e r s 3 8 - 4 1 are c o m m o n l y used i n t h e R B F
community. Recall t h a t i n K a n s a ' s n o n - s y m m e t r i c m e t h o d [ K a n s a (1990b)] one

s t a r t s w i t h t h e expansion
N
u(x)
7= 1
Cj$j(x), x e fl c (42.10)
j u s t as before (c.f. ( 4 2 . 1 ) ) . H o w e v e r , t h e coefficient v e c t o r c is n o w a c t u a l l y c o m -

p u t e d b y i n s e r t i n g (42.10) i n t o t h e P D E a n d b o u n d a r y c o n d i t i o n s b y f o r c i n g these
equations t o be satisfied at t h e c o l l o c a t i o n p o i n t s X i . T h e R B F collocation solution
is therefore o b t a i n e d b y s o l v i n g t h e l i n e a r s y s t e m
C
7" (42.11)
A .9.
where / a n d g are as above, a n d t h e ( r e c t a n g u l a r ) m a t r i c e s Ac a n d A are o f t h e

form
(Ach = CQjixi) = &p{\\x - XJIDU^ , i = 1, ,N - N,

B j = 1,... ,N,
A i:j =&j(x )
i = (p(\\x -x \\),
i j i = N - N B + l,...,N, j = l,...,N.
A s s u m i n g t h a t t h e s y s t e m m a t r i x i n (42.11) is i n v e r t i b l e one t h e n o b t a i n s t h e ap-

p r o x i m a t e s o l u t i o n (at a n y p o i n t x) b y u s i n g t h e coefficients c i n (42.10). H o w e v e r ,
as was m e n t i o n e d earlier, c o u n t e r e x a m p l e s i n [ H o n a n d Schaback (2001)] show t h a t
c e r t a i n c o l l o c a t i o n g r i d s do n o t a l l o w i n v e r t i b i l i t y o f t h e s y s t e m m a t r i x i n ( 4 2 . 1 1 ) .
I f we are i n t e r e s t e d i n t h e R B F c o l l o c a t i o n s o l u t i o n at t h e c o l l o c a t i o n p o i n t s
only, t h e n (using c f r o m (42.11) a n d once a g a i n a s s u m i n g i n v e r t i b i l i t y o f t h e s y s t e m
m a t r i x ) we get
Ac
u = Ac = A
A
w i t h e v a l u a t i o n m a t r i x A such t h a t Aij = &j(xi) as above. T h i s suggests t h a t

( a c c o r d i n g t o o u r discussion i n S e c t i o n 42.1) t h e d i s c r e t i z e d d i f f e r e n t i a l o p e r a t o r L
based o n t h e g r i d p o i n t s X i , i 1,... ,N, a n d basis f u n c t i o n s <&j, j 1 , . . . , N , is
given by
'Ac 1
=
A- .
A
I n d e e d , we have
'Ac
A -l _ 'M P~
A 0 /
w i t h t h e same blocks M, P , 0 a n d / as above. T o see t h i s we i n t r o d u c e t h e f o l l o w i n g
notation:
r T
A = and A 1
= [b x ... b]
N
a
L NJ
w i t h c o l u m n vectors a* a n d 6j such t h a t a[bj For K a n s a ' s m a t r i x f r o m

(42.11) t h i s n o t a t i o n i m p l i e s
T
Ac a
C , N - N B
A T
a
N - N + l B
T
a N
where we have used a n analogous n o t a t i o n t o d e n o t e t h e r o w s o f t h e b l o c k Ac-

N o w t h e discretized d i f f e r e n t i a l o p e r a t o r based o n t h e n o n - s y m m e t r i c c o l l o c a t i o n
a p p r o a c h is g i v e n b y
T
Ac 1
a
C , N - N
A ' = T
B
[&i ... b ] N
A a
N - N + l B
T
aN
AcAj 1
A A I -i
C B
AA.
1 1
Here we p a r t i t i o n e d A i n t o t h e blocks A x w i t h N N& c o l u m n s c o r r e s p o n d i n g t o
1
i n t e r i o r p o i n t s , a n d A^ with columns corresponding to the remaining boundary
p o i n t s . A l s o , we m a d e use o f t h e fact t h a t afbj = 5.13
T h i s is t h e same as (see ( 4 2 . 8 ) )
M P
0 /
w h e r e M a n d P were o b t a i n e d f r o m t h e discrete d i f f e r e n t i a l o p e r a t o r (42.7)
L = ArA - l [AcAj 1
Ac 1
A^ ]
b y r e p l a c i n g c e r t a i n rows w i t h u n i t vectors as we e x p l a i n e d is c o m m o n p r a c t i c e for

handling the Dirichlet b o u n d a r y conditions i n the PS approach.
T h u s , we have j u s t seen t h a t p r o v i d e d we use t h e same basis f u n c t i o n s <>j
a n d t h e same g r i d o f c o l l o c a t i o n p o i n t s ccj t h e n o n - s y m m e t r i c R B F c o l l o c a t i o n
a p p r o a c h for t h e s o l u t i o n o f a n e l l i p t i c P D E w i t h D i r i c h l e t b o u n d a r y conditions
followed b y e v a l u a t i o n at t h e g r i d p o i n t s is i d e n t i c a l t o a p s e u d o s p e c t r a l a p p r o a c h .
However, neither of the two methods is well-defined in general since t h e y b o t h r e l y
on the invertibility of Kansa's collocation m a t r i x .
O n t h e o t h e r h a n d , we showed above t h a t w e c a n a l w a y s form the discretized

differential operator
'Ac 1
'M P~
A' =
_A _ 0 /
even i f K a n s a ' s m a t r i x is n o t i n v e r t i b l e . T h i s i m p l i e s t h a t we c a n safely use t h e

non-symmetric R B F pseudospectral approach whenever inversion of the discretized
d i f f e r e n t i a l o p e r a t o r is n o t r e q u i r e d (e.g., i n the context of explicit time-stepping
for p a r a b o l i c P D E s ) .
A n o t h e r i n t e r e s t i n g feature t h a t w e w i l l i l l u s t r a t e i n t h e n e x t c h a p t e r is t h e
fact s h o w n r e c e n t l y b y a n u m b e r o f a u t h o r s (see, e.g., [ d e B o o r (2006); D r i s c o l l
a n d F o r n b e r g ( 2 0 0 2 ) ; Schaback ( 2 0 0 5 ) ; Schaback ( 2 0 0 6 b ) ] ) t h a t i n t h e l i m i t i n g case
of " f l a t " basis f u n c t i o n s t h e o n e - d i m e n s i o n a l R B F i n t e r p o l a n t y i e l d s a p o l y n o m i a l
i n t e r p o l a n t . Since we also m e n t i o n e d e a r l i e r t h a t t h e d i s c r e t i z e d d i f f e r e n t i a l o p e r a t o r
Lr is i n v e r t i b l e i f a u n i v a r i a t e p o l y n o m i a l basis is used we c a n c o n c l u d e t h a t K a n s a ' s
c o l l o c a t i o n m a t r i x is i n v e r t i b l e i n t h e l i m i t i n g case e > 0.
42.4 A Symmetric R B F - b a s e d Pseudospectral Method
T h e second R B F c o l l o c a t i o n m e t h o d is t h e s y m m e t r i c a p p r o a c h whose s y s t e m m a -
t r i x is i n v e r t i b l e for a l l g r i d c o n f i g u r a t i o n s a n d a n y s t r i c t l y p o s i t i v e d e f i n i t e basic
f u n c t i o n as e x p l a i n e d i n C h a p t e r 38.
R e c a l l t h a t for t h e s y m m e t r i c c o l l o c a t i o n m e t h o d one uses a different basis t h a n
for t h e n o n - s y m m e t r i c a p p r o a c h ( 4 2 . 1 0 ) , i.e., a different f u n c t i o n space t o r e p r e -
sent t h e a p p r o x i m a t e s o l u t i o n . F o r t h e same e l l i p t i c P D E a n d D i r i c h l e t b o u n d a r y
c o n d i t i o n s as above one n o w s t a r t s w i t h
N-N N
(*)= Y. + E ^;(*)-
B
c 42 12
j=N-N B + l
(-)
Since t h e <3>j are assumed t o be r a d i a l f u n c t i o n s , i.e., $>j(x) = <p(\\x xj\\) the
f u n c t i o n a l s | c a n be i n t e r p r e t e d as a n a p p l i c a t i o n o f t o ip v i e w e d as a f u n c t i o n
of t h e second v a r i a b l e f o l l o w e d b y e v a l u a t i o n a t X j (see t h e discussion i n C h a p t e r s 36
T
a n d 3 8 ) . O n e o b t a i n s t h e coefficients c = [ c i , c # ] b y solving the linear system
Ace* Ac ' c~ x f (42.13)

Acs A - C
B . .9.
a n
H e r e t h e b l o c k s Ace* d A, r e s p e c t i v e l y , are square m a t r i c e s c o r r e s p o n d i n g t o t h e
i n t e r a c t i o n o f i n t e r i o r c o l l o c a t i o n p o i n t s w i t h each o t h e r a n d b o u n d a r y c o l l o c a t i o n
p o i n t s w i t h each o t h e r . A s discussed i n C h a p t e r 38 (for centers c o i n c i d i n g w i t h
c o l l o c a t i o n p o i n t s ) t h e i r entries are g i v e n b y
(Acce)ij = \C [</?(!!*- i,j = 1,..., N NB,

X 37 i
Aij =<p(\\x -X \\),

i j i,j = N - N B + 1,...,N.
T h e o t h e r t w o b l o c k s are r e c t a n g u l a r , and correspond to interaction of interior

p o i n t s w i t h b o u n d a r y p o i n t s a n d v i c e versa. T h e y are defined as
(Ach = [&p(\\x - x M * ^ > i = !> . N - N , B j = N - N B + 1,..., N,

(A^)ij= [tfviWxi-xW)]^., i = N N B + 1,..., N, j = 1,..., N N. B
A s a l r e a d y m e n t i o n e d , i t is w e l l k n o w n t h a t t h e s y s t e m m a t r i x i n (42.13) is
i n v e r t i b l e for s t r i c t l y p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s . T h i s implies t h a t we can
o b t a i n t h e a p p r o x i m a t e s o l u t i o n at a n y p o i n t x b y u s i n g t h e c o m p u t e d coefficients
c i n ( 4 2 . 1 2 ) . T h u s t h i s R B F c o l l o c a t i o n m e t h o d is r a t h e r s i m i l a r t o K a n s a ' s n o n -
s y m m e t r i c m e t h o d w i t h t h e n o t a b l e difference t h a t t h e c o l l o c a t i o n a p p r o a c h is w e l l -
defined.
A nice c o n n e c t i o n b e t w e e n t h e s y m m e t r i c a n d n o n - s y m m e t r i c c o l l o c a t i o n m e t h -
ods appears i f we consider t h e c o r r e s p o n d i n g s y m m e t r i c p s e u d o s p e c t r a l approaches.
T o t h i s e n d we use t h e e x p a n s i o n (42.12) o n w h i c h t h e s y m m e t r i c R B F c o l l o c a -
t i o n m e t h o d is based as s t a r t i n g p o i n t for a p s e u d o s p e c t r a l m e t h o d , i.e.,
N-N B N
u(x)= Y c *${x)+
j Y, c
^ ( * ) - (42-14)
j=N-N +l B
I n vectorized n o t a t i o n this corresponds t o
T
u(x) = [a^. (x)$ a (x)]
c
B
T
w i t h a p p r o p r i a t e r o w v e c t o r s a^{x) a n d a (x). Evaluated o n the g r i d of colloca-
t i o n p o i n t s t h i s becomes
T
cx
u=[A * c A]
c
B
T
Here t h e b l o c k s A^z a n d A o f t h e e v a l u a t i o n m a t r i x are r e c t a n g u l a r m a t r i c e s w i t h
entries
(AcOii = [&vi\\xi - x i = l,...,N, j = 1,...,N- N, B
( i ) i j = <p(\\Xi
T
-Xj\\), 1,...,N, j = N - N B + l,...,N,
c o r r e s p o n d i n g t o e v a l u a t i o n o f t h e basis f u n c t i o n s used i n (42.12) a t t h e c o l l o c a t i o n

x 1 S m a c
p o i n t s Xi. N o t e t h a t t h e second m a t r i x w i t h entries ^ ( l l ^ i j\\) f t the
transpose o f t h e c o r r e s p o n d i n g b l o c k o f t h e s y s t e m m a t r i x i n (42.11) for K a n s a ' s
m e t h o d ( a n d t h u s use o f t h e t i l d e - n o t a t i o n is j u s t i f i e d ) .
M o r e o v e r , t h e r a d i a l s y m m e t r y o f t h e basis f u n c t i o n s i m p l i e s t h a t t h e first b l o c k
o f t h e e v a l u a t i o n m a t r i x for t h e s y m m e t r i c c o l l o c a t i o n m e t h o d , A ^ , is a g a i n t h e
transpose of the corresponding block i n Kansa's collocation m e t h o d , Ac-
T o see t h i s we consider d i f f e r e n t i a l o p e r a t o r s o f even o r d e r s a n d o d d o r d e r s
separately. I f is a l i n e a r d i f f e r e n t i a l o p e r a t o r o f o d d order, t h e n will introduce
a s i g n change since i t is a c t i n g o n ip as a f u n c t i o n o f t h e second v a r i a b l e . I n a d d i t i o n ,

odd order derivatives (evaluated at x = x ) 3 include a factor of the f o r m
Now, t r a n s p o s i t i o n o f t h i s f a c t o r w i l l a g a i n l e a d t o a s i g n change. T h e c o m b i n a t i o n
of these t w o effects ensures t h a t = A^. F o r even o r d e r s t h e effects o f t h e
operators C and are i n d i s t i n g u i s h a b l e a n d t h e l i n e a r f a c t o r is n o t present.
T h e r e f o r e , u s i n g s y m m e t r i c R B F c o l l o c a t i o n we o b t a i n t h e a p p r o x i m a t e s o l u t i o n
of t h e b o u n d a r y v a l u e p r o b l e m o n t h e c o l l o c a t i o n g r i d as
T -l
Acca Ac 7"
_A _ Ace A_ .9.
W e emphasize t h a t t h i s is not the solution of a pseudospectral m e t h o d b u i l t o n

t h e same f u n c t i o n space (same basis f u n c t i o n s a n d same c o l l o c a t i o n p o i n t s ) as t h e
symmetric R B F collocation method.
For a pseudospectral m e t h o d we w o u l d require the discretized differential oper-
a t o r . F o r m a l l y ( a s s u m i n g i n v e r t i b i l i t y o f K a n s a ' s m a t r i x ) w e w o u l d have
Acc* Ac 'Ac
(42.15)
Ae c A_ A
w h e r e we a l r e a d y i n c o r p o r a t e d t h e b o u n d a r y c o n d i t i o n s i n a w a y a n a l o g o u s t o o u r
earlier discussion.
T h e p r o b l e m w i t h t h e d i f f e r e n t i a t i o n m a t r i x (42.15) for t h e s y m m e t r i c pseu-
d o s p e c t r a l a p p r o a c h is t h a t we c a n n o t be assured t h a t t h e m e t h o d itself, i.e., the
d i s c r e t i z e d d i f f e r e n t i a l o p e r a t o r , is w e l l - d e f i n e d . I n fact, d u e t o t h e Hon-Schaback
c o u n t e r e x a m p l e [ H o n a n d Schaback (2001)] w e k n o w t h a t t h e r e e x i s t g r i d c o n f i g u -
r a t i o n s for w h i c h t h e "basis" used for t h e s y m m e t r i c P S e x p a n s i o n is n o t l i n e a r l y
independent.
T h e r e f o r e , t h e s y m m e t r i c R B F c o l l o c a t i o n a p p r o a c h is w e l l - s u i t e d for p r o b l e m s
t h a t r e q u i r e i n v e r s i o n o f t h e d i f f e r e n t i a l o p e r a t o r (such as e l l i p t i c P D E s ) . Subse-
q u e n t e v a l u a t i o n o n a g r i d m a k e s t h e s y m m e t r i c c o l l o c a t i o n look like a pseudospec-
t r a l m e t h o d b u t i t m a y n o t be one (since we m a y n o t b e able t o f o r m u l a t e t h e
pseudospectral Ansatz).
42.5 A Unified Discussion
I n b o t h the s y m m e t r i c a n d n o n - s y m m e t r i c c o l l o c a t i o n approaches we can t h i n k o f

t h e a p p r o x i m a t e s o l u t i o n as a l i n e a r c o m b i n a t i o n o f a p p r o p r i a t e basis f u n c t i o n s . I n
v e c t o r i z e d n o t a t i o n t h i s c a n be w r i t t e n as
T
u(x) = p (x)c, (42.16)
w h e r e t h e v e c t o r p(x) c o n t a i n s t h e values o f t h e basis f u n c t i o n s a t a;. I f we c o n s i d e r

t h e n o n - s y m m e t r i c m e t h o d these basis f u n c t i o n s are j u s t j = 1 , . . . , N, while
for t h e s y m m e t r i c m e t h o d t h e y are c o m p r i s e d o f b o t h f u n c t i o n s o f t h e t y p e &j a n d

4$ (c.f. (42.14)).
W e n o w l e t T> d e n o t e t h e l i n e a r o p e r a t o r t h a t c o m b i n e s b o t h t h e d i f f e r e n t i a l
o p e r a t o r C a n d t h e b o u n d a r y o p e r a t o r (for D i r i c h l e t b o u n d a r y c o n d i t i o n s t h e l a t t e r
is j u s t t h e i d e n t i t y ) . T h e n we have
T T
Vu(x) = Vp (x)c = q (x)c (42.17)
for a n a p p r o p r i a t e l y defined v e c t o r q(x). W i t h this notation the boundary value

p r o b l e m for o u r a p p r o x i m a t e s o l u t i o n is g i v e n b y
Vu(x) = /(X),
w h e r e / is a piecewise d e f i n e d f u n c t i o n t h a t collects t h e f o r c i n g f u n c t i o n s i n b o t h
the interior and on the boundary.
N o w we e v a l u a t e t h e t w o r e p r e s e n t a t i o n s (42.16) a n d (42.17) o n t h e g r i d o f
c o l l o c a t i o n p o i n t s X i , i = 1 , . . . , N, and obtain
u = Pc and UD = Qc
T
w i t h m a t r i c e s P a n d Q whose r o w s c o r r e s p o n d t o e v a l u a t i o n o f t h e v e c t o r s p (x)
T
a n d q (x), respectively, a t t h e c o l l o c a t i o n p o i n t s X i , i = 1 , . . . , N. T h e discretized
b o u n d a r y v a l u e p r o b l e m is t h e n
uv = f Qc = f, (42.18)
w h e r e / is t h e v e c t o r o f values o f / o n t h e g r i d .
F o r t h e n o n - s y m m e t r i c c o l l o c a t i o n a p p r o a c h t h e e v a l u a t i o n m a t r i x P is t h e s t a n -
d a r d R B F i n t e r p o l a t i o n m a t r i x , a n d t h e d e r i v a t i v e m a t r i x Q is K a n s a ' s m a t r i x ,
whereas for s y m m e t r i c c o l l o c a t i o n P is g i v e n b y t h e t r a n s p o s e o f K a n s a ' s m a t r i x ,
a n d Q is t h e s y m m e t r i c c o l l o c a t i o n m a t r i x .
I t is o u r g o a l t o f i n d t h e v e c t o r u, i.e., t h e values o f t h e a p p r o x i m a t e s o l u t i o n
o n t h e g r i d o f c o l l o c a t i o n p o i n t s . T h e r e are t w o w a y s b y w h i c h we c a n p o t e n t i a l l y
o b t a i n t h i s answer:
(1) W e solve Qc = f for c, i.e.,

1
c = Q- f.
T h e n we use t h e d i s c r e t i z e d v e r s i o n o f (42.16) t o get t h e d e s i r e d v e c t o r u as
(2) A l t e r n a t i v e l y , we first f o r m a l l y t r a n s f o r m t h e coefficients, i.e., w e r e w r i t e u =

Pc as
1
c = P ' u .
T h e n t h e d i s c r e t i z e d b o u n d a r y v a l u e p r o b l e m (42.18) becomes
x
QP~ u = /,
a n d we c a n o b t a i n t h e s o l u t i o n v e c t o r u b y s o l v i n g t h i s s y s t e m .
T h e first a p p r o a c h corresponds t o R B F c o l l o c a t i o n , t h e second t o t h e pseu-

d o s p e c t r a l a p p r o a c h . B o t h o f these a p p r o a c h e s are e q u i v a l e n t as l o n g as a l l o f t h e
m a t r i c e s i n v o l v e d are i n v e r t i b l e . U n f o r t u n a t e l y , as m e n t i o n e d earlier, t h e r e are c o n -
figurations o f c o l l o c a t i o n p o i n t s for w h i c h K a n s a ' s m a t r i x is n o t i n v e r t i b l e . This
means t h a t for t h e n o n - s y m m e t r i c case ( w h e n Q is K a n s a ' s m a t r i x ) A p p r o a c h 1
c a n n o t be assured t o w o r k i n general, a n d A p p r o a c h 2 c a n o n l y be used i f t h e
d i s c r e t i z e d d i f f e r e n t i a l o p e r a t o r is a p p l i e d d i r e c t l y ( b u t n o t i n v e r t e d ) . F o r t h e s y m -
m e t r i c a p p r o a c h ( w h e n P is K a n s a ' s m a t r i x ) , o n t h e o t h e r h a n d , A p p r o a c h 1 is
g u a r a n t e e d t o w o r k i n general, b u t A p p r o a c h 2 m a y n o t be w e l l - d e f i n e d .
42.6 Summary
Our discussion above revealed t h a t for t h e n o n - s y m m e t r i c ( K a n s a ) Ansatz (42.10)

we c a n a l w a y s f o r m u l a t e t h e discrete d i f f e r e n t i a l o p e r a t o r
H o w e v e r , we c a n n o t ensure i n g e n e r a l t h e i n v e r t i b i l i t y o f Lr- T h i s implies t h a t the

n o n - s y m m e t r i c R B F p s e u d o s p e c t r a l a p p r o a c h is j u s t i f i e d for t i m e - d e p e n d e n t P D E s
( w i t h explicit time-stepping methods).
For t h e s y m m e t r i c Ansatz ( 4 2 . 1 2 ) , o n t h e o t h e r h a n d , we c a n i n g e n e r a l ensure
t h e s o l u t i o n o f Cu = / b y R B F c o l l o c a t i o n . H o w e v e r , i t is n o t possible i n g e n e r a l
t o even f o r m u l a t e t h e discrete d i f f e r e n t i a l o p e r a t o r
A c
C A c
L r = [Acs i T
]
As c A
needed for t h e p s e u d o s p e c t r a l a p p r o a c h . T h i s suggests t h a t w e s h o u l d use the

s y m m e t r i c a p p r o a c h for t i m e - i n d e p e n d e n t P D E s a n d p o s s i b l y for t i m e - d e p e n d e n t
PDEs w i t h implicit time-stepping.
T h e difficulties w i t h b o t h a p p r o a c h e s c a n be a t t r i b u t e d t o t h e possible s i n g u -
l a r i t y o f K a n s a ' s m a t r i x w h i c h a p p e a r s as d i s c r e t i z e d d i f f e r e n t i a l o p e r a t o r for t h e
n o n - s y m m e t r i c a p p r o a c h , a n d ( v i a i t s t r a n s p o s e ) as t h e e v a l u a t i o n m a t r i x i n t h e
symmetric approach.
Since t h e n o n - s y m m e t r i c a p p r o a c h is q u i t e a b i t easier t o i m p l e m e n t t h a n t h e
s y m m e t r i c a p p r o a c h , a n d since t h e g r i d c o n f i g u r a t i o n s for w h i c h t h e K a n s a m a t r i x
is s i n g u l a r seem t o be v e r y r a r e (see [ H o n a n d Schaback (2001)]) m a n y researchers
( i n c l u d i n g ourselves) o f t e n prefer t o use t h e n o n - s y m m e t r i c a p p r o a c h even u n d e r
q u e s t i o n a b l e c i r c u m s t a n c e s (such as w i t h i m p l i c i t t i m e - s t e p p i n g p r o c e d u r e s , o r for
e l l i p t i c p r o b l e m s ) . T h e c o n n e c t i o n t o p o l y n o m i a l s i n t h e l i m i t i n g case s = 0 j u s t i f i e s
t h i s p o i n t o f v i e w a t least for 1-D p r o b l e m s .
A n i n t e r e s t i n g q u e s t i o n for f u t u r e research is t h e s t u d y o f R B F - p s e u d o s p e c t r a l
m e t h o d s w i t h m o v i n g o r a d a p t i v e g r i d s . T h i s w i l l be c o m p u t a t i o n a l l y m u c h m o r e
i n v o l v e d t h a n t h e case discussed here ( a n d i l l u s t r a t e d i n t h e n e x t c h a p t e r ) , b u t

t h e use o f R B F s s h o u l d i m p l y t h a t t h e r e is n o m a j o r r e s t r i c t i o n i m p o s e d b y t h e
use o f m o v i n g ( s c a t t e r e d ) c o l l o c a t i o n g r i d s . I n p a r t i c u l a r , w i t h R B F - P S m e t h o d s
one w i l l n o longer be r e s t r i c t e d t o a t e n s o r - p r o d u c t s t r u c t u r e as w i t h t r a d i t i o n a l
p o l y n o m i a l p s e u d o s p e c t r a l m e t h o d s , i.e., w i t h R B F e x p a n s i o n s we s h o u l d be able
t o t a k e advantage o f s c a t t e r e d m u l t i v a r i a t e g r i d s as w e l l as s p a t i a l d o m a i n s w i t h
n o n - r e c t a n g u l a r geometries.
X 4
Chapter 43
R B F - P S Methods in MATLAB
Overall, the c o u p l i n g of R B F collocation a n d pseudospectral methods discussed

i n t h e p r e v i o u s c h a p t e r has p r o v i d e d a n u m b e r o f n e w i n s i g h t s . For e x a m p l e , i t
s h o u l d n o w be clear t h a t we c a n a p p l y m a n y s t a n d a r d p s e u d o s p e c t r a l procedures
t o R B F solvers. I n p a r t i c u l a r , we n o w have " s t a n d a r d " p r o c e d u r e s for s o l v i n g t i m e -
dependent P D E s w i t h R B F s .
I n t h i s c h a p t e r w e i l l u s t r a t e h o w t h e R B F p s e u d o s p e c t r a l a p p r o a c h c a n be ap-
plied i n a w a y very similar t o s t a n d a r d p o l y n o m i a l pseudospectral methods. A m o n g
our n u m e r i c a l i l l u s t r a t i o n s are several examples t a k e n f r o m t h e b o o k [ T r e f e t h e n
(2000)] (see P r o g r a m s 17, 35 a n d 36 t h e r e ) . W e w i l l also use t h e I D t r a n s p o r t equa-
t i o n o f E x a m p l e 42.1 t o c o m p a r e t h e R B F a n d p o l y n o m i a l p s e u d o s p e c t r a l m e t h o d s .
43.1 C o m p u t i n g the R B F - D i f f e r e n t i a t i o nM a t r i x in M A T L A B
W e b e g i n b y e x p l a i n i n g h o w t o c o m p u t e t h e d i s c r e t i z e d d i f f e r e n t i a l o p e r a t o r s (dif-
f e r e n t i a t i o n m a t r i c e s ) t h a t came u p i n o u r discussion i n t h e p r e v i o u s c h a p t e r .
I n o r d e r t o c o m p u t e , for e x a m p l e , a f i r s t - o r d e r d i f f e r e n t i a t i o n m a t r i x we need
t o r e m e m b e r t h a t b y t h e c h a i n r u l e t h e d e r i v a t i v e o f a n R B F w i l l be o f t h e
general f o r m
T h u s , we r e q u i r e n o t o n l y t h e distances, r, b u t also differences i n x, w h e r e x is t h e

first c o m p o n e n t o f x. T h e r e f o r e , t h e m a i n s t a t e m e n t s i n o u r first M A T L A B s u b r o u -
t i n e ( l i s t e d as P r o g r a m 43.1) are t h e c o m p u t a t i o n o f these d i s t a n c e a n d difference
m a t r i c e s o n lines 5 a n d 6. A c c o r d i n g t o t h e discussion i n t h e p r e v i o u s c h a p t e r ,
_ 1
t h e d i f f e r e n t i a t i o n m a t r i x is t h e n g i v e n b y D = A A X . T h i s is i m p l e m e n t e d o n
lines 9 - 1 1 . N o t e t h e use o f t h e m a t r i x r i g h t d i v i s i o n o p e r a t o r / or m r d i v i d e i n
M A T L A B o n line 11 used t o solve t h e s y s t e m DA = A x for D.
P r o g r a m 43.1 is a c t u a l l y a l i t t l e m o r e c o m p l i c a t e d t h a n i t needs t o be since
we i n c l u d e d a n o p t i m i z a t i o n o f t h e R B F shape p a r a m e t e r v i a leave-one-out cross
v a l i d a t i o n as described i n C h a p t e r 17 (see lines 4,7 a n d 8 ) . H e r e we use a m o d -
401
i f i c a t i o n o f t h e basic r o u t i n e C o s t E p s i l o n w h i c h w e c a l l C o s t E p s i l o n D R B F (see
P r o g r a m 43.2 b e l o w ) so t h a t w e o p t i m i z e t h e choice o f e f o r t h e m a t r i x p r o b l e m
l T T T
D = A A~ x ^ AD = (A ) .
X
P r o g r a m 4 3 . 1 . DRBF.m
'/. C D , x ] = D R B F ( N , r b f , d x r b f )
% Computes t h e differentiation matrix D for 1-D derivative
% u s i n g Chebyshev p o i n t s a n d L00CV f o r o p t i m a l shape parameter
% I n p u t : N, create N+l c o l l o c a t i o n points
% rbf, dxrbf function handles f o r rbf and i t s derivative
% Calls on: DistanceMatrix, DifferenceMatrix
'/, R e q u i r e s : CostEpsilonDRBF
1 function [D,x] = DRBF(N,rbf,dxrbf)
2 i f N==0, D=0; x = l ; r e t u r n , end
3 x = cos(pi*(0:N)/N)'; / C h e b y s h e v points
4 m i n e = . 1 ; maxe = 1 0 ; % Shape p a r a m e t e r interval
5 r = DistanceMatrix(x,x);
6 dx = D i f f e r e n c e M a t r i x ( x , x ) ;
7 ep = f m i n b n d ( @ ( e p ) CostEpsilonDRBF(ep,r,dx,rbf,dxrbf),mine,maxe);
8 f p r i n t f ( ' U s i n g e p s i l o n = / f \ n ' , 0 ep)
9 A = rbf(ep,r);
10 Ax = d x r b f ( e p , r , d x ) ;
11 D = Ax/A;
Note that C o s t E p s i l o n D R B F . m is v e r y s i m i l a r t o C o s t E p s i l o n . m (c.f. Pro-

g r a m 17.3). N o w , however, w e c o m p u t e a r i g h t - h a n d side matrix corresponding
t o the transpose of A . x Therefore, t h e d e n o m i n a t o r w h i c h remains the same
for a l l r i g h t - h a n d sides (see f o r m u l a ( 1 7 . 1 ) ) needs t o be c l o n e d o n l i n e 6 v i a t h e
r e p m a t c o m m a n d . T h e cost o f e is n o w t h e F r o b e n i u s n o r m o f t h e m a t r i x EF. O t h e r
measures for t h e e r r o r m a y also be a p p r o p r i a t e . F o r t h e s t a n d a r d i n t e r p o l a t i o n set-
t i n g R i p p a c o m p a r e d use o f t h e \ a n d 2 n o r m s (see [ R i p p a ( 1 9 9 9 ) ] ) a n d c o n c l u d e d
t h a t t h e \ n o r m y i e l d s m o r e a c c u r a t e " o p t i m a " . F o r t h e R B F - P S p r o b l e m s t o be
p r e s e n t e d here we have o b s e r v e d v e r y g o o d r e s u l t s w i t h t h e 2 (or F r o b e n i u s ) n o r m ,
a n d therefore t h a t is w h a t is u s e d o n l i n e 7 o f P r o g r a m 43.2.
P r o g r a m 43.2. CostEpsilonDRBF.m
7, c e p s = C o s t E p s i l o n D R B F ( e p , r , d x , r b f , d x r b f )
% Provides the "cost of epsilon" f u n c t i o n f o r L00CV o p t i m i z a t i o n
7o o f s h a p e parameter
/ I n p u t :
0 ep, values of shape parameter
% r, dx, Distance and D i f f e r e n c e matrices
% rbf, dxrbf, definition o f r b f and its derivative
1 f u n c t i o n ceps = C o s t E p s i l o n D R B F ( e p , r , d x , r b f , d x r b f )
43. RBF-PS Methods in MATLAB 403
2 N = size(r,2);
3 A = rbf(ep,r); '/, = A~T s i n c e A i s symmetric
4 rhs = dxrbf(ep,r,dx)'; 7. A_x~T
5 invA = p i n v ( A ) ;
6 EF = (invA*rhs)./repmat(diag(invA),1,N);
7 ceps = n o r m ( E F ( : ) ) ;
43.1.1 Solution of a 1-D Transport Equation
We i l l u s t r a t e t h e use o f t h e s u b r o u t i n e DBRF. m b y s o l v i n g a o n e - d i m e n s i o n a l t r a n s -
port equation. Consider
u (x,
t t) + cu (x,x t) = 0, x > 1, t > 0,
u ( - l , t ) = 0,
u(x,0) = f(x),
w i t h the well-known solution
u(x, t) = f(x ct).
I n P r o g r a m 43.3 we i m p l e m e n t a s o l u t i o n o f t h i s p r o b l e m w i t h t h e h e l p o f t h e
d i f f e r e n t i a t i o n m a t r i x f r o m P r o g r a m 43.1 above. N o t e t h a t w e c o u l d use a l m o s t t h e
i d e n t i c a l code t o solve t h i s p r o b l e m w i t h a C h e b y s h e v p s e u d o s p e c t r a l method as
discussed i n [ T r e f e t h e n (2000)]. I n fact, i n F i g u r e 43.1 w e d i s p l a y side-by-side t h e
solutions obtained w i t h Gaussian R B F s and w i t h Chebyshev polynomials. Both
s o l u t i o n s were c o m p u t e d o n a g r i d o f 21 C h e b y s h e v p o i n t s . T h e cross-validation
a l g o r i t h m r e t u r n e d a value o f e = 1.874049 for t h e G a u s s i a n . T h e m a x i m u m error
for t h e Gaussian s o l u t i o n a t t i m e t = 1 was 0.0416, w h i l e for t h e PS s o l u t i o n w e
get 0.0418. T h e o n l y difference i n t h e PS-code is t h e r e p l a c e m e n t o f line 4 i n
P r o g r a m 43.3 b y
4 [D,x] = cheb(N)
w h e r e cheb .m is t h e s u b r o u t i n e p r o v i d e d o n page 54 o f [ T r e f e t h e n (2000)] for s p e c t r a l

differentiation.
P r o g r a m 43.3. TransportDRBF. m
7. TransportDRBF
7. S c r i p t t h a t s o l v e s c o n s t a n t c o e f f i c i e n t wave e q u a t i o n
7o u _ t + c*u_x = 0, u s i n g RBF-PS approach
7, C a l l s on: DRBF
1 r b f = @(e,r) e x p ( - ( e * r ) . ~2) ; 7. G a u s s i a n RBF
2 dxrbf = @(e,r,dx) - 2 * d x * e ~ 2 . * e x p ( - ( e * r ) . ~ 2 ) ;
3 N = 20;
4 [D,x] = DRBF(N,rbf,dxrbf);
5 x = f l i p u d ( x ) ; dt = 0.001; t = 0; c = -1;
6 v = 64*(-x).~3.*(l+x)."3;
7 v(find(x>0)) = zeros(length(find(x>0)),1);
% Time-stepping by e x p l i c i t E u l e r formula:
8 tmax = 1; t p l o t = .02; p l o t g a p = r o u n d ( t p l o t / d t ) ;
9 dt = t p l o t / p l o t g a p ; n p l o t s = r o u n d ( t m a x / t p l o t ) ;
5
10 data = [ v ; z e r o s ( n p l o t s , N + l ) ] ; t d a t a = t ;
11 I = e y e ( s i z e ( D ) ) ;
12 f o r i = l : n p l o t s
13 for n = l:plotgap
14 t = t+dt;
15 vv = v(end-1);
16 v = v - dt*c*(D*v); / e x p l i c i t E u l e r
17 v ( l ) = 0; v(end) = vv;
18 end
19 d a t a ( i + l , : ) = v'; t d a t a = [ t d a t a ; t ] ;
20 end
21 s u r f ( x , t d a t a , d a t a ) , view(10,70), colormap('default');
22 a x i s ( [ - l 1 0 tmax 0 1 ] ) , y l a b e l t , z l a b e l u, g r i d o f f
% exact s o l u t i o n and e r r o r
23 xx = l i n s p a c e ( - l , 1 , 1 0 1 ) ;
24 vone = 64*(1-xx)."3.*xx."3;
25 v o n e ( f i n d ( x x < 0 ) ) = z e r o s ( l e n g t h ( f i n d ( x x < 0 ) ) , 1 ) ;
26 w = i n t e r p l ( x , v , x x ) ;
27 maxErr = norm(w-vone,inf)
T h e g r a p h i n F i g u r e 43.1 shows t h e t i m e p r o f i l e o f t h e s o l u t i o n s for t h e t i m e

3 3
i n t e r v a l [0,1] w i t h i n i t i a l p r o f i l e f(x) = 64(1 x) x a n d u n i t w a v e speed.
J
0.5- J
0.5
Fig. 43.1 Solution to transport equation based on Gaussian R B F s with e = 1.874049 (left) and
Chebyshev P S method (right). Explicit Euler time-stepping with (At = 0.001), and 21 Chebyshev
points.
43.2 U s e of the C o n t o u r - P a d e A l g o r i t h m w i t h the P S A p p r o a c h
We now give a brief explanation of h o w the Contour-Pade a l g o r i t h m of [Fornberg

a n d W r i g h t (2004)] c a n be used t o c o m p u t e R B F d i f f e r e n t i a t i o n m a t r i c e s . I n its
o r i g i n a l f o r m t h e C o n t o u r - P a d e a l g o r i t h m a l l o w s us t o s t a b l y e v a l u a t e r a d i a l basis
f u n c t i o n i n t e r p o l a n t s based o n i n f i n i t e l y s m o o t h R B F s for e x t r e m e choices o f t h e
shape p a r a m e t e r e ( i n p a r t i c u l a r e 0 ) . M o r e specifically, t h e Contour-Pade
a l g o r i t h m uses F F T s a n d P a d e a p p r o x i m a t i o n s t o e v a l u a t e t h e f u n c t i o n
T 1
u(x,e) = 6 (a;,e)(A())- / (43.1)
w i t h b(x,s)j = ip (\\x
e Xj\\) a t some e v a l u a t i o n p o i n t x a n d A(e)ij ip (\\xi
e Xj\\)
(c.f. t h e discussion i n t h e p r e v i o u s c h a p t e r a n d i n C h a p t e r 1 7 ) . T h e p a r a m e t e r e
is used t o d e n o t e t h e dependence o f b a n d A o n t h e choice o f t h a t p a r a m e t e r as a
s c a l i n g factor i n t h e basic f u n c t i o n ip = <p(e-).

I f we evaluate u at a l l o f t h e c o l l o c a t i o n p o i n t s X i , i = 1 , . . . , N, for some f i x e d

T
value o f e, t h e n b (x, e) t u r n s i n t o t h e m a t r i x A(e). I n t h e case o f i n t e r p o l a t i o n t h i s
exercise is, o f course, p o i n t l e s s . H o w e v e r , i f t h e C o n t o u r - P a d e a l g o r i t h m is a d a p t e d
T
to replace t h e v e c t o r b (x,e) ( c o r r e s p o n d i n g t o e v a l u a t i o n a t a single p o i n t x)
w i t h t h e m a t r i x Ac based o n t h e d i f f e r e n t i a l o p e r a t o r used earlier ( c o r r e s p o n d i n g
to evaluation at a l l collocation p o i n t s ) , t h e n
1
Ac(e)(A(e))- u
c o m p u t e s t h e values o f t h e ( s p a t i a l ) d e r i v a t i v e o f u o n t h e c o l l o c a t i o n p o i n t s Xi.
B o u n d a r y c o n d i t i o n s can t h e n be i n c o r p o r a t e d l a t e r as i n t h e s t a n d a r d pseudospec-
t r a l a p p r o a c h (see, e.g., [ T r e f e t h e n (2000)] or o u r discussion i n S e c t i o n 4 2 . 2 ) .
T h i s means t h a t we are able t o s u p p l y y e t a n o t h e r s u b r o u t i n e t o c o m p u t e t h e
d i f f e r e n t i a t i o n m a t r i x o n l i n e 4 o f P r o g r a m 43.3 v i a t h e C o n t o u r - P a d e a l g o r i t h m .
43.2.1 Solution of the ID Transport Equation Revisited
We use t h e same e x a m p l e as i n S u b s e c t i o n 4 3 . 1 . 1 . I n t h i s s u b s e c t i o n we c o m p a r e
a s o l u t i o n based o n t h e C o n t o u r - P a d e a l g o r i t h m for G a u s s i a n R B F s i n t h e l i m i t i n g
case e > 0 t o t h e t w o m e t h o d s d e s c r i b e d e a r l i e r (based o n DRBF a n d cheb). A l l o f
these approaches use a n i m p l i c i t E u l e r m e t h o d w i t h t i m e s t e p At 0.001 for t h e
t i m e d i s c r e t i z a t i o n . W e p o i n t o u t t h a t for a n i m p l i c i t t i m e - s t e p p i n g m e t h o d b o t h
t h e C o n t o u r - P a d e a p p r o a c h a n d t h e DRBF a p p r o a c h used earlier, o f course, r e q u i r e
an inversion of the differentiation m a t r i x . Recall t h a t our theoretical discussion
suggested t h a t t h i s is j u s t i f i e d as l o n g as w e confine ourselves t o t h e l i m i t i n g case
s > 0 a n d one space d i m e n s i o n . W e w i l l see t h a t t h e n o n - l i m i t i n g case ( u s i n g DRBF)
seems t o w o r k j u s t as w e l l .
I n F i g u r e s 43.2 a n d 43.3 we p l o t t h e m a x i m u m e r r o r s a t t i m e t = 1 for a
t i m e step At = 0.001) a n d s p a t i a l d i s c r e t i z a t i o n s c o n s i s t i n g o f Af + 1 = 7 , . . . , 19
c o l l o c a t i o n p o i n t s . E r r o r s for t h e C o n t o u r - P a d e G a u s s i a n R B F s o l u t i o n are o n t h e
left o f F i g u r e 43.2 a n d for t h e C h e b y s h e v PS s o l u t i o n o n t h e r i g h t . The errors

for t h e G a u s s i a n R B F s o l u t i o n w i t h A f - d e p e n d e n t " o p t i m a l " shape p a r a m e t e r are
s h o w n i n t h e left p a r t o f F i g u r e 43.3, w h i l e t h e c o r r e s p o n d i n g " o p t i m a l " e-values
are d i s p l a y e d i n t h e r i g h t p l o t . T h e y r a n g e a l m o s t l i n e a r l y i n c r e a s i n g f r o m 0 . 1 2 2 6 6 1
a t N = 6 t o 1.566594 a t N = 18.
W e c a n see t h a t t h e e r r o r s f o r a l l t h r e e m e t h o d s are v i r t u a l l y i d e n t i c a l . U n f o r t u -
n a t e l y , i n t h i s e x p e r i m e n t w e are l i m i t e d t o t h i s s m a l l r a n g e o f N since for N > 19
t h e C o n t o u r - P a d e s o l u t i o n becomes u n r e l i a b l e . H o w e v e r , t h e a g r e e m e n t o f a l l t h r e e
s o l u t i o n s for these s m a l l values o f N is r e m a r k a b l e . I n f a c t , t h i s seems t o i n d i c a t e
t h a t t h e e r r o r s i n t h e s o l u t i o n are m o s t l y d u e t o t h e t i m e - s t e p p i n g m e t h o d used.
6 8 10 12 14 16 18 6 8 10 12 14 16 18
N N
Fig. 43.2 Errors at t = 1 for transport equation. Gaussian R B F with e = 0 (left) and Chebyshev
PS-solution (right); variable spatial discretization N. Implicit Euler method with A t = 0.001.
6 8 10 12 14 16 18 6 8 10 12 14 16 18
N N
Fig. 43.3 Errors at t = 1 for transport equation using Gaussian R B F with "optimal" e (left)
and corresponding e-values (right); variable spatial discretization N. Implicit Euler method with
A t = 0.001.
T h e s p e c t r a o f t h e d i f f e r e n t i a t i o n m a t r i c e s for b o t h t h e G a u s s i a n Contour-Pade
a n d t h e C h e b y s h e v P S a p p r o a c h e s are p l o t t e d i n F i g u r e s 43.4 a n d 4 3 . 5 , r e s p e c t i v e l y .
T h e s u b p l o t s c o r r e s p o n d t o t h e use o f AT -f- 1 = 5, 9 , 1 3 , 1 7 C h e b y s h e v c o l l o c a t i o n
p o i n t s for t h e s p a t i a l d i s c r e t i z a t i o n . T h e p l o t s for t h e G a u s s i a n a n d Chebyshev
m e t h o d s s h o w some s i m i l a r i t i e s , b u t also some differences. T h e g e n e r a l d i s t r i b u t i o n
o f t h e eigenvalues for t h e t w o m e t h o d s is q u i t e s i m i l a r . H o w e v e r , t h e s p e c t r a for
t h e C o n t o u r - P a d e a l g o r i t h m w i t h G a u s s i a n R B F s seem t o be m o r e or less a s l i g h t l y
s t r e t c h e d r e f l e c t i o n a b o u t t h e i m a g i n a r y axis o f t h e s p e c t r a o f t h e C h e b y s h e v pseu-
d o s p e c t r a l m e t h o d . T h e differences increase as N increases. T h i s , however, is n o t
s u r p r i s i n g since t h e C o n t o u r - P a d e a l g o r i t h m is k n o w n t o be u n r e l i a b l e for l a r g e r
values o f N.
x 10"
0.2
0.15
0.1
0.05
1 o E 0
-0.05
-1
-0.1
-0.15
-2 0 -0.1 0 0.1 0.2

Re Re
x 10
M o E 0
-2
-1
-4
-6
-2
-8
-2 10
Re Re
Fig. 43.4 Spectra of differentiation matrices for Gaussian R B F with e = 0 on Chebyshev collo-
cation points obtained with the Contour-Pade algorithm and N = 5, 9,13,17.
43.3 Computation of Higher-Order Derivatives
A r a t h e r nice feature o f p o l y n o m i a l d i f f e r e n t i a t i o n m a t r i c e s is t h e fact t h a t h i g h e r -

o r d e r d e r i v a t i v e s c a n be c o m p u t e d b y r e p e a t e d l y a p p l y i n g t h e f i r s t - o r d e r differen-
k
t i a t i o n m a t r i x , i.e., = D, w h e r e D is t h e s t a n d a r d f i r s t - o r d e r d i f f e r e n t i a t i o n
matrix and is t h e m a t r i x c o r r e s p o n d i n g t o t h e > t h ( u n i v a r i a t e ) d e r i v a t i v e .
U n f o r t u n a t e l y , t h i s nice f e a t u r e does n o t c a r r y over t o t h e g e n e r a l R B F case ( j u s t
Fig. 43.5 Spectra of differentiation matrices for Chebyshev pseudospectral method on Chebyshev
collocation points with N = 5, 9,13, 17.
as is does n o t h o l d for p e r i o d i c F o u r i e r s p e c t r a l d i f f e r e n t i a t i o n m a t r i c e s , either).

W e therefore need t o p r o v i d e separate M A T L A B code for higher-order differentia-
t i o n m a t r i c e s . A s P r o g r a m 43.4 shows, t h i s is n o t f u n d a m e n t a l l y m o r e c o m p l i c a t e d
t h a n t h e f i r s t - o r d e r case. T h e o n l y differences b e t w e e n P r o g r a m s 4 3 . 1 a n d 43.4 are
g i v e n b y t h e c o m p u t a t i o n o f t h e A (k)
D m a t r i x o n l i n e 10 for t h e f i r s t - o r d e r case i n
P r o g r a m 4 3 . 1 a n d lines 9 for t h e s e c o n d - o r d e r case i n P r o g r a m 43.4, a n d b y the
use o f t h e s u b r o u t i n e C o s t E p s i l o n D 2 R B F i n s t e a d o f C o s t E p s i l o n D R B F . T h e s e dif-
ferences are m i n u t e , a n d e s s e n t i a l l y a l l t h a t is n e e d e d is t h e a p p r o p r i a t e f o r m u l a f o r
t h e d e r i v a t i v e o f t h e R B F passed t o D2RBF v i a t h e p a r a m e t e r d 2 r b f . We do not
list t h e f u n c t i o n C o s t E p s i l o n D 2 R B F . I t differs f r o m C o s t E p s i l o n D R B F o n l y i n t h e
d e f i n i t i o n o f t h e r i g h t - h a n d side m a t r i x w h i c h n o w becomes
4 rhs = d2rbf(ep,r)';
Also, the number and type of parameters that are passed t o t h e f u n c t i o n s are
d i f f e r e n t since t h e f i r s t - o r d e r d e r i v a t i v e r e q u i r e s differences of collocation points
a n d the second-order derivative does n o t .
P r o g r a m 43.4. D2RBF.m
% [D2,x] = D2RBF(N,rbf,d2rbf)
% Computes the second-order d i f f e r e n t i a t i o n m a t r i x D2 f o r 1-D
% d e r i v a t i v e u s i n g Chebyshev p o i n t s and L00CV f o r optimal e p s i l o n
% Input: N, number of p o i n t s -1
% r b f , d2rbf, f u n c t i o n handles f o r r b f and i t s d e r i v a t i v e
% R e q u i r e s : CostEpsilonD2RBF
1 f u n c t i o n [D2,x] = D2RBF(N,rbf,d2rbf)
2 i f N==0, D2=0; x = l ; r e t u r n , end
3 x = cos(pi*(0:N)/N) ' ; % Chebyshev p o i n t s
4 mine = .1; maxe = 1 0 ; % Shape parameter i n t e r v a l
5 r = DistanceMatrix(x,x);
6 ep = fminbnd(@(ep) CostEpsilonD2RBF(ep,r,rbf,d2rbf),mine,maxe);
7 f p r i n t f ('Using e p s i l o n = /f\n', ep) 0
8 A = rbf(ep,r);
9 AD2 = d 2 r b f ( e p , r ) ;
10 D2 = AD2/A;
43.3.1 Solution of the Allen-Cahn Equation
T o i l l u s t r a t e t h e use o f t h e s u b r o u t i n e D2RBF. m we present a m o d i f i c a t i o n o f P r o -

g r a m 35 i n [Trefethen (2000)] w h i c h is concerned w i t h t h e s o l u t i o n o f t h e n o n l i n e a r
reaction-diffusion (or A l l e n - C a h n ) e q u a t i o n . T h e specific p r o b l e m we w i l l solve is
of the form
3
u t = Ltu xx + u - u, x e (-1,1), t>0,
w i t h parameter /J, initial condition
7 n E x e _ 1
u(x,Q) = 0.53a; + 0.47sin ^ - ^ ) > [ > !]
a n d non-homogeneous (time-dependent) b o u n d a r y c o n d i t i o n s u(l,t) = 1 a n d

2
u(l,t) = sin (i/5). T h e s o l u t i o n t o t h i s e q u a t i o n has t h r e e steady states (u =
1, 0,1) w i t h t h e t w o nonzero s o l u t i o n s b e i n g stable. T h e t r a n s i t i o n b e t w e e n these
states is governed b y t h e p a r a m e t e r /x. I n o u r c a l c u l a t i o n s b e l o w we use /x = 0.01,
a n d t h e u n s t a b l e s t a t e s h o u l d v a n i s h a r o u n d t 30.
T h e m o d i f i e d M A T L A B code is p r e s e n t e d i n P r o g r a m 43.5. N o t e h o w easily t h e
n o n l i n e a r i t y is d e a l t w i t h b y i n c o r p o r a t i n g i t i n t o t h e t i m e - s t e p p i n g m e t h o d o n
l i n e 13.
P r o g r a m 43.5. M o d i f i c a t i o n o f P r o g r a m 35 o f [Trefethen (2000)]
% p35
% S c r i p t t h a t s o l v e s Allen-Cahn equation w i t h boundary c o n d i t i o n
% imposed explicitly ("method (II)") (from Trefethen (2000))

7, We r e p l a c e t h e C h e b y s h e v m e t h o d b y a n RBF-PS method
7. C a l l s o n : D2RBF
7, M a t e r n c u b i c as RBF b a s i c function
1 r b f = @(e,r) e x p ( - e * r ) . * ( 1 5 + 1 5 * e * r + 6 * ( e * r ) . ~ 2 + ( e * r ) . ~3) ;
2 d2rbf = @(e,r) e~2*((e*r).~3-3*e*r-3).*exp(-e*r);
3 N = 20;
4 [D2,x] = D2RBF(N,rbf,d2rbf);
7o H e r e is the rest of Trefethen's code.
5 mu = 0 . 0 1 ; dt = min([.01,50*N~(-4)/mu]);
6 t = 0; v = .53*x + . 4 7 * s i n ( - l . 5 * p i * x ) ;
7o S o l v e PDE b y E u l e r f o r m u l a and p l o t results:
7 tmax = 100; tplot = 2; nplots = round(tmax/tplot);
8 plotgap = round(tplot/dt); dt = tplot/plotgap;
9 xx = - 1 : . 0 2 5 : 1 ; vv = polyval(polyfit(x,v,N),xx);
10 plotdata = [vv; zeros(nplots,length(xx))]; tdata = t;
11 for i = l:nplots
12 for n = l:plotgap
13 t = t+dt; v = v + dt*(mu*D2*v + v - v. "3); 7. E u l e r
14 v(l) = 1 + sin(t/5)~2; v(end) = - 1 ; 7. BC
15 end
16 vv = polyval(polyfit(x,v,N),xx);
17 p l o t d a t a ( i + l , : ) = vv; tdata = [tdata; t ] ;
18 end
19 surf(xx,tdata,plotdata), grid on
20 axis([-l 1 0 tmax - 1 2 ] ) , view(-40,55)
21 colormap('default'); xlabel x, ylabel t , zlabel u
T h e o r i g i n a l p r o g r a m i n [ T r e f e t h e n ( 2 0 0 0 ) ] is o b t a i n e d b y d e l e t i n g lines 1-2 and

r e p l a c i n g l i n e 4 b y a c a l l t o c h e b . m f o l l o w e d b y t h e s t a t e m e n t D2 = D~2 w h i c h y i e l d s
t h e second-order d i f f e r e n t i a t i o n m a t r i x i n t h e C h e b y s h e v case.
Note that i n our R B F - P S implementation the m a j o r i t y of the m a t r i x computa-
t i o n s are r e q u i r e d o n l y once o u t s i d e t h e t i m e - s t e p p i n g p r o c e d u r e w h e n c o m p u t i n g
t h e d e r i v a t i v e m a t r i x as t h e s o l u t i o n o f a l i n e a r s y s t e m . Inside the time-stepping
l o o p (lines 1 2 - 1 5 ) we r e q u i r e o n l y m a t r i x - v e c t o r m u l t i p l i c a t i o n . W e p o i n t o u t t h a t
t h i s a p p r o a c h is m u c h m o r e efficient t h a n c o m p u t a t i o n o f R B F e x p a n s i o n coeffi-
cients a t e v e r y t i m e s t e p (as s u g g e s t e d , e.g., i n [ H o n a n d M a o ( 1 9 9 9 ) ] ) . I n f a c t , t h i s
is t h e m a i n difference b e t w e e n t h e R B F - P S a p p r o a c h a n d t h e c o l l o c a t i o n a p p r o a c h
o f C h a p t e r s 3 8 - 4 0 (see also o u r c o m p a r i s o n o f t h e c o l l o c a t i o n a p p r o a c h e s a n d the
R B F - P S approach i n the previous chapter).
I n F i g u r e 43.6 we s h o w t h e s o l u t i o n o b t a i n e d v i a t h e C h e b y s h e v p s e u d o s p e c t r a l
m e t h o d a n d v i a a n R B F pseudospectral a p p r o a c h based o n the "cubic" Matern
2 3 r
f u n c t i o n cp(r) = (15 + 15er + 6 ( e r ) - f (er) )e~ w i t h " o p t i m a l " shape p a r a m e t e r
e = 0.350952. N o t e t h a t these c o m p u t a t i o n s are r a t h e r sensitive t o t h e value o f e

a n d t h e n o r m used t o measure t h e "cost" o f e i n C o s t E p s i l o n D 2 R B F . m . I n fact, use
of t h e l\ or norms instead o f the t 2 n o r m b o t h l e a d t o i n a c c e p t a b l e results for
t h i s test p r o b l e m . T h e reasons for t h i s h i g h s e n s i t i v i t y o f t h e s o l u t i o n t o t h e v a l u e
of e are t h e e x t r e m e i l l - c o n d i t i o n i n g o f t h e m a t r i x a l o n g w i t h t h e changes o f t h e
s o l u t i o n over t i m e . A n a d a p t i v e m e t h o d w o u l d m o s t l i k e l y p e r f o r m m u c h b e t t e r i n
t h i s case.
T h e c o m p u t a t i o n s for t h i s e x a m p l e are based o n 2 1 C h e b y s h e v p o i n t s , a n d t h e
d i f f e r e n t i a t i o n m a t r i x for t h e R B F is o b t a i n e d d i r e c t l y w i t h t h e s u b r o u t i n e D2RBF. m
(i.e., w i t h o u t t h e C o n t o u r - P a d e a l g o r i t h m ) . W e use t h i s a p p r o a c h since for 2 1 p o i n t s
t h e C o n t o u r - P a d e a l g o r i t h m n o longer c a n be r e l i e d u p o n . M o r e o v e r , i t is a p p a r e n t
f r o m t h e figures t h a t reasonable s o l u t i o n s c a n also be o b t a i n e d v i a t h i s d i r e c t ( a n d
m u c h s i m p l e r ) R B F a p p r o a c h . T r u e s p e c t r a l accuracy, however, w i l l n o longer be
g i v e n i f e > 0. W e c a n see f r o m t h e figure t h a t t h e s o l u t i o n based o n C h e b y s h e v
p o l y n o m i a l s appears t o be s l i g h t l y m o r e a c c u r a t e since t h e t r a n s i t i o n occurs at a
s l i g h t l y l a t e r a n d correct t i m e (i.e., at t 30) a n d is also a l i t t l e "sharper".
Fig. 43.6 Solution of the Allen-Cahn equation using the Chebyshev pseudospectral method (left)
and an R B F - P S method with cubic Matern functions (right) with N = 20.
43.4 Solution of a 2D Helmholtz Equation
W e consider t h e 2 D H e l m h o l t z e q u a t i o n (see P r o g r a m 17 i n [Trefethen (2000)])

2 2
u x x + u y y + ku = f(x,y), x,ye(-l,l) ,
w i t h boundary condition u = 0 and
2 2
f(x,y)=exp(-10 (y-l) + (x-)
T o solve t h i s t y p e o f ( e l l i p t i c ) p r o b l e m we a g a i n need t o assume i n v e r t i b i l i t y o f

t h e d i f f e r e n t i a t i o n m a t r i x . E v e n t h o u g h t h i s m a y n o t be w a r r a n t e d t h e o r e t i c a l l y
(see o u r discussion i n t h e p r e v i o u s c h a p t e r ) , we c o m p a r e a n o n - s y m m e t r i c RBF

pseudospectral m e t h o d w i t h a Chebyshev pseudospectral m e t h o d .
W e a t t e m p t t o solve t h e p r o b l e m w i t h r a d i a l basis f u n c t i o n s i n t w o different
ways. F i r s t , we a p p l y t h e same t e c h n i q u e as i n [ T r e f e t h e n (2000)] u s i n g t h e k r o n
f u n c t i o n t o express t h e d i s r e t i z e d L a p l a c i a n o n a t e n s o r - p r o d u c t g r i d o f (N + 1) x
(N + 1) p o i n t s as
L = f <g> D2 + D2 <g> f, (43.2)
w h e r e D2 is t h e ( u n i v a r i a t e ) second-order d i f f e r e n t i a t i o n m a t r i x , / is a n i d e n t i t y
m a t r i x o f size (N + 1) x (N + 1 ) , a n d denotes t h e Kronecker tensor-product. For
p o l y n o m i a l PS m e t h o d s t h e second-order d i f f e r e n t i a t i o n m a t r i x c a n be computed
2
as t h e square o f t h e one for t h e f i r s t - o r d e r d e r i v a t i v e , i.e., D2 = D , a n d t h i s is
w h a t is used i n [ T r e f e t h e n ( 2 0 0 0 ) ] .
A s we p o i n t e d o u t earlier, for R B F s w e c a n n o t f o l l o w t h i s a p p r o a c h d i r e c t l y
2 2
since D ^ D^ \ T h u s , w e generate t h e m a t r i x D2 d i r e c t l y w i t h t h e h e l p o f t h e
s u b r o u t i n e D2RBF. H o w e v e r , as l o n g as t h e c o l l o c a t i o n p o i n t s f o r m a t e n s o r - p r o d u c t
g r i d a n d t h e R B F is separable (such as a G a u s s i a n or a p o l y n o m i a l ) , w e c a n s t i l l
employ the Kronecker tensor-product c o n s t r u c t i o n ( 4 3 . 2 ) . T h i s is i m p l e m e n t e d i n
lines 4 a n d 9 o f P r o g r a m 43.6
P r o g r a m 4 3 . 6 . M o d i f i c a t i o n o f P r o g r a m 17 o f [ T r e f e t h e n (2000)]
7. pl7
7. S c r i p t that solves Helmholtz equation
7. u _ x x + u _ y y + ( k " 2 ) u = f on [-1,1]x[-1,1]
7o We r e p l a c e t h e C h e b y s h e v m e t h o d b y a n RBF-PS method
7o a n d e x p l i c i t l y enforce the boundary conditions
7. C a l l s o n : D2RBF
7o G a u s s i a n RBF b a s i c function
1 r b f = @(e,r) exp(-(e*r).~2);
2 d2rbf = @(e,r) 2*e~2*(2*(e*r).~2-l).*exp(-(e*r)."2);
3 N = 24;
4 [D2,x] = D2RBF(N,rbf,d2rbf); y = x;
5 [xx,yy] = meshgrid(x,y);
6 xx = x x ( : ) ; yy = yy(:);
7 1 = eye(N+l);
8 k = 9;
9 L = kron(I,D2) + kron(D2,I) + k~2*eye((N+l)~2);
7o I m p o s e boundary c o n d i t i o n s by r e p l a c i n g appropriate rows o f L
10 b = find(abs(xx)==l I abs(yy)==l); '/ b o u n d a r y pts
11 L(b,:) = zeros(4*N,(N+l)~2); L ( b , b ) = eye(4*N);
12 f = exp(-10*((yy-l).~2+(xx-.5).~2));
13 f(b) = zeros(4*N,1);
7. S o l v e f o r u , reshape t o 2D g r i d , and plot:
il
14 u = L\f;
15 uu = r e s h a p e ( u , N + l , N + l ) ;
16 [xx,yy] = meshgrid(x,y);
17 [xxx.yyy] = meshgrid(-l:.0333:1,-1:.0333:1);
18 uuu = interp2(xx,yy,uu,xxx,yyy,'cubic');
19 figure, elf, surf(xxx,yyy,uuu),
20 xlabel x, ylabel y, zlabel u
21 t e x t ( . 2 , 1 , . 0 2 2 , s p r i n t f ( ' u ( O . O ) = /.13.1 I f ' , u u ( N / 2 + 1 , N / 2 + 1 ) ) )
T h e s o l u t i o n o f t h e H e l m h o l t z e q u a t i o n for k 9 w i t h Gaussians u s i n g a n
" o p t i m a l " shape p a r a m e t e r e = 2.549845 a n d N = 24 (i.e., 625 t o t a l p o i n t s ) is
d i s p l a y e d n e x t t o t h e Chebyshev p s e u d o s p e c t r a l s o l u t i o n o f [Trefethen (2000)] i n
F i g u r e 43.7. A g a i n , t h e s i m i l a r i t y o f t h e t w o s o l u t i o n s is r e m a r k a b l e .
A s an a l t e r n a t i v e a p p r o a c h t h a t allows also t h e use o f non-tensor product

c o l l o c a t i o n g r i d s we m o d i f y P r o g r a m 43.6 a n d use a d i r e c t i m p l e m e n t a t i o n o f t h e
Laplacian of the R B F s . T h e only advantage of doing this o n a tensor-product g r i d
is t h a t n o w a l l r a d i a l basis f u n c t i o n s c a n be used. T h i s v a r i a t i o n o f t h e code takes
considerably longer t o execute since t h e d i f f e r e n t i a t i o n m a t r i x is n o w c o m p u t e d w i t h
m a t r i c e s o f size 625 x 625 i n s t e a d o f t h e 25 x 25 m a t r i c e s used for t h e u n i v a r i a t e
d i f f e r e n t i a t i o n m a t r i x D2 earlier. M o r e o v e r , t h e results are l i k e l y t o be less a c c u r a t e
since t h e larger m a t r i c e s are m o r e p r o n e t o i l l - c o n d i t i o n i n g . H o w e v e r , t h e a d v a n t a g e
o f t h i s a p p r o a c h is t h a t i t frees us o f t h e l i m i t a t i o n o f p o l y n o m i a l PS m e t h o d s t o
tensor-product collocation grids.
6
T h e m o d i f i e d code is l i s t e d i n P r o g r a m 43.7 w h e r e we have used t h e C Wendland
f u n c t i o n ^3,3(r) = ( 1 - er)%(32(er) 3
+ 25(er) 2
+ 8sr + 1) w i t h a n " o p t i m a l " scale
p a r a m e t e r e = 0.129440. N o t e t h a t we used t h e c o m p a c t l y s u p p o r t e d W e n d l a n d
functions i n " g l o b a l m o d e " ( w i t h s m a l l e, i.e., large s u p p o r t size) a n d t h i s e x p l a i n s
t h e d e f i n i t i o n o f t h e basic f u n c t i o n as i n lines 1 a n d 2 o f P r o g r a m 43.7 i n p r e p a r a t i o n
for the use with the dense code D i s t a n c e M a t r i x .m in the subroutine LRBF .m (which
is listed below as Program 43.8). The output of Program 43.7 is displayed in
Figure 43.8.
P r o g r a m 4 3 . 7 . Modification I I of Program 17 of [Trefethen (2000)]
7. pl7_2D
7o S c r i p t t h a t s o l v e s Helmholtz equation
7. u_xx + u_yy + ( k ~ 2 ) u = f on [-1,1]x[-1,1]
7o We r e p l a c e t h e Chebyshev method by an RBF-PS method,
7o e x p l i c i t l y e n f o r c e t h e boundary c o n d i t i o n s , and
7o use a 2-D implementation of t h e L a p l a c i a n
7. C a l l s on: LRBF
% Wendland C6 RBF b a s i c f u n c t i o n
1 r b f = @(e,r) m a x ( l - e * r , 0 ) . ~ 8 . * ( 3 2 * ( e * r ) . ~ 3 + 2 5 * ( e * r ) . ~ 2 + 8 * e * r + l ) ;
2a L r b f = @(e,r) 44*e"2*max(l-e*r,0)."6.*...
2b (88*(e*r).~3+3*(e*r).~2-6*e*r-l);
3 [L,x,y] = L R B F ( N , r b f , L r b f ) ;
4 [xx,yy] = m e s h g r i d ( x , y ) ;
5 xx = x x ( : ) ; yy = y y ( : ) ;
6 k = 9;
7 L = L + k~2*eye((N+l)~2);
7o Impose boundary c o n d i t i o n s by r e p l a c i n g a p p r o p r i a t e rows of L
8 b = find(abs(xx)==l I abs(yy)==l); 7, boundary p t s
9 L ( b , : ) = z e r o s ( 4 * N , ( N + l ) ~ 2 ) ; L(b,b) = eye(4*N);
10 f = e x p ( - 1 0 * ( ( y y - l ) . ~ 2 + ( x x - . 5 ) . ~ 2 ) ) ;
11 f ( b ) = z e r o s ( 4 * N , 1 ) ;
7o Solve f o r u, reshape t o 2D g r i d , and p l o t :
12 u = L \ f ;
13 uu = reshape(u,N+l,N+l);
14 [xx,yy] = m e s h g r i d ( x , y ) ;
15 [xxx,yyy] = meshgrid(-l:.0333:1,-1:.0333:1);
16 uuu = i n t e r p 2 ( x x , y y , u u , x x x , y y y c u b i c ' ) ;
17 f i g u r e , e l f , s u r f ( x x x , y y y , u u u ) ,
18 x l a b e l x, y l a b e l y, z l a b e l u
19 t e x t ( . 2 , 1 , . 0 2 2 , s p r i n t f O u ( 0 , 0 ) = 7.13. l l f ' ,uu (N/2+1, N/2+1) ) )
P r o g r a m 4 3 . 8 . LRBF.m
7. [L,x,y] = LRBF(N,rbf ,Lrbf )
7o Computes t h e L a p l a c i a n d i f f e r e n t i a t i o n m a t r i x L f o r 2-D
7o d e r i v a t i v e s u s i n g Chebyshev p o i n t s and L00CV f o r optimal e p s i l o n
% Input: N number of p o i n t s -1
7o r b f , L r b f , f u n c t i o n handles f o r r b f and i t s d e r i v a t i v e
% Requires: CostEpsilonLRBF
1 f u n c t i o n [L,x,y] = LRBF(N,rbf,Lrbf)
2 i f N==0, L=0; x=l; r e t u r n , end
3 x = cos(pi*(0:N)/N)'; % Chebyshev p o i n t s
4 y = x; [xx.yy] = meshgrid(x,y);
% S t r e t c h 2D g r i d s t o ID v e c t o r s and put i n one a r r a y
5 points = [xx(:) y y ( : ) ] ;
6 mine = .1; maxe = 10; % Shape parameter i n t e r v a l
7 r = DistanceMatrix(points.points);
8 ep = fminbnd(@(ep) CostEpsilonLRBF(ep,r,rbf,Lrbf),mine,maxe);
9 f p r i n t f ('Using e p s i l o n = 7,f \n', ep)
10 A = r b f ( e p , r ) ;
11 AL = L r b f ( e p , r ) ;
12 L = AL/A;
Fig. 43.8 Solution of the 2D Helmholtz equation using a direct implementation of the Laplacian
6
based on C Wendland functions with e = 0.129440 on 625 tensor-product Chebyshev collocation
points.
43.5 Solution o f a 2 D Laplace E q u a t i o n w i t h Piecewise Boundary

Conditions
O u r f i n a l e x a m p l e is a n o t h e r e l l i p t i c e q u a t i o n . T h i s t i m e we use t h e G a u s s i a n R B F
w i t h a n " o p t i m a l " shape p a r a m e t e r e = 2.549845. A g a i n , t h e s p a t i a l d i s c r e t i z a t i o n
consists o f a tensor p r o d u c t o f 25 x 25 C h e b y s h e v p o i n t s , a n d t h e d i f f e r e n t i a t i o n m a -
t r i x for t h e R B F - P S a p p r o a c h is c o m p u t e d u s i n g t h e D2RBF a n d k r o n c o n s t r u c t i o n
as i n t h e p r e v i o u s e x a m p l e .
<L 4 .
W e consider t h e 2 D L a p l a c e e q u a t i o n
2
u x x + u y y = 0, x,y(-l,l) ,
w i t h boundary conditions
4
sin (7nr), y = 1 a n d 1 < x < 0,
!
isin(37ry), x = 1,
0, otherwise.
T h i s is t h e same p r o b l e m as u s e d i n P r o g r a m 36 o f [ T r e f e t h e n ( 2 0 0 0 ) ] , a n d we d o
n o t l i s t i t here due t o t h e s i m i l a r i t y w i t h p r e v i o u s e x a m p l e s a n d t h e o r i g i n a l code
i n [Trefethen (2000)].
F i g u r e 43.9 shows t h e s o l u t i o n o b t a i n e d v i a t h e C h e b y s h e v a n d R B F pseu-
dospectral methods, respectively. T h e q u a l i t a t i v e b e h a v i o r o f t h e t w o s o l u t i o n s is
very similar.
Fig. 43.9 Solution of the 2D Laplace equation using a Chebyshev P S approach (left) and Gaussian
R B F s (right) with e 2.549845 on 625 tensor-product Chebyshev collocation points.
43.6 Summary
W h i l e t h e r e is n o a d v a n t a g e i n g o i n g t o a r b i t r a r i l y spaced i r r e g u l a r c o l l o c a t i o n
p o i n t s for a n y o f t h e p r o b l e m s p r e s e n t e d here, t h e r e is n o t h i n g t h a t p r e v e n t s us
f r o m d o i n g so for t h e R B F p s e u d o s p e c t r a l a p p r o a c h . I n p a r t i c u l a r , as w e saw i n
S e c t i o n 43.4, we are n o t l i m i t e d t o u s i n g t e n s o r p r o d u c t g r i d s for h i g h e r - d i m e n s i o n a l
spatial discretizations. T h i s is a p o t e n t i a l a d v a n t a g e o f t h e R B F p s e u d o s p e c t r a l
a p p r o a c h over t h e s t a n d a r d p o l y n o m i a l m e t h o d s .
M o r e a p p l i c a t i o n s o f t h e R B F - P S m e t h o d c a n be f o u n d i n t h e r e c e n t p a p e r s
[ F e r r e i r a a n d Fasshauer ( 2 0 0 6 ) ; F e r r e i r a a n d Fasshauer ( 2 0 0 7 ) ] .
F u t u r e challenges i n c l u d e d e a l i n g w i t h l a r g e r p r o b l e m s i n a n efficient a n d s t a -
b l e way. T h u s , such issues as p r e c o n d i t i o n i n g a n d F F T - t y p e a l g o r i t h m s need t o
be s t u d i e d i n t h e c o n t e x t o f R B F p s e u d o s p e c t r a l m e t h o d s . Some first studies o f

t h e eigenvalue s t a b i l i t y o f R B F p s e u d o s p e c t r a l m e t h o d s have been r e p o r t e d v e r y
r e c e n t l y i n [ P l a t t e a n d D r i s c o l l (2006)].
Chapter 44
R B F Galerkin Methods
44.1 A n Elliptic P D E with Neumann Boundary Conditions
A v a r i a t i o n a l a p p r o a c h t o t h e s o l u t i o n o f P D E s w i t h R B F s i n E u c l i d e a n spaces has
so far o n l y been considered i n [ W e n d l a n d (1999a); W e n d l a n d (1999b)] a n d t h e v e r y
recent p a p e r [ H u et al. (2005)]. O n t h e sphere w h e r e we d o n o t have t o w o r r y
a b o u t b o u n d a r y c o n d i t i o n s w e also have [Le G i a (2004)]. I n [ W e n d l a n d (1999b)]
t h e a u t h o r studies t h e H e l m h o l t z e q u a t i o n w i t h n a t u r a l b o u n d a r y c o n d i t i o n s , i.e.,
Au + u = / i n fl,
d
u = 0 on dfl,
on
w h e r e n denotes t h e u n i t o u t e r n o r m a l v e c t o r .
T h e classical G a l e r k i n f o r m u l a t i o n t h e n leads t o t h e p r o b l e m o f f i n d i n g a f u n c t i o n
1
u GH (fl) such t h a t
1
a(u,v) = (f,v) n)
L2{ for a l l v G H (fl),
f)L (f2)
s u s u a
where ( / , 2 * the l Z/2-hiner p r o d u c t , a n d for t h e H e l m h o l t z e q u a t i o n t h e
b i l i n e a r f o r m a is g i v e n b y
a(u,v)
I (Vu V f + uv)dx.
Jo.
1
I n order t o o b t a i n a n u m e r i c a l scheme t h e i n f i n i t e - d i m e n s i o n a l space H (fl) is re-
1
placed b y some finite-dimensional subspace Sx ^ ^ (^) ) where X denotes t h e
c o m p u t a t i o n a l g r i d t o be used for t h e s o l u t i o n . I n t h e c o n t e x t o f R B F s Sx is t a k e n
as
S x = span{<>(|| -Xj\\), Xj G X}.

T h i s results i n a square s y s t e m o f l i n e a r e q u a t i o n s for t h e coefficients o f u G Sx
determined by
a(u,v) = (f,v) ^n) L for a l l v G Sx-
M o r e specifically, ii X = {xi,... ,XN}, then
N
u = Y C
M \ \ X
- J\\),
419
a n d t h e s y s t e m (44.1) is g i v e n b y
/ [Vu(x)-V<p(\\x-Xi\\)+utp(\\x-Xi\\)]dx= [ f(x)<p(\\ x Xi\^)dx,
i = l,. N.
U s i n g l i n e a r i t y a n d t h e d e f i n i t i o n o f u g i v e n above t h i s t u r n s i n t o
Y,cn / [ v ( i i - j i i ) - ' ( i i - < i i ) + ' ( i i - i i i M i iX

v a ; a ! v a j a j a ; a j
Xi\\)] dx
[ f(x)<p(\\x-Xi\\)dx, i = l,...,N.
Aij= [ [V^llaj-ajj-ID-V^dlaj-aJilD + ^ l l x - ^ I I M I I x Xi\\)] dx,
a n d t h e r i g h t - h a n d side entries
T h e e v a l u a t i o n o f these i n t e g r a l s is w h a t is m o s t t i m e - c o n s u m i n g i n t h e RBF
G a l e r k i n a p p r o a c h (see t h e n u m e r i c a l e x p e r i m e n t s o f t h e n e x t c h a p t e r ) . Wend-
l a n d r e p o r t s t h a t t h e n u m e r i c a l e v a l u a t i o n o f these w e a k - f o r m i n t e g r a l s presents a
m a j o r p r o b l e m for t h e r a d i a l basis f u n c t i o n G a l e r k i n a p p r o a c h .
I n a d d i t i o n , R B F G a l e r k i n m e t h o d s w i l l face d i f f i c u l t i e s w i t h D i r i c h l e t (or some-
t i m e s also c a l l e d essential) b o u n d a r y c o n d i t i o n s . B o t h o f these d i f f i c u l t i e s are also
w e l l - k n o w n i n m a n y o t h e r flavors o f meshfree w e a k - f o r m m e t h o d s . A n especially
p r o m i s i n g s o l u t i o n t o t h e issue o f D i r i c h l e t b o u n d a r y c o n d i t i o n s seems t o be t h e use
of .R-functions as p r o p o s e d b y H o l l i g a n d R e i f i n t h e c o n t e x t o f web-splines (see,
e.g., [ H o l l i g (2003)] or o u r e a r l i e r d i s c u s s i o n i n t h e c o n t e x t o f c o l l o c a t i o n m e t h o d s i n
C h a p t e r 3 8 ) . A n o t h e r p o p u l a r a p p r o a c h uses L a g r a n g e m u l t i p l i e r s i n a c o n s t r a i n e d
optimization setting.
For m o r e o n t h e G a l e r k i n m e t h o d see, e.g., [Braess ( 1 9 9 7 ) ; B r e n n e r a n d S c o t t
(1994)] ( i n t h e c o n t e x t o f f i n i t e e l e m e n t s ) , o r [ B a b u s k a et al. (2003)] ( i n t h e c o n t e x t
of M L S - b a s e d meshfree m e t h o d s ) .
44.2 A Convergence Estimate
I t was s h o w n i n [ W e n d l a n d (1999a)] t h a t for those R B F s ( g l o b a l l y as w e l l as l o -

c a l l y s u p p o r t e d ) whose F o u r i e r t r a n s f o r m decays l i k e ( 1 + || ||2) _2/3
the following
convergence e s t i m a t e for t h e R B F G a l e r k i n m e t h o d h o l d s :
\\u - u\\ i
H ( C l ) < Ch a 1
||u||H(f2), (44.2)
44- RBF Galerkin Methods 421
w h e r e h hx,n is t h e fill distance o f X, t h e s o l u t i o n satisfies t h e r e g u l a r i t y r e q u i r e -

a
m e n t s u e H (Q), a n d w h e r e t h e convergence r a t e is d e t e r m i n e d b y (3 > a > s / 2 + 1 .
F r o m o u r discussion i n C h a p t e r 13 we k n o w t h a t t h e F o u r i e r t r a n s f o r m o f W e n d -
l a n d ' s c o m p a c t l y s u p p o r t e d f u n c t i o n s decays as ( 1 + || \\2)~ ~ ~
s 2k 1
So for these
functions t h e above e s t i m a t e i m p l i e s t h a t f u n c t i o n s w h i c h are i n C 2 K
and strictly
S K+ s 1 2
p o s i t i v e definite o n R satisfying K > a w i l l have 0(h ^ ~ ^ ) conver-
2 2
gence order, i.e., t h e C f u n c t i o n = ( 1 r) + yields 0{h) and the C function
2 3
(P3A = ( 1 r ) + ( 4 r + 1) delivers 0{h ) convergence i n R .
A s w i t h t h e convergence e s t i m a t e for s y m m e t r i c c o l l o c a t i o n t h e r e is a l i n k be-
tween t h e r e g u l a r i t y r e q u i r e m e n t s o n t h e s o l u t i o n a n d t h e space d i m e n s i o n s. A l s o ,
we p o i n t o u t t h a t so far t h e t h e o r y is o n l y e s t a b l i s h e d for P D E s w i t h n a t u r a l b o u n d -
ary conditions.
T h e convergence e s t i m a t e (44.2) holds for t h e n o n - s t a t i o n a r y s e t t i n g , i.e., if
we are u s i n g c o m p a c t l y s u p p o r t e d basis f u n c t i o n s , for fixed s u p p o r t r a d i i . B y t h e
same a r g u m e n t s used i n C h a p t e r s 12, 16 a n d 4 1 , one w i l l w a n t t o s w i t c h t o t h e
s t a t i o n a r y s e t t i n g a n d e m p l o y a m u l t i l e v e l a l g o r i t h m i n w h i c h t h e s o l u t i o n a t each
step is u p d a t e d b y a fit t o t h e m o s t recent r e s i d u a l . T h i s s h o u l d ensure b o t h
convergence a n d n u m e r i c a l efficiency.
44.3 A Multilevel R B F Galerkin Algorithm
Here is t h e v a r i a n t o f t h e s t a t i o n a r y m u l t i l e v e l c o l l o c a t i o n a l g o r i t h m l i s t e d i n C h a p -
ter 4 1 a d a p t e d for t h e weak f o r m u l a t i o n o f t h e P D E discussed at t h e b e g i n n i n g o f
t h i s chapter (see [ W e n d l a n d ( 1 9 9 9 b ) ] ) :
A l g o r i t h m 4 4 . 1 . Multilevel Galerkin
(1) u = 0
0
(2) For k f r o m 1 t o K do
(a) F i n d u k E Sx k such t h a t a(u ,v) k = (f,v) a(uk-i,v) for a l l v 6 Sxk
(b) Update u k < u -i k + u k
T h i s a l g o r i t h m does not converge in general (see T a b . 1 i n [ W e n d l a n d ( 1 9 9 9 b ) ] ) .

Since t h e weak f o r m u l a t i o n c a n be i n t e r p r e t e d as a H i l b e r t space p r o j e c t i o n
method, W e n d l a n d was able t o show t h a t a modified version of the multilevel
Galerkin algorithm, namely
A l g o r i t h m 4 4 . 2 . Nested M u l t i l e v e l G a l e r k i n
(1) F i x K and M G N , a n d set v 0 = 0.

(2) For j f r o m 0 w h i l e resiudal > tolerance to M do
(a) Set wo = Vj.

(b) A p p l y t h e A;-loop o f t h e p r e v i o u s a l g o r i t h m a n d d e n o t e t h e result w i t h u(vj).
(c) Set Vj \
+ = u(vj).
does converge. I n fact, u s i n g t h i s a l g o r i t h m W e n d l a n d proves, a n d also observes

n u m e r i c a l l y , convergence w h i c h is a t least l i n e a r (see T h e o r e m 3 a n d T a b . 2 i n
[Wendland (1999b)]).
T h e i m p o r t a n t difference b e t w e e n t h e t w o m u l t i l e v e l G a l e r k i n a l g o r i t h m s is t h e
a d d e d o u t e r i t e r a t i o n i n t h e n e s t e d v e r s i o n w h i c h is a w e l l - k n o w n i d e a f r o m l i n e a r
algebra i n t r o d u c e d i n [ K a c z m a r z ( 1 9 3 7 ) ] . A p r o o f o f t h e l i n e a r convergence for
general H i l b e r t space p r o j e c t i o n m e t h o d s c o u p l e d w i t h K a c z m a r z i t e r a t i o n c a n be
f o u n d i n [ S m i t h et al. ( 1 9 7 7 ) ] . T h i s a l t e r n a t e p r o j e c t i o n idea is also t h e f u n d a m e n t a l
i n g r e d i e n t i n t h e convergence p r o o f o f t h e d o m a i n d e c o m p o s i t i o n m e t h o d o f [ B e a t s o n
et al. (2000)] d e s c r i b e d i n t h e C h a p t e r 35. W e m e n t i o n here t h a t i n t h e m u l t i g r i d
l i t e r a t u r e K a c z m a r z ' m e t h o d is f r e q u e n t l y used as a s m o o t h e r (see e.g. [ M c C o r m i c k
(1992)]).
I n t h e recent p a p e r [Schaback (2003)] t h e a u t h o r presents a f r a m e w o r k for t h e
r a d i a l basis f u n c t i o n s o l u t i o n o f p r o b l e m s b o t h i n t h e s t r o n g ( c o l l o c a t i o n ) a n d w e a k
(Galerkin) form.
M a n y o t h e r meshfree m e t h o d s for t h e s o l u t i o n o f p a r t i a l d i f f e r e n t i a l e q u a t i o n s i n
the weak f o r m appear i n the ( m o s t l y engineering) l i t e r a t u r e . These m e t h o d s come
u n d e r such names as s m o o t h e d p a r t i c l e h y d r o d y n a m i c s ( S P H ) (e.g., [Monaghan
( 1 9 8 8 ) ] ) , r e p r o d u c i n g k e r n e l p a r t i c l e m e t h o d ( R K P M ) (see, e.g., [ L i a n d L i u ( 1 9 9 6 ) ;
L i u et al. ( 1 9 9 7 ) ] ) , p o i n t i n t e r p o l a t i o n m e t h o d ( P I M ) (see, [ L i u ( 2 0 0 2 ) ] ) , e l e m e n t
free G a l e r k i n m e t h o d ( E F G ) (see, e.g., [ B e l y t s c h k o et al. ( 1 9 9 6 ) ] ) , meshless l o -
cal P e t r o v - G a l e r k i n m e t h o d ( M L P G ) [ A t l u r i a n d Z h u (1998)], h-p-cloud m e t h o d
[ D u a r t e a n d O d e n ( 1 9 9 6 b ) ] , p a r t i t i o n o f u n i t y f i n i t e element m e t h o d (PUFEM)
[ B a b u s k a a n d M e l e n k ( 1 9 9 7 ) ; M e l e n k a n d B a b u s k a ( 1 9 9 6 ) ] , o r g e n e r a l i z e d f i n i t e ele-
m e n t m e t h o d ( G F E M ) [ B a b u s k a et al. ( 2 0 0 3 ) ] . M o s t o f these m e t h o d s are based o n
t h e m o v i n g least squares a p p r o x i m a t i o n m e t h o d discussed i n C h a p t e r 22. T h e t w o
recent b o o k s [ A t l u r i a n d S h e n ( 2 0 0 2 a ) ] a n d [ L i u (2002)] s u m m a r i z e m a n y o f these
methods. H o w e v e r , these b o o k s focus m o s t l y o n a s u r v e y o f t h e v a r i o u s m e t h o d s
a n d r e l a t e d c o m p u t a t i o n a l a n d i m p l e m e n t a t i o n issues w i t h l i t t l e e m p h a s i s o n t h e
m a t h e m a t i c a l f o u n d a t i o n o f these m e t h o d s . T h e recent s u r v e y p a p e r [ B a b u s k a et al.
(2003)] fills a l a r g e p a r t o f t h i s v o i d .
Chapter 45
R B F Galerkin Methods in MATLAB
W e consider t h e f o l l o w i n g H e l m h o l t z test p r o b l e m (c.f. [Wendland (1999b)]):

2
Au(x, y) + u(x, y) COS(-KX) cos(7ry) i n fl = [ 1, l ] ,
d ,
-û(x,y)=0 on dfl,
on
2
where x = (x, y) G M a n d n denotes t h e u n i t o u t e r n o r m a l v e c t o r . I t is easy t o
verify t h a t t h e exact s o l u t i o n for t h i s p r o b l e m is g i v e n b y
. . cos(7ra:) cos(7ry)
I n P r o g r a m 45.1 we p r o v i d e a s i m p l e M A T L A B i m p l e m e n t a t i o n for t h e G a l e r k i n
solution of this problem. N o t e t h a t o u r p r o g r a m does n o t a t t e m p t t o p r o v i d e a
m u l t i l e v e l s o l u t i o n as described i n t h e p r e v i o u s c h a p t e r , n o r d o we p r e t e n d t o be
especially efficient ( a n d therefore t h e p r o g r a m is v e r y s l o w ) . A s p o i n t e d o u t i n t h e
previous chapter, t h e m o s t t i m e c o n s u m i n g p a r t is t h e c a l c u l a t i o n o f t h e i n t e g r a l s
needed for t h e stiffness m a t r i x A w i t h entries
A^ = / Vip(\\x - Xi\\) Vip(\\x - Xj\\)dx

J[-i,i]'
+ / <p(\\n ~ a5i||MI|au - Xj\\)dx,
a n d t h e r i g h t - h a n d side v e c t o r w i t h entries
f(x)ip(\\x - \\)dx.
Xi
-I,IP
W e c o m p u t e these integrals u s i n g t h e d b l q u a d n u m e r i c a l i n t e g r a t i o n r o u t i n e o n
lines 15-20 o f P r o g r a m 4 5 . 1 . N o t e t h a t we e x p l o i t t h e s y m m e t r y o f t h e stiffness
m a t r i x i n t h e for-loop, a n d t h e n c o m p l e t e t h e m a t r i x o n l i n e 2 1 . T h e f u n c t i o n s
needed for t h e i n t e g r a t i o n are p r o v i d e d o n lines 1-3 a n d 5 - 6 . I n [ W e n d l a n d (1999b)]
t h e a u t h o r details a s t r a t e g y for c o n v e r t i n g t h e d o u b l e i n t e g r a l s t o u n i v a r i a t e i n t e -
grals since a l l t h e functions i n v o l v e d are r a d i a l l y s y m m e t r i c . W e d o n o t p u r s u e t h a t
p o s s i b i l i t y here.
For t h i s e x a m p l e we use t h e C 2
W e n d l a n d f u n c t i o n <^3,i(r) = ( 1 r ) ^ _ ( 4 r + 1)
w i t h a s u p p o r t scaled b y e = 0.7. O n l i n e 4 we p r o v i d e t h e s t a n d a r d representation
423
of t h e basic f u n c t i o n as i t is needed for t h e e v a l u a t i o n a n d p l o t t i n g p a r t o f t h e

p r o g r a m (lines 2 3 - 3 4 , w h i c h are o f t h e same f o r m as o u r earlier p r o g r a m s ) .
P r o g r a m 45.1. RBFGalerkin2D .m
% RBFGalerkin2D
70 S c r i p t t h a t performs G a l e r k i n s o l u t i o n of 2D Helmholtz equation
% - u_xx - u_yy + u = f
% D e f i n i t i o n of t h e RBF and i t s g r a d i e n t , Wendland C2
l a rbf = @(e,x,y,xi,yi) m a x ( l - e * s q r t ( ( x - x i ) . ~ 2 + ( y - y i ) . ~ 2 ) , 0 ) . " 4 . * . . .
lb (4*e*sqrt((x-xi).~2+(y-yi).~2)+l);
2a dxrbf = @ ( e , x , y , x i , y i ) - 2 0 * ( x - x i ) * e ~ 2 . * . . .
2b max(l-e*sqrt((x-xi).~2+(y-yi)."2),0)."3;
3a dyrbf = Q ( e , x , y , x i , y i ) - 2 0 * ( y - y i ) * e ~ 2 . * . . .
3b max(l-e*sqrt((x-xi).~2+(y-yi).~2),0)."3;
4 e v a l r b f = @(e,r) m a x ( l - e * r , 0 ) . " 4 . * ( 4 * e * r + l ) ;
/ Products f o r i n t e g r a t i o n
0
5 rp = @(e,x,y,xi,yi,xj,yj) r b f ( e , x , y , x i , y i ) . * r b f ( e , x , y , x j , y j ) ;
6a gp = @ ( e , x , y , x i , y i , x j , y j ) d x r b f ( e , x , y , x i , y i ) . * . . .
6b dxrbf(e,x,y,xj,yj)+dyrbf(e,x,y,xi,yi).*dyrbf(e,x,y,xj,yj);
% Parameter f o r b a s i s f u n c t i o n
7 ep = .7;
% Right-hand s i d e f u n c t i o n f o r Helmholtz equation
8 f = @(x,y) c o s ( p i * x ) . * c o s ( p i * y ) ;
/ Exact s o l u t i o n
0
9 u = @(x,y) c o s ( p i * x ) . * c o s ( p i * y ) / ( 2 * p i " 2 + l ) ;
/ Number and type of c e n t e r s :
0
5
10 N = 25; g r i d t y p e = u ' ;
R e s o l u t i o n of e v a l u a t i o n g r i d f o r e r r o r s and p l o t t i n g
11 n e v a l = 40;
7. Load d a t a p o i n t s
5
12 name = s p r i n t f ('Data2D_7od7.s , N,gridtype); load(name)
7o S h i f t c e n t e r s t o t h e square [-1,1] ~2
13 ctrs = 2*dsites-l;
7o B u i l d s t i f f n e s s m a t r i x and r i g h t - h a n d s i d e
14 A = zeros(N,N); r h s = z e r o s ( N , l ) ;
15 f o r i=l:N
16 for j=l:i
17a A ( i , j ) = dblquad(@(x,y) g p ( e p , x , y , c t r s ( i , 1 ) , c t r s ( i , 2 ) , . . .
17b c t r s ( j , 1 ) , c t r s ( j , 2 ) ) , - 1 , 1 , - 1 , 1 ) + ...
17c dblquad(@(x,y) r p ( e p , x , y , c t r s ( i , 1 ) , c t r s ( i , 2 ) , . . .
17d ctrs(j,1),ctrs(j,2)),-1,1,-1,1);
45. RBF Galerkin Methods in MATLAB 425
18 end
19a r h s ( i ) = dblquad(@(x,y) f ( x , y ) . * . . .
19b rbf(ep,x,y,ctrs(i,l),ctrs(i,2)),-1,1,-1,1);
20 end
7, Make m a t r i x symmetric
5
21 A = A + A - d i a g ( d i a g ( A ) ) ;
7. Solve l i n e a r system, i . e . , compute expansion c o e f f i c i e n t s
22 c = A\rhs;
7o E v a l u a t i o n
23 g r i d = l i n s p a c e ( - 1 , 1 , n e v a l ) ; [xe, y e ] = m e s h g r i d ( g r i d ) ;
24 e p o i n t s = [ x e ( : ) y e ( : ) ] ;
27 EM = evalrbf(ep,DM_eval);
28 Pf = EM * c;
29 maxerr = n o r m ( P f - e x a c t , i n f ) ; rms.err = n o r m ( P f - e x a c t ) / n e v a l ;
30 f p r i n t f ('RMS e r r o r : 7.e\n', r m s . e r r )
31 f p r i n t f ('Maximum e r r o r : 7.e\n , maxerr) J
7o P l o t approximate s o l u t i o n
32 fview = [-30,30]; % viewing angle f o r p l o t
E r r o r s , c o n d i t i o n n u m b e r s o f t h e stiffness m a t r i x , a n d observed convergence rates

are l i s t e d i n T a b l e 4 5 . 1 . A plot o f the approximate solution and error distribution
u s i n g 81 e q u a l l y spaced centers t o generate t h e t r i a l a n d test spaces is p r o v i d e d i n
F i g u r e 45.1.
Table 45.1 Errors and condition numbers for Galerkin solution of

2
Helmholtz equation using the C Wendland function with e = 0.7.
N ôo-error rate RMS-error rate cond(A)
9 4.774434e-003 1.013915e-003 8.159139e+000

25 3.223359e-003 0.5668 9.561258e-004 0.0847 1.408312e+002
81 9.346870e-005 5.1079 2.494297e-005 5.2605 3.232525e+004
289 9.701313e-005 -0.0537 2.239018e-005 0.1558 6.897924e+007
W e c a n see t h a t t h e convergence is r a t h e r e r r a t i c , a n d t h a t t h e c o n d i t i o n n u m b e r
increases r a p i d l y . T h e W\-convergence r a t e p r e d i c t e d i n [ W e n d l a n d (1999a)] for t h e
basic f u n c t i o n used here is 0(h). O n average, t h e r e s u l t s l i s t e d i n T a b l e 45.1 i n d i c a t e
2
r o u g h l y a n 0(h ) RMS-convergence rate.
Appendix A
Useful Facts from Discrete Mathematics
A.l Halton Points
H a l t o n p o i n t s (see [ H a l t o n ( 1 9 6 0 ) ; W o n g et al. (1997)]) are c r e a t e d f r o m van der

Corput sequences. T h e y f o r m so-called l o w d i s c r e p a n c y sequences a n d are used
f r e q u e n t l y i n q u a s i - M o n t e C a r l o m e t h o d s for m u l t i - d i m e n s i o n a l i n t e g r a t i o n a p p l i -
cations.
T h e s t a r t i n g p o i n t i n t h e c o n s t r u c t i o n o f a v a n der C o r p u t sequence is t h e fact
t h a t every n o n n e g a t i v e integer n c a n be w r i t t e n u n i q u e l y u s i n g a p r i m e base p, i.e.,
k
n = Jâip\
i=0
w h e r e each coefficient ai is a n integer such t h a t 0 < ai < p. F o r e x a m p l e , i f n = 10
a n d p = 3, t h e n
1 2
10 = 1 3 + 0 3 + 1 -3 ,
so t h a t k = 2 a n d an = a 2 = 1 a n d a\ = 0.
N e x t we define a f u n c t i o n h p t h a t m a p s t h e n o n n e g a t i v e integers t o t h e i n t e r v a l
[0,1) via
A:
y
i=0
For example
T h e r e s u l t i n g sequence h N Pt = {h (n) p : n = 0 , 1 , 2 , . . . , N} is k n o w n as a v a n
d e r C o r p u t sequence. For e x a m p l e
h ,w
3 = {0,1/3,2/3,1/9,4/9,7/9,2/9,5/9,8/9,1/27,10/27}.
I n o r d e r t o generate a Halton point set i n s - d i m e n s i o n a l space ( m o r e precisely

s
i n t h e s - d i m e n s i o n a l u n i t c u b e [0, l ) ) w e t a k e s ( u s u a l l y d i s t i n c t ) p r i m e s p,... ,p s
427
a n d use t h e r e s u l t i n g v a n der C o r p u t sequences h P l t N , . . . , h P a t N as t h e c o o r d i n a t e s

o f t h e s - d i m e n s i o n a l H a l t o n p o i n t s , i.e., t h e set
n
HS,N = { ( V ( ) > -> Ps( ))
h n
' n = 0 , 1 , . . . ,iV}
s
is t h e set o f N + 1 H a l t o n p o i n t s i n [0, l ) . H a l t o n p o i n t sets for s = 2 are d i s p l a y e d
i n F i g u r e 1.1 a n d t h e b o t t o m p a r t o f F i g u r e 14.5.
An nice p r o p e r t y o f H a l t o n p o i n t s is t h e fact t h a t t h e y are nested p o i n t sets,
i.e., HM SI C HN
S> for M < N. I n fact, t h e p o i n t sets c a n even be c o n s t r u c t e d
sequentially, i.e., one does n o t need t o s t a r t over i f one w a n t s t o a d d m o r e p o i n t s
t o a n e x i s t i n g set o f H a l t o n p o i n t s . T h i s d i s t i n g u i s h e s t h e H a l t o n p o i n t s f r o m t h e
related Hammersley points.
I t is k n o w n t h a t i n l o w space d i m e n s i o n s , t h e m u l t i - d i m e n s i o n a l H a l t o n sequence
q u i c k l y "fills u p " t h e u n i t cube i n a w e l l - d i s t r i b u t e d p a t t e r n . H o w e v e r , for h i g h e r
d i m e n s i o n s (such as s = 4 0 ) , u s i n g a r e l a t i v e l y s m a l l v a l u e o f N results i n p o o r l y
d i s t r i b u t e d H a l t o n p o i n t s . O n l y w h e n N is l a r g e e n o u g h r e l a t i v e t o s d o t h e p o i n t s
become w e l l - d i s t r i b u t e d . Since n o n e o f o u r e x a m p l e s exceed s = 6 t h i s is n o t a
c o n c e r n for us.
I n t h e M A T L A B p r o g r a m s t h r o u g h o u t t h i s b o o k we use t h e f u n c t i o n haltonseq
w r i t t e n b y D a n i e l D o u g h e r t y . T h i s f u n c t i o n c a n be d o w n l o a d e d f r o m t h e M A T L A B
C e n t r a l F i l e E x c h a n g e , see [ M C F E ] . I n t h i s i m p l e m e n t a t i o n o f H a l t o n sequences
t h e o r i g i n is n o t p a r t o f t h e p o i n t set, i.e., t h e H a l t o n p o i n t s are g e n e r a t e d s t a r t i n g
w i t h n 1 i n s t e a d o f n = 0 as d e s c r i b e d above.
A. 2 fed-Trees
I n o r d e r t o d e a l w i t h large sets o f d a t a e f f i c i e n t l y w e f r e q u e n t l y use compactly sup-
ported basic f u n c t i o n s (see, e.g., C h a p t e r 1 2 ) . F o r t h e i r successful i m p l e m e n t a t i o n

c e r t a i n g e o m e t r i c i n f o r m a t i o n is r e q u i r e d . M o s t i m p o r t a n t l y , we need t o k n o w w h i c h
d a t a sites lie i n t h e s u p p o r t o f a g i v e n basis f u n c t i o n . S u c h a q u e r y is k n o w n as a
range search. W e also m a y be i n t e r e s t e d i n finding a l l centers whose s u p p o r t c o n -
t a i n s a g i v e n ( e v a l u a t i o n ) p o i n t x. S u c h a q u e r y is k n o w n as a containment query.
F u r t h e r m o r e , we m i g h t also be i n t e r e s t e d i n f i n d i n g t h e ( n ) nearest neighbors of a

g i v e n p o i n t (for i n s t a n c e i f we need t o find t h e s e p a r a t i o n d i s t a n c e qx o f a set o f
p o i n t s X). O n e w a y t o a c c o m p l i s h these t a s k s is v i a kd-trees. A A:d-tree ( s h o r t for
/c-dimensional t r e e ) is a s p a c e - p a r t i t i o n i n g d a t a s t r u c t u r e for o r g a n i z i n g p o i n t s i n
fc-dimensional space. T h u s , i f w e w e r e t o be t r u e t o t h e n o t a t i o n used t h r o u g h o u t
t h i s b o o k , we s h o u l d t e c h n i c a l l y be r e f e r r i n g t o these trees as sd-trees. We will,
however, s t i c k w i t h t h e u s u a l t e r m i n o l o g y a n d refer t o t h e m as kd-trees.
T h e p u r p o s e o f A:d-trees is t o h i e r a r c h i c a l l y decompose a set o f N d a t a p o i n t s i n
s
R i n t o a r e l a t i v e l y s m a l l n u m b e r o f subsets such t h a t each subset c o n t a i n s r o u g h l y
t h e same n u m b e r o f d a t a sites. E a c h n o d e i n t h e t r e e is defined b y a s p l i t t i n g p l a n e
t h a t is p e r p e n d i c u l a r t o one o f t h e c o o r d i n a t e axes a n d passes t h r o u g h one o f t h e
A. Useful Facts from Discrete Mathematics 429
d a t a p o i n t s . T h e r e f o r e t h e s p l i t t i n g planes p a r t i t i o n t h e set o f p o i n t s a t t h e m e d i a n
i n t o "left" a n d " r i g h t " (or " t o p " a n d " b o t t o m " ) subsets, each w i t h r o u g h l y h a l f t h e
p o i n t s o f t h e p a r e n t node. T h e s e c h i l d r e n are a g a i n p a r t i t i o n e d i n t o e q u a l halves,
u s i n g planes t h r o u g h a different d i m e n s i o n ( u s u a l l y one keeps o n c y c l i n g t h r o u g h t h e
d i m e n s i o n s w h e n d e t e r m i n i n g t h e n e x t s p l i t t i n g p l a n e ) . T h i s p a r t i t i o n i n g process
7
stops after l o g A " levels. I n t h e e n d every n o d e o f t h e fcd-tree, f r o m t h e r o o t t o t h e
leaves, stores a p o i n t . T h e c o m p u t a t i o n a l c o m p l e x i t y for b u i l d i n g a fcd-tree f r o m JV
s
points i n R is 0(sN l o g N). O n c e t h e t r e e is b u i l t , a r a n g e q u e r y c a n be p e r f o r m e d
i n 0(\ogN) t i m e . T h i s compares, f a v o r a b l y w i t h t h e O(N) time it would take to
search t h e " r a w " d a t a set.
I n o u r M A T L A B examples we use t h e f u n c t i o n s k d t r e e a n d k d r a n g e q u e r y from
t h e fcd-tree l i b r a r y ( g i v e n as a set o f M A T L A B M E X - f i l e s w r i t t e n b y G u y Shechter
t h a t c a n be d o w n l o a d e d f r o m t h e M A T L A B C e n t r a l F i l e E x c h a n g e , see [ M C F E ] ) .
F i g u r e A . l shows a s t a n d a r d m e d i a n - b a s e d p a r t i t i o n i n g o f n i n e H a l t o n p o i n t s
2
i n [0, l ] o n t h e left a l o n g w i t h t h e associated A;d-tree o n t h e r i g h t .
Fig. A . l kd partitioning (left) and tree (right) for 9 Halton points.

Appendix B
Useful Facts from Analysis
B.l Some Important Concepts from Measure T h e o r y
B o c h n e r ' s t h e o r e m (c.f. T h e o r e m 3.3) a n d a n u m b e r o f o t h e r results are f o r m u l a t e d

i n t e r m s o f Borel measures.
Since we refer t o t h e b o o k [ W e n d l a n d (2005a)] for m a n y o f t h e t h e o r e t i c a l results
presented i n t h i s b o o k we f o l l o w t h e e x p o s i t i o n i n [ W e n d l a n d (2005a)]. W e s t a r t
w i t h a n a r b i t r a r y set X, a n d d e n o t e t h e set o f a l l subsets o f X b y V(X). The empty
set is d e n o t e d b y 0.
D e f i n i t i o n B . l . A subset A o f V(X) is c a l l e d a cr-algebra on X if
(1) XeA,
(2) A A i m p l i e s t h a t i t s c o m p l e m e n t ( i n X) is also c o n t a i n e d i n A ,
(3) Ai E A , i E N , i m p l i e s t h a t t h e u n i o n o f these sets is c o n t a i n e d i n A .
D e f i n i t i o n B . 2 . G i v e n a n a r b i t r a r y set X a n d a cr-algebra A o f subsets o f X, a

measure o n A is a f u n c t i o n fj, : A > [0, oo] such t h a t
(1) M0) = o,
(2) for any sequence {Ai} o f d i s j o i n t sets i n A we have
oo oo
i=l i=l
Definition B . 3 . I f X is a t o p o l o g i c a l space, a n d O is t h e c o l l e c t i o n o f o p e n sets

i n X, t h e n t h e cr-algebra g e n e r a t e d b y O is called t h e Borel a-algebra and denoted
b y B(X). I f in addition X is a H a u s d o r f f space, t h e n a measure \x defined o n B(X)
t h a t satisfies n(K) < oo for a l l c o m p a c t sets K C X is called a Borel measure.
T h e carrier o f a B o r e l measure is g i v e n b y t h e set X \ { 0 : O E O and /i(O) = 0 } .
431
B.2 A B r i e f S u m m a r y of I n t e g r a l T r a n s f o r m s
W e s u m m a r i z e f o r m u l a s for v a r i o u s i n t e g r a l t r a n s f o r m s used t h r o u g h o u t t h e t e x t .
T h e F o u r i e r t r a n s f o r m c o n v e n t i o n s w e adhere t o are l a i d o u t i n
s
D e f i n i t i o n B . 4 . T h e Fourier transform o f / E L i ( R ) is g i v e n b y
/() = -=L= f ?
(27V) s
jRs
i
f(x)e- ^dx, u; E R*, (B.l)
a n d i t s inverse Fourier transform is g i v e n b y
f{x) = -JL= s
/ f(oj)e*"du,, x E W.
v /(27r) JWLS
T h i s d e f i n i t i o n o f t h e F o u r i e r t r a n s f o r m c a n be f o u n d i n [ R u d i n ( 1 9 7 3 ) ] . A n o t h e r ,
j u s t as c o m m o n , d e f i n i t i o n uses
2
/() = ( f(x)e- "*dx, (B.2)
JR S
a n d can be f o u n d i n [Stein a n d Weiss ( 1 9 7 1 ) ] . T h e f o r m ( B . l ) w e use c a n also be

f o u n d i n t h e b o o k s [ W e n d l a n d (2005a); S c h o l k o p f a n d S m o l a (2002)], whereas ( B . 2 )
is used i n t h e b o o k s [ B u h m a n n ( 2 0 0 3 ) ; C h e n e y a n d L i g h t (1999)].
S i m i l a r l y , we c a n define t h e F o u r i e r t r a n s f o r m o f a f i n i t e (signed) measure fi o n
W by
A ( )
W = _ L = / e - ^ d ^ x ) , weR . 3
x /(27r) s
JRS
Since we are m o s t l y i n t e r e s t e d i n p o s i t i v e d e f i n i t e r a d i a l f u n c t i o n s , we n o t e t h a t
t h e F o u r i e r t r a n s f o r m o f a r a d i a l f u n c t i o n is a g a i n r a d i a l . I n d e e d ,
s
T h e o r e m B . l . Let $ E Z / i ( R ) be continuous and radial, i.e., &(x) = <^(||cc||).
Then its Fourier transform < is also radial, i.e., <3>(o;) = .F,s(/?(||u>||) with
1 i
i t t2
FsVir) - i o / P( ) J(s-2)/2{rt)dt,
s A
Vr ~ Jo
where J( -2)/2
s is the classical Bessel function of the first kind of order (s 2 ) / 2 .
T h e p r o o f o f t h i s t h e o r e m c a n be f o u n d i n [ W e n d l a n d (2005a)]. The integral

t r a n s f o r m a p p e a r i n g i n T h e o r e m B . l is also referred t o as a Fourier-Bess el transform
or Hankel transform.
T h e Hankel inversion theorem [Sneddon (1972)] ensures t h a t t h e F o u r i e r t r a n s -
f o r m for r a d i a l f u n c t i o n s is i t s o w n inverse, i.e., for r a d i a l f u n c t i o n s cp w e have
A t h i r d i n t e g r a l t r a n s f o r m t h a t p l a y s a n i m p o r t a n t role is t h e Laplace transform.

W e have
Definition B . 5 . L e t / be a piecewise c o n t i n u o u s f u n c t i o n t h a t satisfies \f(t)\ <

at
Me for some c o n s t a n t s a a n d M. T h e Laplace transform o f / is g i v e n b y
poo
Cf(s)= / f(t)e~ dt, st
s > a.
JQ
B. Useful Facts from Analysis 433
S i m i l a r l y , t h e L a p l a c e t r a n s f o r m o f a B o r e l measure fx o n [0, oo) is g i v e n b y

poo
st
/i(s) = / e- dii(t).
Jo
T h e L a p l a c e t r a n s f o r m is c o n t i n u o u s a t t h e o r i g i n i f a n d o n l y i f \x is f i n i t e .
B.3 T h e Schwartz Space and the Generalized Fourier Transform
G e n e r a l i z e d F o u r i e r t r a n s f o r m s are r e q u i r e d i n t h e t r e a t m e n t o f c o n d i t i o n a l l y p o s i -
t i v e d e f i n i t e f u n c t i o n s . For t h e d e f i n i t i o n o f t h e generalized F o u r i e r t r a n s f o r m g i v e n
b e l o w we have t o define t h e Schwartz space o f r a p i d l y decreasing test f u n c t i o n s
S
S = { 7 G C(M ) : lim xîD^Xx) = 0, cx, 3 & W }, 0
||as||-oo
w h e r e we use t h e m u l t i - i n d e x n o t a t i o n
S
T h e space S consists o f a l l those f u n c t i o n s 7 G C ( M ) w h i c h , t o g e t h e r w i t h
all t h e i r d e r i v a t i v e s , decay faster t h a n a n y p o w e r o f l / | | c c | | . T h e space S contains
s s
t h e space C o ( I R ) , t h e space o f a l l i n f i n i t e l y d i f f e r e n t i a b l e f u n c t i o n s o n M with
c o m p a c t s u p p o r t . W e also n o t e t h a t CQ(R ) S
is a t r u e subspace o f S since, e.g., t h e
x s
f u n c t i o n 7(0?) = e~H H belongs t o S b u t n o t t o C o ( R ) . A r e m a r k a b l e fact a b o u t
t h e S c h w a r t z space is t h a t 7 G <S has a classical F o u r i e r t r a n s f o r m 7 w h i c h is also
in S.
O f p a r t i c u l a r i m p o r t a n c e are t h e f o l l o w i n g subspaces S m of S
S m = {7 G S : 70*0 = 0(\\x\\ ) m
for ||a:|| 0, m G N } . 0
F u r t h e r m o r e , t h e set V o f s l o w l y i n c r e a s i n g f u n c t i o n s is g i v e n b y
S
V = {f e C ( K ) : | / ( x ) | < |p(as)| for some p o l y n o m i a l p G L T } .
T h e generalized F o u r i e r t r a n s f o r m is n o w g i v e n b y
S
Definition B . 6 . Let / G V be c o m p l e x - v a l u e d . A c o n t i n u o u s f u n c t i o n / : 1R \
{ 0 } > C is called t h e generalized Fourier transform o f / i f t h e r e exists a n integer
m G No such t h a t
/ f(x)j(x)dx = / f(x)-f(x)dx
is satisfied for a l l 7 G c > 2 . T h e smallest such integer m is c a l l e d t h e order

m of / .
V a r i o u s d e f i n i t i o n s o f t h e generalized F o u r i e r t r a n s f o r m exist i n t h e l i t e r a t u r e .
A classical reference is t h e b o o k [ G e l ' f a n d a n d V i l e n k i n (1964)].
Since one can show t h a t t h e generalized F o u r i e r t r a n s f o r m o f a n s-variate p o l y -
n o m i a l of degree at m o s t 2 m is zero, i t follows t h a t t h e inverse generalized F o u r i e r
t r a n s f o r m is o n l y u n i q u e u p t o a d d i t i o n o f s u c h a p o l y n o m i a l . T h e o r d e r o f t h e
g e n e r a l i z e d F o u r i e r t r a n s f o r m is n o t h i n g b u t t h e o r d e r o f t h e s i n g u l a r i t y a t t h e o r i -
s
g i n o f t h e generalized F o u r i e r t r a n s f o r m . For functions i n L i ( R ) the generalized
F o u r i e r t r a n s f o r m c o i n c i d e s w i t h t h e classical F o u r i e r t r a n s f o r m , a n d for f u n c t i o n s
S
i n L 2 ( M ) i t coincides w i t h t h e d i s t r i b u t i o n a l F o u r i e r t r a n s f o r m .
Appendix C
Additional Computer Programs
I n t h i s a p p e n d i x we list several M A T L A B a n d one M a p l e p r o g r a m t h a t are used i n

various places t h r o u g h o u t t h e b o o k .
C l MATLAB Programs
A s a test f u n c t i o n for m u l t i - d i m e n s i o n a l p r o b l e m s we s o m e t i m e s use

s
f (x)
8 = 4 fj
s
x (l
d - x ),
d x = (x ...,x )e
u s
s
[0, l ] .
d=l
P r o g r a m C . l . t e s t f unct i o n . m
% tf = testfunction(s,points)
/ E v a l u a t e s
0 testfunction
/ p r o d _ { d = l } ~ s x _ d * ( l - x _ d )
0 ( n o r m a l i z e d so t h a t i t s max i s 1)
% at s-dimensional points
function t f = testfunction(s,points)
tf = 4"s*prod(points.*(1-points),2);
A n o t h e r test f u n c t i o n used i n some o f t h e n u m e r i c a l e x p e r i m e n t s is t h e sine

s
f u n c t i o n defined for any x = (xi,..., x)
s M as
- A sin(7rrr ) d
sine (x) = A
I IX
-.
TTXd
d = l
T h e sine f u n c t i o n is n o t a s t a n d a r d M A T L A B f u n c t i o n . I t can, however, be f o u n d i n

t h e S i g n a l Processing T o o l b o x . F o r t h e sake o f completeness we p r o v i d e M A T L A B
code for t h e sine f u n c t i o n o f a single v a r i a b l e , x 1R.
P r o g r a m C.2. sinc.m
% f = sinc(x)
% Defines sine function
function f = sinc(x)
435
f = ones(size(x));
nz = f i n d ( x ~ = 0 ) ;
f(nz) = sin(pi*x(nz))./(pi*x(nz));
N o t e t h a t w h i l e s i n c . m takes a v e c t o r i n p u t x i t p r o d u c e s a v e c t o r o f values
o f t h e u n i v a r i a t e sine f u n c t i o n at t h e c o m p o n e n t s o f x n o t t h e value o f t h e
m u l t i v a r i a t e sine f u n c t i o n a t t h e v e c t o r a r g u m e n t x .
A m u l t i - d i m e n s i o n a l g r i d o f e q u a l l y spaced p o i n t s is used several t i m e s t h r o u g h -
out the book. M A T L A B provides the c o m m a n d n d g r i d t h a t can accomplish this.
However, i n o r d e r t o be able t o use t h i s c o m m a n d f l e x i b l y for a l l space d i m e n s i o n s
s we require a l i t t l e e x t r a w o r k . T h i s is i m p l e m e n t e d M a k e S D G r i d . m .
Program C . 3 . MakeSDGrid.m
% gridpoints = MakeSDGrid(s,neval)
% Produces m a t r i x of e q u a l l y spaced p o i n t s i n s-dimensional unit cube
/ ( o n e p o i n t p e r
0 row)
% Input
% s: space dimension
% neval: number of points i n each c o o r d i n a t e direction
% Output
7o gridpoints: neval~s-by-s matrix (one p o i n t p e r row,
7o d - t h column c o n t a i n s d - t h c o o r d i n a t e of point)
function gridpoints = MakeSDGrid(s,neval)
if (s==l)
gridpoints = linspace(0,1,neval)';
return;
end
7o M i m i c t h i s statement f o r general s:
7o [ x l , x2] = n d g r i d ( l i n s p a c e ( 0 , 1 , n e v a l ) ) ;
outputarg = ' x l ' ;
for d = 2:s
outputarg = strcat(outputarg,',x',int2str(d));
end
makegrid = s t r c a t ( ' [ ' , o u t p u t a r g , ' ] = ndgrid(linspace(0,1,neval));');
eval(makegrid);
7o M i m i c t h i s statement f o r general s:
7o g r i d p o i n t s = [xl(:) x2(:)];
gridpoints = zeros(neval"s,s);
for d = l:s
matrices = strcat('gridpoints(:,d) = x',int2str(d),'(:);');
eval(matrices);
end
C. Additional Computer Programs 437
D u e t o i t s r e m o v a b l e s i n g u l a r i t y a t t h e o r i g i n t h e t h i n - p l a t e spline basic f u n c t i o n
requires a separate f u n c t i o n d e f i n i t i o n .
P r o g r a m C.4. tps.m
% rbf = tps(e,r)
'/, Defines t h i n p l a t e s p l i n e RBF
function rbf = tps(e,r)
rbf = z e r o s ( s i z e ( r ) ) ;
nz = f i n d ( r ~ = 0 ) ; % t o d e a l with s i n g u l a r i t y a t o r i g i n
rbf(nz) = (e*r(nz)).~2.*log(e*r(nz));
S t a n d a r d p l o t t i n g r o u t i n e s for 2 D f u n c t i o n a n d e r r o r g r a p h s are used b y m o s t

programs.
P r o g r a m C.5. P l o t S u r f .m
% P l o t S u r f ( x e , ye, Pf,neval,exact,maxerr,fview)
% Generates p l o t of s u r f a c e Pf f a l s e c o l o r e d by t h e
% maximum e r r o r a b s ( P f - e x a c t )
% fview d e f i n e s t h e view.
function PlotSurf(xe,ye,Pf,neval,exact,maxerr,fview)
/ P l o t s u r f a c e
0
figure
Pfplot = surf(xe,ye.reshape(Pf,neval,neval),.. .
reshape(abs(Pf-exact),neval,neval));
5
s e t ( P f p l o t , ' F a c e C o l o r ' ,'interp','EdgeColor','none )
[cmin cmax] = c a x i s ;
caxis([cmin-.25*maxerr cmax]);
view(fview);
colormap hsv
vcb = c o l o r b a r ( ' v e r t ' ) ;
y l i m ( v c b , [ 0 maxerr])
set(get(vcb,'YLabel'),'String','Error')
P r o g r a m C.6. P l o t E r r o r 2 D .m
'/, P l o t E r r o r 2 D (xe, ye, Pf , exact,maxerr,neval, fview)
/ Generates p l o t of abs e r r o r f o r s u r f a c e P f , i . e . , abs ( P f - e x a c t )
0
fview d e f i n e s t h e view,
function PlotError2D(xe,ye,Pf,exact,maxerr,neval,fview)
% P l o t maximum e r r o r
f igure
errorplot = surf(xe,ye,reshape(abs(Pf-exact),neval,neval));
set(errorplot,'FaceColor','interp','EdgeColor','none')
[ c m i n cmax] = caxis;
c a x i s ( [ c m i n - . 2 5 * m a x e r r cmax])
view(fview);
colormap hsv
ylim(vcb,[0 maxerr])
set(get(vcb,'YLabel') , 'String >,'Error')
For 3 D p l o t s we use t h e f o l l o w i n g r o u t i n e s .
P r o g r a m C . 7 . P l o t l s o s u r f .m
% Plotlsosurf(xe,ye,ze,Pf,neval,exact.maxerr,isomin,
7, isostep,isomax)
7o G e n e r a t e s p l o t of isosurfaces of Pf f a l s e c o l o r e d by
7o t h e error abs ( P f - e x a c t )
7o i s o m i n , i s o s t e p , i s o m a x define the r a n g e and number o f
7o isosurfaces.
function Plotlsosurf(xe,ye,ze,Pf,neval,exact,maxerr,...
isomin,isostep,isomax)
7 P l o t isosurfaces
figure
hold on
for isovalue=isomin:isostep:isomax
pfit = patch(isosurface(xe,ye,ze,reshape(Pf,neval,.. .
neval,neval),isovalue,reshape(abs(Pf-exact),. . .
neval,neval,neval)));
isonormals(xe,ye,ze,reshape(Pf,neval,neval,neval),pfit)
5
set(pf i t , ' F a c e C o l o r ' , ' i n t e r p ' , EdgeColor','none');
daspect([l 11])
view(3); axis([0 1 0 1 0 1])
end
[ c m i n cmax] = caxis;
caxis([cmin-.25*cmax cmax])
colormap hsv
ylim(vcb,[0 cmax])
set(get(vcb,'YLabel'),'String',* Error')
hold off
P r o g r a m C.8. PlotSlices.m
% PlotSlices(xe,ye,ze,Pf,neval,xslice,yslice,zslice)
% Generates s l i c e plot o f volume Pf
7 x s l i c e , y s l i c e , z s l i c e d e f i n e the r a n g e and number o f slices
function P l o t S l i c e s ( x e , y e , z e , P f , n e v a l , x s l i c e , y s l i c e , z s l i c e )
% Plot s l i c e s
figure
p f i t = slice(xe,ye,ze,reshape(Pf,neval,neval,neval),...
xslice,yslice,zslice);
set(pf it,'FaceColor','interp','EdgeColor','none')
daspect([l 1 1 ] )
view(3); a x i s ( [ 0 1 0 1 0 1])
set(get(vcb,'YLabel'),'String','Function value')
P r o g r a m C . 9 . P l o t E r r o r S l i c e s .m
7 P l o t E r r o r S l i c e s (xe, ye, ze, Pf , exact ,ne, x s l i c e , y s l i c e , z s l i c e )
% Generates s l i c e p l o t of volume e r r o r a b s ( P f - e x a c t )
% x s l i c e , y s l i c e , z s l i c e d e f i n e the range and number of s l i c e s ,
function PlotErrorSlices(xe,ye,ze,Pf,exact,ne,...
xslice,yslice,zslice)
% Plot s l i c e s f o r error
figure
errorplot = slice(xe,ye,ze,reshape(abs(Pf-exact),ne,ne,ne),...
xslice,yslice,zslice);
set(errorplot,'FaceColor','interp','EdgeColor','none')
daspect([l 1 1])
view(3); a x i s ( [ 0 1 0 1 0 1])
[cmin cmax] = c a x i s ;
caxis([cmin-.25*cmax cmax])
colormap hsv
y l i m ( v c b , [0 cmax])
set(get(vcb,'YLabel'),'String','Error')
T h e f o l l o w i n g a l g o r i t h m is a v e r y p r i m i t i v e ( a n d v e r y inefficient) i m p l e m e n t a t i o n
of a n a d a p t i v e t h i n n i n g a l g o r i t h m for s c a t t e r e d d a t a . I t removes 500 p o i n t s a t a
t i m e a n d w r i t e s t h e i n t e r m e d i a t e r e s u l t t o a file.
P r o g r a m C . 1 0 . Thin.m
load('Data2D_Beethoven')
% T h i s loads v a r i a b l e s d s i t e s and r h s
x = dsites(:,1);
y = dsites(:,2);
f igure
triplot(tes,x,y,'g')
f o r 1=1:5
for j=l:500
n = size(dsites,1);
d = zeros(1,n);
for i=l:n
temp = d s i t e s ;
temp(i,:) = ;
[ k , d ( i ) ] = dse archil (temp, d s i t e s ( i , : ) ) ;
i f (k >= i )
k=k+l;
end
end
r = min(d);
idx = find(d==r);
d s i t e s ( i d x ( l ) ,: ) = [] ;
x(idx(l)) = [ ] ;
y(idx(l)) = [ ] ;
r h s ( i d x ( l ) ) = [] ;
end
figure
triplot(tes,x,y,'r')
name = s p r i n t f ('Data2D_Beethoven/ d', 1 ) ; 0
save(name, ' d s i t e s ' , ' r h s ' )

end
C.2 Maple Programs
T h e M L S basis f u n c t i o n s a n d d u a l basis f u n c t i o n s d i s p l a y e d i n C h a p t e r 24 were

c o m p u t e d w i t h t h e f o l l o w i n g M a p l e code.
P r o g r a m C . l l . MLSDualBases .mws
restart; with(plots): with(linalg):

N:=10: m:=3: DD:=4: h:=l/N: ep : = 1 / ( s q r t (DD) *h) :
p h i := ( x , y ) -> e x p ( - e p " 2 * ( x - y ) " 2 ) ;
f o r k from 1 t o m do
p p l l k := p l o t ( x ~ ( k - l ) , x=0..1):
od:
display([seq(ppI|k,k=l..m)],insequence=true,thickness=2);
X := v e c t o r ( [ s e q ( h * k , k=0..N)]);
# or use 11 Halton p o i n t s
# X := vector(CO.5000,0.2500,0.7500,0.1250,0.6250,
# 0.3750,0.8750,0.0625,0.5625,0.3125,0.8125] ) ;
G := matrix(m,m):
f o r i from 1 t o m do
f o r j from 1 t o m do
G [ i , j ] := e v a l f ( a d d ( ( X [ k ] ) ~ ( i - 1 ) * ( X [ k ] ) ~ ( j - l ) *
phi(x,X[k]), k=l..N+l));
od:
od:
P := v e c t o r ( [ e v a l f ( s e q ( y ~ ( k - 1 ) , k = l . . m ) ) ] ) ;
Lambda := l i n s o l v e ( G . P ) :
l | | k := unapply(Lambda [ k ] , ( x , y ) ) ;
od:
l p l l k := p l o t ( l | | k ( x , x ) , x=0..1):
od:
d i s p l a y ( [ s e q ( l p I I k , k=l..m)],insequence=true,thickness=2);
K := ( x , y ) -> p h i ( x , y ) * a d d ( l | | k ( x , x ) * y ~ ( k - 1 ) , k=l..m):
approxK := ( x , y ) -> 1 / s q r t ( D D * P i ) * ( 3 / 2 - e p ~ 2 * ( x - y ) " 2 )
*phi(x,y);
f o r i from 1 t o N+l do
aKplli := p l o t ( [ K ( x , X [ i ] ) , a p p r o x K ( x , X [ i ] ) ] , x=0..1,
color=[green,red]):
od:
display(seq(aKpI|i,i=l..N+l),insequence=true,thickness=2);
Appendix D
Catalog of R B F s with Derivatives
D.l Generic Derivatives
W e p r o v i d e f o r m u l a s for a l l first a n d second-order d e r i v a t i v e s o f r a d i a l f u n c t i o n s o f

t w o variables, i.e., <p(r) (p(\\x\\) = ip(y/x 2
+ y ), 2
w h e r e x = (x,y) R . 2
The
chain rule implies
J ^ ( M I ) = lv(r)^r(x,y)
d , . x
2 2
dr yjx + y
x d . .
r dr
2 2 r - e n e r i c
since r = = y/x + y . S i m i l a r l y , J^>(||ar||) = r ^ ^ ( ) ^ê S second-
o r d e r d e r i v a t i v e s are g i v e n b y
2 2 2
9 2
/,i us d , . ( d . A d . , d . ,
fl^ND = ^ ( r ) (^(*>v)J + - ^ r ) - ^ r ( x , y )
x
2
d 2
, . y 2
d
2 3
dr r dr
as w e l l as
2 2 2 2
d v d x d
2 2 2 3
dy r dr r dr
2 2 3
dxdy r dr r dr
and the Laplacian
2 2 2
d d \ ... ... d , . H M
2 2 2
\ <9a: ch/ / dr r dr
D e r i v a t i v e s o f h i g h e r o r d e r o r i n h i g h e r space d i m e n s i o n s c a n be c o m p u t e d s i m -
i l a r l y b y a p p l y i n g t h e c h a i n r u l e . F o r e x a m p l e , t h e generic f o u r t h - o r d e r b i h a r m o n i c
(or d o u b l e L a p l a c i a n ) t u r n s o u t t o be
3
/ d 4
d 4
d 4
\ d 4
2d I d 2
I d
+ 2
dxW +
W) ^ ' (I X|I) =
^ { r )+
r^ r )
' ^d7^ ( r ) +
ôV^ ( r )
-
443
D.2 F o r m u l a s for S p e c i f i c B a s i c Functions
T h e generic d e r i v a t i v e s o f t h e basic f u n c t i o n w i t h respect t o r i n t h e p r e v i o u s s e c t i o n

need t o be r e p l a c e d b y t h e f o l l o w i n g f o r m u l a s .
D.2.1 Globally Supported, Strictly Positive Definite Functions
E x a m p l e D . l . Gaussian R B F :
dr
2 2 2
J^(r) = 2e e-(-) (2(er) - l) .
T h i s f u n c t i o n is C at t h e o r i g i n .
E x a m p l e D . 2 . Inverse m u l t i q u a d r i c ( I M Q ) R B F :
1
y/l + {er) ' 2
2
d .. er
-if(r)
2 3 2
dr" ' (l + ( r) ) / '
d ..
2
2(er) - 1 22
2
Mr) =e
2
dr (! + ( e r )2)5/2
E x a m p l e D . 3 . Generalized I M Q R B F :
i
tp(r) (l + (er) ) ' 2 2
2
d .. 4e r
Mr)
dr ( l + ( e r ) )3 ' 2
2
d 2
, , .
r 4
2 5( r) - 1
2
M) = v 2 4
dr ^ (1 + ( e r ) ) '
T h i s f u n c t i o n is C at the o r i g i n .
E x a m p l e D . 4 . Inverse q u a d r a t i c ( I Q ) R B F :
1
P(r) = 2
(l + M ) '
2
d .. 2e r
-r^(r) 2 '
dr (i + ( e r ) 2 )
d ,s
2
0 2 3(er) - 1 2
( e r ) )3
2 K
dr ^ (1 + 2
D. Catalog of RBFs with Derivatives 445
E x a m p l e D . 5 . Basic M a t e r n R B F :
r
<p(r) e~ .
T h i s f u n c t i o n is n o t differentiable at t h e o r i g i n .
E x a m p l e D . 6 . Linear M a t e r n R B F :
_ r
(p(r) = e ( l + er),
2 er
-^-tp(r) = e re~ ,
dr
2
| ^ ( r ) = e ( e r - 1).
2
T h i s f u n c t i o n is C at t h e o r i g i n , b u t n o t s m o o t h e r .
E x a m p l e D . 7 . Quadratic Matern R B F :
r 2
y,(r) = e - ( 3 + 3 r + ( e r ) ) ,
2 _ e r
j-<p(r) = e re (l +er),
CX/
2 r 2
j^2<p(r) = e e~ ((er) - er - l ) .
4
E x a m p l e D . 8 . Cubic M a t e r n R B F :
r 2 3
cp(r) = e - ( 1 5 + 15er + 6 ( e r ) + (er) ),
d
2 r 2
<p(r) = -e re~ ((er) + 3er + 3) ,
2 r
^ > ( r ) = e e~ ((erf - 3er - 3) .
6
D.2.2 Globally Supported, Strictly Conditionally Positive Definite

Functions of Order 1
E x a m p l e D . 9 . L i n e a r or n o r m R B F :
(p(r) r.
T h i s f u n c t i o n is n o t differentiable at t h e o r i g i n .
E x a m p l e D . 1 0 . Multiquadric (MQ) RBF:

2
<p(r) = y/l + (er) ,
2
d . . e r
K 2
dr^ y/i + (er) '
2 2
d , . e
np(r)
2 2 3 / 2
dr (1 + (er) )

E x a m p l e D . l l . Generalized M Q R B F :
2 3 2
y>(r) = ( 1 + ( e r ) ) / ,
2 2
4-<p(r) = 3e ry/l + (er) ,
dr
2 2
d . . Q 2
2
2(er) + 1
;(p(r) = 3e
2 2
dr y/i + (er)
E x a m p l e D . 1 2 . Cubic R B F :
3
<p(r) = r,
2
| ( r ) = 3r ,
2
d , ,
2 ;<p{r) = 6 r .
dr
E x a m p l e D . 1 3 . T h i n plate spline ( T P S ) R B F :
2
(p(r) = r log(r),
- ^ ( r ) = r (2 l o g ( r ) + 1 ) ,
2
d
dr Mr)
2 = 21og(r) + 3 .
W h i l e t h e s i n g u l a r i t i e s o f t h e f u n c t i o n a n d first d e r i v a t i v e a t t h e o r i g i n are r e m o v -
able, t h e s i n g u l a r i t y o f t h e second d e r i v a t i v e a t t h e o r i g i n is n o t .

E x a m p l e D . 1 4 . Generalized M Q R B F :
2 5 2
^(r) = ( l + (er) ) / ,
2
- ^ ( r ) = 5e r ( l + 2
{erff ,
dr
J^(r) = 5eVl + (er) 2
(4(er) + l ) . 2
T h i s f u n c t i o n is C a t t h e o r i g i n .
E x a m p l e D . 1 5 . Quintic R B F :
5
= r ,
4
- 5r ,
2
d , x
3
2 r )
= 20r
dr ^
D. Catalog of RBFs with Derivatives 447
E x a m p l e D . 1 6 . Second-order T P S R B F :
4
(p(r) = r log(r),
3
^^(r) = r (41og(r) + l ) ,
2
d
2
< p ( ) = r (121og(r) + 7).
r

E x a m p l e D . 1 7 . Septic R B F :
7
<f(r) = r ,
6
d
drMr) = 7r ,
_ 5
ip(r) = 42r .
2
dr
D.2.6 Globally Supported, Strictly Positive Definite and Oscilla-

tory Functions
2
E x a m p l e D . 1 8 . L i n e a r L a g u e r r e - G a u s s i a n R B F for R :
y>(r) = e - ^ \ 2 - ( s r ) %
2 2 2
îp(r) = 2e re-^ ((er) - 3) ,
2 e r 2 4 2
jj^<p(r) = -2e e-( ) (2(er) - 9(er) + 3) .
2
E x a m p l e D . 1 9 . Q u a d r a t i c L a g u e r r e - G a u s s i a n R B F for R :
p ( r ) = e - ( - ) 2
( 3 _ 3 ( r ) 2 + l ( r ) 4 ) )
2 2 4 2
^ ( r ) = - e r e - ^ ((er) - 8(er) + 12) ,
2 2 4 2
~ ^ ( r ) = e e~^ (2(erf - 21(er) + 48(er) - 12) .
E x a m p l e D . 2 0 . Linear generalized I M Q R B F :
2
2 - (er)
<p(r) 2 4
(l + ( r ) ) '
2
d
( \ 2 (er) - 3
<f(r) = 6e r-
2 5
^ ( l + (er) ) '
2 4 2
d 2 7(gr) -30( r) + 3
(pi \ - 6 c r -~ .
2 y
dr ^ (l + (er) ) 2 6
^ fk fi
E x a m p l e D . 2 1 . Q u a d r a t i c generalized I M Q R B F :
2 4
3 - 6(er) + (er)
tp(r) 2 6
(1 + ( e r ) )
d , s o 2 (er 4
- 8 (er + 6 2
ip(r)y
= -8e^r-- -'-^ ,
d r ( i + ( e r ) 2 ) 7
6 4 2
l_ n M _ 0 / l . 3(er) -31(er)
2 + 34(er) -2
2 r 2 A 2
d r ^ ) - (l + (er) f
D.2.7 Compactly Supported, Strictly Positive Definite Functions

3
E x a m p l e D . 2 2 . W e n d l a n d ' s cps^ ( s t r i c t l y p o s i t i v e definite i n R ) :
<p(r) = (l-er)\.
T h i s f u n c t i o n is n o t differentiable a t t h e o r i g i n .
E x a m p l e D . 2 3 . W e n d l a n d ' s ipz,\ ( s t r i c t l y p o s i t i v e definite i n
tp(r) = ( 1 -er)\(4er + 1),
4-<p(r) = - 2 0 e r ( l - e r ) , 2 3
dr
d?
2 2
^2<p(r) = 20e (4er - 1)(1 - er) ..
2
E x a m p l e D . 2 4 . W e n d l a n d ' s ipz,i ( s t r i c t l y p o s i t i v e definite i n
6 2
<p{r) = ( 1 - er) (35(er) + + 18er + 3 ) ,
4-<p(r) = - 5 6 e r ( 5 e r + 1 ) ( 1 - 2
er) ,,5
dr
2
d
2 2 4
2 : <p(r) = 5 6 e ( 3 5 ( e r ) - 4er - l ) (1 - e r ) .
dr
4
E x a m p l e D . 2 5 . W e n d l a n d ' s (^3,3 ( s t r i c t l y p o s i t i v e definite i n R ) 3
3 2
cp(r) = ( 1 - er)5_(32(er) + 2 5 ( e r ) + 8er + 1),
4~<p(r) = - 2 2 e r ( l 6 ( e r ) + 7er + l ) ( 1 - 2 2 7
er) ,,
dr
2
d
2 3 2
2 ip(r) = Tie (I60(er) + 15(er) - 6er - l ) (1 - er)\.
dr
6
T h i s f u n c t i o n is C at t h e o r i g i n
Catalog of RBFs with Derivatives 449
7
E x a m p l e D . 2 6 . W u ' s -03,3 ( s t r i c t l y p o s i t i v e definite i n R ) :
3 2
(p(r) = ( 1 - er)\(5(er) + 20(er) + 29er + 1 6 ) .
T h i s f u n c t i o n is n o t differentiable a t t h e o r i g i n .
E x a m p l e D . 2 7 . W u ' s ^2,3 ( s t r i c t l y p o s i t i v e definite i n R ) : 5
4 3 2
(p(r) = ( 1 - er)%(5(er) + 25(er) + 48(er) + 40er + 8 ) ,
2 3 2 4
< ^ ( r ) = -9e r ( 5 ( e r ) + 2 0 ( e r ) + 29er + 16) ( 1 - er) ,
dr
2
d 2 4 3 2 3
- ^ t p ( r ) = 1 8 e ( 2 0 ( e r ) + 6 0 ( e r ) + 5 7 ( e r ) + Her - 8) ( 1 -
z
er) ,.
dr
2
T h i s f u n c t i o n is C at the origin.
E x a m p l e D . 2 8 . W u ' s ^1,3 ( s t r i c t l y p o s i t i v e definite i n R ) : 3
5 4 2
(p(r) = ( 1 - er)%(5(er) + 30(er) + 12(erf + 82{er) + S6er + 6 ) ,
2 3 2
îp(r) = - l l e r ( e r + 2) (5(er) + 15(er) + 18er + 4) ( 1 - er)%,
Co/
2
rl
2 5 4 3 2
-f?(r) = 22e (25(er) + lOO(er) + 142(er) + 6 8 ( e r )
2
- 16er - 4) ( 1 - er)%.
dr
4
T h i s f u n c t i o n is C at the o r i g i n .
E x a m p l e D . 2 9 . W u ' s -0n,3 ( s t r i c t l y p o s i t i v e definite i n R ) :

7 6 5 4 3 2
(p(r) = ( 1 - er) (5(er) + + 35(er) + 101(er) + 147(er) + 101(er) + 35er + 5 ) ,
4-ip(r) = -13e r 2
(b(er) 5
+ S0(er) 4
+ 72(er) 3
+ 82(er) 2
+ 3 6 e r + 6) ( 1 - er)%,
dr
2 6 5 4 3 2
^<p(r)
z
= 7 8 e ( l O ( e r ) + 50{er) + 95(er) + 75(er) + 7(er) - her - l ) ( 1 - er)%.
dr
6
T h i s f u n c t i o n is C a t t h e o r i g i n , b u t o n l y s t r i c t l y p o s i t i v e definite i n R .
E x a m p l e D . 3 0 . E u c l i d ' s h a t <pi:
<p{r) = (l-er/2)+.
N o n e o f t h e E u c l i d ' s h a t f u n c t i o n s are differentiable a t t h e o r i g i n .

I s
Bibliography
Abramowitz, M. and Stegun, I. A. (1972). Handbook of Mathematical Functions with For-

mulas, Graphs, and Mathematical Tables, Dover (New York).
Acosta, F. M. (1995). Radial basis functions and related models, Signal Proc. 45, pp. 37
58.
Adams, R. (1975). Sobolev Spaces, Academic Press (New York).
Alfeld, P. (1989). Scattered data interpolation in three or more variables, in Mathematical
Methods in Computer Aided Geometric Design, T. Lyche and L. Schumaker (eds.),
Academic Press (New York), pp. 1-33.
Allasia, G. and Giolito, P. (1997). Fast evaluation of cardinal radial basis interpolants, in
Surface Fitting and Multiresolution Methods, A. Le Mehaute, C. Rabut, and L. L.
Schumaker (eds.), Vanderbilt University Press (Nashville, TN), pp. 1-8.
Allison, J. (1993). Multiquadratic radial basis functions for representing multidimensional
high energy physics data, Comp. Phys. Comm. 77, pp. 377-395.
Alves, C. J. S. and Silvestre, A. L. (2004). Density results using Stokeslets and a method of
fundamental solutions for the Stokes equations, Engineering Analysis with Boundary
Elements 28, pp. 1245-1252.
Andrews, G. E., Askey, R. and Roy, R. (1999). Special Functions, Cambridge University
Press (Cambridge).
Arad, N., Dyn, N., Reisfeld, D. and Yeshurun, Y. (1994). Image warping by radial basis
functions: applications to facial expressions, CVGIP: Graphical models and image
processing 56, pp. 161-172.
Armentano, M. G. (2001). Error estimates in Sobolev spaces for moving least square
approximations, SIAM J. Numer. Anal. 39 1, pp. 38-51.
Arnott, R. (1993). An adaptive radial basis function diversity combiner for multipath
channels, IEE Electronics Letters 29, pp. 1092-1094.
Aronszajn, N. (1950). Theory of reproducing kernels, Trans. Amer. Math. Soc. 68, pp. 337
404.
Askey, R. (1973). Radial characteristic functions, T S R #1262, University of Wisconsin-
Madison.
Askey, R. (1975). Orthogonal Polynomials and Special Functions, Reg. Conf. Ser. in Appl.
Math. (SIAM).
Atluri, S. N. (2004). The Meshless Methods (MLPG) For Domain & BIE Discretizations,
Tech Science Press (Encino, CA).
Atluri, S. N. and Shen, S. (2002a). The Meshless Local Petrov-Galerkin (MLPG) Method,
Tech Science Press (Encino, CA).
Atluri, S. N. and Shen, S. (2002b). The meshless local Petrov-Galerkin (MLPG) method: a
451
simple & less costly alternative to the finite element and boundary element methods,
Comput. Model. Eng. Sci. 3, pp. 11-51.
Atluri, S. N. and Zhu, T. (1998). A new meshless local Petrov-Galerkin (MLPG) approach
in computational mechanics, Comput. Mech. 22, pp. 117-127.
Babuska, I., Banerjee U. and Osborn, J. E. (2003). Survey of meshless and generalized
finite element methods: A unified approach, Acta Numerica 12, pp. 1-125.
Babuska, I. and Melenk, M. (1997). The partition of unity method, Int. J. Numer. Meths.
Eng. 40, pp. 727-758.
Backus, G. and Gilbert, F. (1968). The resolving power of gross earth data, Geophys. J.
R. Astr. Soc. 16, pp 169-205.
Ball, K. (1992). Eigenvalues of Euclidean distance matrices, J. Approx. Theory 68, pp. 74
82.
Ball, K., Sivakumar, N. and Ward, J. D. (1992). On the sensitivity of radial basis inter-
polation to minimal data separation distance, Constr. Approx. 8, pp. 401-426.
Barnes, R. J. (1994). A modified conjugate gradient algorithm for scattered data interpo-
lation using radial basis functions, J. Appl. Sci. Comput. 1, pp. 227-238.
Barnhill, R. E. and Ou, H. S. (1990). Surfaces defined on surfaces, Comput. Aided Geom.
Design 7, pp. 323-336.
Barron, A. R. (1993). Universal approximation bounds for superpositions of a sigmoidal
function, IEEE Trans. Inform. Theory 39, pp. 930-945.
Bates, D., Lindstrom, M., Wahba, G. and Yandell, B. (1987). G C V P A C K routines for
generalized cross validation, Commun. Statist. B Simulation and Computation
16, pp. 263-297.
Bates, D. and Wahba, G. (1982). Computational methods for generalized cross valida-
tion with large data sets, in Treatment of Integral Equations by Numerical Methods,
C. T. H. Baker, and G. F. Miller (eds.), Academic Press (New York), pp. 283-296.
Baule, R. (2000). Moving Least Squares Approximation mit parameterabhangigen
Gewichtsfunktionen, Diplomarbeit, Universitat Gottingen.
Baxter, B. J. C. (1991). Conditionally positive functions and p-norm distance matrices,
Constr. Approx. 7, pp. 427-440.
Baxter, B. J. C. (1992a). Norm estimates for inverses of distance matrices, in Mathematical
Methods in Computer Aided Geometric Design II, T. Lyche and L. Schumaker (eds.),
Baxter, B. J. C. (1992b). The asymptotic cardinal function of the multiquadric <p(r) =
2 2
(r + c ) as c > oo, Comput. Math. Appl. 24, pp. 1-6.
1 / 2
Baxter, B. J. C. (1992c). The interpolation theory of radial basis functions, Ph.D. Disser-
tation, University of Cambridge.
Baxter, B. J. C. (2002). Preconditioned conjugate gradients, radial basis functions, and
Toeplitz matrices, Comput. Math. Appl. 43, pp. 305318.
Baxter, B. J. C. (2006). Scaling radial basis functions via Euclidean distance matrices,
Comput. Math. Appl. 51 8, pp. 1163-1170.
Baxter, B. J. C. and Hubbert, S. (2001). Radial basis functions for the sphere, in Recent
progress in multivariate approximation, Internat. Ser. Numer. Math., 137, Birkhauser
(Basel), pp. 33-47.
Baxter, B. J. C. and Roussos, G. (2002). A new error estimate of the fast Gauss transform,
SIAM J. Sci. Comput. 24 1, pp. 257-259.
Baxter, B. J. C. and Sivakumar, N. (1996). On shifted cardinal interpolation by Gaussians
and multiquadrics, J. Approx. Theory 8 7 , pp. 36-59.
Baxter, B. J. C , Sivakumar, N. and Ward, J. D. (1994). Regarding the p-norms of radial
basis interpolation matrices, Constr. Approx. 10, pp. 451-468.
Bibliography 453
Beatson, R. K., Cherrie, J. B. and Mouat, C. T. (1999). Fast fitting of radial basis func-
tions: methods based on preconditioned GMRES iteration, Adv. Comput. Math. 11,
pp. 253-270.
Beatson, R. K. and Bui, H.-Q. (2003). Mollification formulas and implicit smoothing,
Research Report UCDMS 2003/19, University of Canterbury.
Beatson, R. K., Bui, H.-Q. and Levesley, J. (2005). Embeddings of Beppo-Levi spaces in
Holder-Zygmund spaces, and a new method for radial basis function interpolation
error estimates, J. Approx. Theory 137, pp. 166-178.
Beatson, R. K. and Dyn, N. (1996). Multiquadric B-splines, J. Approx. Theory 87, pp. 1
24.
Beatson, R. K., Goodsell, G. and Powell, M. J. D. (1996). On multigrid techniques for
thin plate spline interpolation in two dimensions, in The Mathematics of Numerical
Analysis, Lectures in Appl. Math., 32, Amer. Math. Soc. (Providence, RI), pp. 77-97.
Beatson, R. K. and Greengard, L. (1997). A short course on fast multipole methods, in
Wavelets, Multilevel Methods and Elliptic PDEs (Leicester, 1996), M. Ainsworth, J.
Levesley, M. Marietta and W. A. Light (eds.), Numer. Math. Sci. Comput., Oxford
Univ. Press (New York), pp. 1-37.
Beatson, R. K. and Langton, M. K. (2006). Integral interpolation, in Algorithms for Ap-
proximation V, A. Iske and J. Levesley (eds.), Springer-Verlag, Heidelberg, pp. 199
218.
Beatson, R. K. and Levesley, J. (2002). Good point/bad point iterations for solving the
thin-plate spline interpolation equations, in Approximation Theory X, C. K. Chui,
L. L. Schumaker, and J. Stockier (eds.), Vanderbilt Univ. Press (Nashville, TN),
pp. 17-25.
Beatson, R. K. and Light, W. A. (1992). Quasi-interpolation in the absence of polynomial
reproduction, in Numerical Methods in Approximation Theory, ISNM 105, D. Braess,
L. L. Schumaker (ed.), Birkhauser (Basel), pp. 21-39.
Beatson, R. K. and Light, W. A. (1993). Quasi-interpolation with thin plate splines on a
square, Constr. Approx. 9, pp. 407-433.
Beatson, R. K. and Light, W. A. (1997). Fast evaluation of radial basis functions: methods
for two-dimensional polyharmonic splines, IMA J. Numer. Anal. 17, pp. 343372.
Beatson, R. K., Light, W. A. and Billings, S. (2000). Fast solution of the radial basis func-
tion interpolation equations: domain decomposition methods, SIAM J. Sci. Comput.
22, pp. 1717-1740.
Beatson, R. K. and Newsam, G. N. (1992). Fast evaluation of radial basis functions: I,
Comput. Math. Appl. 24, pp. 7-19.
Beatson, R. K. and Newsam, G. N. (1998). Fast evaluation of radial basis functions:
Moment based methods, SIAM J. Sci. Comput. 19, pp. 1428-1449.
Beatson, R. K. and Powell, M. J. D. (1992a). Univariate multiquadric approximation:
quasi-interpolation to scattered data, Constr. Approx. 8, pp. 275-288.
Beatson, R. K. and Powell, M. J. D. (1992b). Univariate interpolation on a regular finite
grid by a multiquadric plus a linear polynomial, J. Inst. Math. Applies. 12, pp. 107
133.
Beatson, R. K. and Powell, M. J. D. (1993). An iterative method for thin plate spline inter-
polation that employs approximations to Lagrange functions, in Numerical Analysis
1993 (Dundee, 1993), G. A. Watson and D. F. Griffiths (ed.), Pitman Res. Notes
Math. Ser., 303, Longman Sci. Tech. (Harlow), pp. 17-39.
Behrens, J., Iske, A. and Kaser, M. (2002). Adaptive meshfree method of backward charac-
teristics for nonlinear transport equations, in Lecture Notes in Computer Science and
Engineering Vol.26: Meshfree Methods for Partial Differential Equations, M. Griebel
and M. A. Schweitzer (eds.), Springer Verlag, pp. 21-36.
Bejancu, A. (1999)., Local accuracy for radial basis function interpolation on finite uniform
grids, J. Approx. Theory 99, pp. 242-257.
Belytschko, T., Krongauz, Y., Organ, O., Fleming, M. and Krysl, P. (1996). Meshless
methods: an overview and recent developments, Comp. Meth. Appl. Mech. Eng.
139, pp. 3-47.
Ben Moussa, B., Lanson, N. and Vila, J. P. (1999). Convergence of meshless methods for
conservation laws. Applications to Euler equations, in Hyperbolic problems: The-
ory, numerics, applications. Proceedings of the 7th international conference, Zurich,
Switzerland, February 1998. Vol. I, Fey, Michael et al. (eds.), Birkhauser, Basel,
pp. 31-40.
Berg, C , Christensen, J. P. R. and Ressel, P. (1984). Harmonic Analysis on Semigroups,
Springer (Berlin).
Berlinet, A. and Thomas-Agnan, C. (2004). Reproducing Kernel Hilbert Spaces in Proba-
bility and Statistics, Kluwer (Dordrecht).
Bernal, F. and Kindelan, M. (2006). Meshless simulation of Hele-Shaw flow, submitted.
Beyer, A. (1994). Optimale Centerverteilung bei Interpolation mit radialen Basisfunktio-
nen, Diplomarbeit, Universitat Gottingen.
Binev, P. and Jetter, K. (1992). Estimating the condition number for multivariate inter-
polation problems, in Numerical Methods in Approximation Theory, ISNM 105, D.
Braess, L. L. Schumaker (ed.), Birkhauser (Basel), pp. 41-52.
Bingham, N. H. (1973). Positive definite functions on spheres, Proc. Camb. Phil. Soc. 73,
pp. 145-156.
Bishop, C. (1991). Improving the generalization properties of radial basis function neural
networks, Neural Computation 3, pp. 579-588.
Blumenthal, L. M. (1938). Distance Geometries, Univ. of Missouri Studies, 13, 142pp.
Bochner, S. (1932). Vorlesungen iiber Fouriersche Integrale, Akademische Verlagsge-
sellschaft (Leipzig).
Bochner, S. (1933). Monotone Funktionen, Stieltjes Integrale und harmonische Analyse,
Math. Ann. 108, pp. 378-410.
Bochner, S. (1941). Hilbert distances and positive definite functions, Ann. of Math. 42,
pp. 647-656.
deBoor, C. (1993). Approximation order without quasi-interpolants, in Approximation
Theory VII, E. W. Cheney, C. Chui, and L. Schumaker (eds.), Academic Press (New
York), pp. 1-18.
de Boor, C. On interpolation by radial polynomials, Adv. in Comput. Math. 24, pp. 143
153.
deBoor, C , DeVore, R. A. and Ron, A. (1994a). Approximation from shift-invariant sub-
spaces of L2(Rd), Trans. Amer. Math. Soc. 341, pp. 787-806.
deBoor, C , DeVore, R. A. and Ron, A. (1994b). The structure of finitely generated shift-
invariant spaces in Z/20R ), J- Fund. Anal. 119, pp. 37-78.
d
deBoor, C. and Ron, A. (1990). On multivariate polynomial interpolation, Constr. Approx.

6, pp. 287-302.
deBoor, C. and Ron, A. (1992a). The least solution for the polynomial interpolation
problem, Math. Z. 210, pp. 347-378.
deBoor, C. and Ron, A. (1992b). Fourier analysis of the approximation power of principal
shift-invariant spaces, Constr. Approx. 8, pp. 427-462.
Bors, A.G. and Pitas, I. (1996). Median radial basis function neural network, IEEE Trans.
Neural Networks 7, pp. 1351-1364.
Bos, L. P. and Maier, U. (2002). On the asymptotics of Fekete-type points for univariate
radial basis interpolation, J. Approx. Theory 119, pp. 252-270.
Bibliography 455
3
Bos, L. P. and Salkauskas, K. (1987). On the matrix [\xi Xj\ ] and the cubic spline
continuity equations, J. Approx. Theory 51, pp. 81-88.
Bos, L. P. and Salkauskas, K. (1989). Moving least-squares are Backus-Gilbert optimal, J.
Approx. Theory 59, pp. 267-275.
Bouhamidi, A. and Le Mehaute, A. (2004). Radial basis functions under tension, J. Approx.
Theory 127, pp. 135-154.
Bozzini, M., Lenarduzzi, L. and Schaback, R. (2002). Adaptive interpolation by scaled
multiquadrics, Adv. in Comp. Math. 16, pp. 375-387.
Bozzini, M., Lenarduzzi, L. and Schaback, R. (2006). Kernel Z?-splines and interpolation,
Numer. Algorithms 41 1, pp. 1-16.
Braess, D. (1997). Finite Elements: Theory, Fast Solvers, and Applications in Solid Me-
chanics, Cambridge University Press (Cambridge).
Brenner, S. C. and Scott, L. R. (1994). The Mathematical Theory of Finite Element Meth-
ods, Springer Verlag (New York).
Broomhead, D. S. and Lowe, D. (1988). Multivariate functional interpolation and adaptive
networks, Complex Systems 2, pp. 321-355.
Brown, A. L. (1992). Uniform approximation by radial basis functions, Appendix B to
Radial basis functions in 1990, in Advances in Numerical Analysis II: Wavelets,
Subdivision, and Radial Basis Functions, W. Light (ed.), Oxford University Press
(Oxford), pp. 203-206.
Brown, D., Ling, L., Kansa, E. and Levesley, J. (2005). On approximate cardinal precondi-
tioning methods for solving PDEs with radial basis functions, Engineering Analysis
with Boundary Elements 29, pp. 343-353.
Brownlee, R. and Light, W. (2004). Approximation orders for interpolation by surface
splines to rough functions, IMA J. Numer. Anal. 24, pp. 179-192.
Buckley, M. J. (1994). Fast computation of a discretized thin-plate smoothing spline for
image data, Biometrika 81, pp. 247-258.
Buhmann, M. D. (1988). Convergence of univariate quasi-interpolation using multi-
quadrics, IMA J. Numer. Anal. 8, pp. 365-383.
Buhmann, M. D. (1989a). Multivariate interpolation using radial basis functions, Ph.D.
Dissertation, University of Cambridge.
Buhmann, M. D. (1989b). Cardinal interpolation with radial basis functions: an inte-
gral approach, in Multivariate Approximation Theory IV, ISNM 90, C. Chui, W.
Schempp, and K. Zeller (eds.), Birkhauser Verlag (Basel), pp. 41-64.
Buhmann, M. D. (1990a). Multivariate interpolation in odd-dimensional Euclidean
spaces using multiquadrics, Constr. Approx. 6, pp. 2134.
Buhmann, M. D. (1990b). Multivariate cardinal interpolation with radial basis functions,
Buhmann, M. D. (1993a). New developments in the theory of radial basis function inter-
polation, in Multivariate Approximation: From CAGD to Wavelets, Kurt Jetter and
Florencio Utreras (eds.), World Scientific Publishing (Singapore), pp. 35-75.
Buhmann, M. D. (1993b). On quasi-interpolation with radial basis functions, J. Approx.
Theory 72, pp. 103-130.
Buhmann, M. D. (1993c). Discrete least squares approximation and prewavelets from radial
function spaces, Proc. Camb. Phil. Soc. 114, pp. 533-558.
Buhmann, M. D. (1995a). Multiquadric prewavelets on nonequally spaced knots in one
dimension, Math. Comp. 64, pp. 1611-1625.
Buhmann, M. D. (1995b). Pre-wavelets on scattered knots and from radial function spaces:
a review, in The Mathematics of Surfaces VI, R. R. Martin (ed.), Clarendon Press
(Oxford), pp. 309-324.
Buhmann, M. D. (1998). Radial functions on compact support, Proc. Edin. Math. Soc. II
41, pp. 33-46.
Buhmann, M. D. (2000). Radial basis functions, Acta Numerica 2000 9, pp. 1-38.
Buhmann, M. D. (2003). Radial Basis Functions: Theory and Implementations, Cambridge
University Press (Cambridge).
Buhmann, M. D. (2006). Half-plane approximation with radial functions I, Comput. Math.
Appl. 51 8, pp. 1171-1184.
Buhmann, M. D. and Chui, C. K. (1993). A note on the local stability of translates of
radial basis functions, J. Approx. Theory 74, pp. 36-40.
Buhmann, M. D., Derrien, F. and Le Mehaute, A. (1995). Spectral properties and knot
removal for interpolation by pure radial sums, in Mathematical Methods for Curves
and Surfaces, M. Dffihlen, T. Lyche, and L. Schumaker (eds.), Vanderbilt University
Press (Nashville), pp. 55-62.
Buhmann, M. D. and Dyn, N. (1991). Error estimates for multiquadric interpolation, in
Curves and Surfaces, P.-J. Laurent, A. Le Mehaute, and L. L. Schumaker (eds.),
Buhmann, M. D. and Dyn, N. (1993). Spectral convergence of multiquadric interpolation,
Proc. Edinburgh Math. Soc. 36, pp. 319-333.
Buhmann, M. D., Dyn, N. and Levin, D. (1993). On quasi-interpolation with radial basis
functions on non-regular grids, Constr. Approx. 11, pp. 239254.
Buhmann, M. D. and Le Mehaute, A. (1995). Knot removal with radial basis function
interpolantion, C. R. Acad. Sci. Paris Ser. I Math. 320, pp. 501-506.
Buhmann, M. D. and Micchelli, C. A. (1989). Completely monotonic functions for cardinal
interpolation, in Approximation Theory VI, C. Chui, L. Schumaker, and J. Ward
(eds.), Academic Press (New York), pp. 1-4.
Buhmann, M. D. and Micchelli, C. A. (1991). Multiply monotone functions for cardinal
interpolation, Adv. in Appl. Math. 12, pp. 358-386.
Buhmann, M. D. and Micchelli, C. A. (1992a). Multiquadric interpolation improved, Com-
put. Math. Appl. 24, pp. 21-25.
Buhmann, M. D. and Micchelli, C. A. (1992b). On radial basis approximation on periodic
grids, Proc. Camb. Phil. Soc. 112, pp. 317-334.
Buhmann, M. D. and Pinkus, A. (1997). On a recovery problem, Annals of Numerical
Mathematics 4, pp. 129-142,
Buhmann, M. D. and Powell, M. J. D. (1990). Radial basis function interpolation on an
infinite regular grid, in Algorithms for Approximation II, M. G. Cox and J. C. Mason
(eds.), Chapman & Hall (London), pp. 146-169.
p
Buhmann, M. D. and Ron, A. (1994). Radial basis functions: L -approximation orders
with scattered centres, in Wavelets, Images, and Surface Fitting, P.-J. Laurent, A.
Le Mehaute, and L. L. Schumaker (eds.), A. K. Peters (Wellesley, MA), pp. 93-112.
Caiti, A., Magenes, G., Panisini, T. and Simpson, R. (1994). Smooth approximation by
radial basis functions: three case studies, J. Appl. Sc. Comp. 1, pp. 88-113.
Cambanis, S., Keener, R. and Simons, G. (1983). On a-symmetric multivariate distribu-
tions, J. Multivariate Anal. 13, pp.213233.
Canuto, C , Hussaini, M. Y., Quarteroni, A. and Zang, T. A. (1988). Spectral Methods in
Fluid Dynamics, Springer Verlag, Berlin.
Carlson, R. E. and Foley, T. A. (1991). The parameter R2 in multiquadric interpolation,
Carlson, R. E. and Foley, T. A. (1992). Interpolation of track data with radial basis
methods, Comput. Math. Appl. 24, pp. 27-34.
Bibliography 457
Carlson, R. E. and Natarajan, B. K. (1994). Sparse approximate multiquadric interpola-

tion, Comput. Math. Appl. 27, pp. 99-108.
Carr, J. C, Beatson, R. K., Cherie, J. B., Mitchell, T. J., Fright, W. R., McCallum, B.
C. and Evans, T. R. (2001). Reconstruction and representation of 3D objects with
radial basis functions, in SIGGRAPH 2001, pp. 67-76.
Carr, J. C , Fright, W. R. and Beatson, R. K. (1997). Surface Interpolation with Radial
Basis Functions for Medical Imaging, IEEE Transactions on Medical Imaging 16,
pp. 96-107.
Casciola, G., Lazzaro, D., Montefusco, L. B. and Morigi, S. (2006). Shape preserving
surface reconstruction using locally anisotropic R B F interpolants, Comput. Math.
Appl. 51 8, pp. 1185-1198.
Cha, I. and Kassam, S. A. (1992). Blind equalisation using radial basis function networks,
Proc. Canadian Conference Elec. Eng. and Comp. Sci. 1, TM.3.13.14.
Cha, I. and Kassam, S. A. (1995a). Channel equalization using adaptive complex radial
basis function networks, IEEE J. on Selected Areas in Communications 13, pp. 121
131.
Cha, I. and Kassam, S. A. (1995b). Interference canellation using radial basis function
networks, EURASIP Signal Processing 147, pp. 247-268.
Chakravarthy, S. V. and Ghosh, J. (1996). Scale-based clustering using the radial basis
function network, IEEE Trans. Neural Networks 7, pp. 1250-1261.
Chan, A. K., Chui, C. K. and Guan, L. T. (1990). Radial basis function approach to
interpolation of large reflecting surfaces, in Curves and Surfaces in Computer Vision
and Graphics, L. A. Ferrari and R. J. P. de Figueiredo (eds.), SPIE (Vol. 1251),
pp. 62-72.
Chan, R. and Strang, G. (1989). Toeplitz equations by conjugate gradients with circulant
preconditioners, SIAM J. Sci. Statist. Comput. 10, pp. 104-119.
Chan, T. F. and Foulser, D. E. (1988). Effectively well-conditioned linear systems, SIAM
J. Sci. Statist. Comput. 9, pp. 963-969.
Chang, K. F. (1996). Strictly positive definite functions, J. Approx. Theory 87, 148-158.
Addendum: J. Approx. Theory 88 1997, p. 384.
Chen, C. S. and Brebbia, C. A. (1998). The dual reciprocity method for Helmholtz-type
operators, in Boundary Elements XX, A. Kassab, C. A. Brebbia and M. Chopra
(eds.), Computational Mechanics Publications, pp. 495-504.
Chen, C. S., Brebbia, C. A. and Power, H. (1999). Boundary element methods using com-
pactly supported radial basis functions, Comm. Numer. Meths. Eng. 15, pp. 137
150.
Chen, C. S., Ganesh, M., Golberg, M. A. and Cheng, A. H.-D. (2002). Multilevel compact
radial functions based computational schemes for some elliptic problems, Comput.
Math. Appl. 43, pp. 359-378.
Chen, C. S., Golberg, M. A. and Schaback, R. (2003). Recent developments in the dual
reciprocity method using compactly supported radial basis functions, in Transfor-
mation of Domain Effects to the Boundary, Y. F. Rashed and C. A. Brebbia (eds.),
W I T Press, Southampton, Boston, pp. 138-225.
Chen, J. S., Wu, C. T., Yoon, S. and You, Y. (2001). A stabilized conforming nodal integra-
tion for Galerkin meshfree methods, Int. J. Numer. Meth. Engng. 50 2, pp. 435-466.
Chen, J.-T., Lee, Y.-T., Chen, I-L. and Chen, K.-H. (2004). Mathematical analysis and
treatment for the true and spurious eigenequations of circular plates by the meshless
method using radial basis function, J. Chinese Inst. Engineers 27, pp. 547-561.
Chen, S. (1994). Radial basis functions for signal prediction and system modelling, J. Appl.
Sci. Comput. 1, pp. 172-207.
Chen, S., Billings, S. A., Cowan, C. F. N. and Grant, P. M. (1990). Non-linear systems
identification using radial basis functions, Int. J . Systems Science 21, pp. 2513-2539.
Chen, S., Billings, S. A. and Grant, P. M. (1992). Recursive hybrid algorithm for non-
linear system identification using radial basis function networks, Int. J . Control 55,
pp. 1051-1070.
Chen, S., Cowan, C. F. N. and Grant, P. M. (1991a). Orthogonal least squares learn-
ing algorithm for radial basis function networks, IEEE Trans. Neural Networks 2,
pp. 302-309.
Chen, S., Gibson, G. J., Cowan, C. F. N. and Grant, P. M. (1991b). Reconstruction of
binary signals using an adaptive radial-basis-function equaliser, EURASIP Signal
Processing 22, pp. 77-93.
Chen, S., McLaughlin, S., Grant, P. M. and Mulgrew, B. (1993). Reduced-complexity
multistage blind clustering equaliser, IEEE Int. Communications Conference (ICC)
Proceedings, pp. 11491153.
Chen, S., McLaughlin, S. and Mulgrew, B. (1994a). Complex valued radial basis function
networks: network architecture and learning algorithms (part I), EURASIP Signal
Processing J . 25, pp. 19-31.
Chen, S., McLaughlin, S. and Mulgrew, B. (1994b). Complex valued radial basis func-
tion networks: application to digital communications channel equalisation (part II),
EURASIP Signal Processing J . 36, pp. 175-188.
Chen, S. and Mulgrew, B. (1992). Overcoming co-channel interference using an adaptive
radial basis function equaliser, EURASIP Signal Processing J . 28, pp. 91-107.
Chen, T. and Chen, H. (1995a). Approximation capability to functions of several variables,
nonlinear functionals and operators by radial basis function neural networks, IEEE
Trans. Neural Networks 6, pp. 904-910.
2 n
Chen, T. and Chen, H. (1995b). Denseness of radial-basis functions in L ( M ) and its appli-
cations in neural networks, in Approximation Theory VIII, Vol. 1: Approximation
and Interpolation, C. Chui, and L. Schumaker (eds.), World Scientific Publishing
(Singapore), pp. 137-144.
Cheney, E. W. (1992). Approximation by functions of nonclassical form, in Approximation
Theory, Spline Functions and Applications, S. P. Singh (ed.), Kluwer (Dordrecht),
pp. 1-18.
Cheney, E. W. (1995a). Approximation and interpolation on spheres, in Approximation
Theory, Wavelets and Applications, S. P. Singh (ed.), Kluwer (Dordrecht), pp. 47-53.
Cheney, E. W. (1995b). Approximation using positive definite functions, in Approximation
Theory VIII, Vol. 1: Approximation and Interpolation, C. Chui, and L. Schumaker
(eds.), World Scientific Publishing (Singapore), pp. 145-168.
Cheney, E. W. and Light, W. A. (1999). A Course in Approximation Theory, Brooks/Cole
(Pacific Grove, CA).
Cheney, E. W., Light, W. A. and Xu, Y. (1992). On kernels and approximation orders,
in Proc. Sixth Southeastern Approximation Theory Conference, George Anastassiou
(ed.), Lecture Notes in Pure and Applied Mathematics, Vol. 138, pp. 227-242.
Cheney, E. W. and Xu, Y. (1993). A set of research problems in approximation the-
ory, in Topics in Polynomials of One and Several Variables and Their Applications,
T. M. Rassias, H. M. Srivastava, and A. Yanushauskas (eds.), World Scientific Pub-
lishers (London), pp. 109-123.
Cheng, A. H. D., Golberg, M. A., Kansa, E. J. and Zammito, G. (2003). Exponential con-
vergence and H-c multiquadric collocation method for partial differential equations,
Numer. Methods Partial Differential Equations 19 5, pp. 571-594.
Cheng, A. H.-D. and Cabral, J. J. S. P. (2005). Direct solution of certain ill-posed boundary
Bibliography 459
value problems by collocation method, in Boundary Elements XXVII, A. Kassab, C.

A. Brebbia, E. Divo, and D. Poljak (eds.), W I T Press (Southampton), pp. 35-44.
Cheng, C. C. and Zheng, Y. F. (1994). Thin plate spline surface approximation using
Coons' patches, Comput. Aided Geom. Design 11, pp. 269-287.
Cherrie, J. B., Beatson, R. K. and Newsam, G. N. (2002). Fast evaluation of radial basis
functions: methods for generalized multiquadrics in R , SIAM J. Sci. Comput. 23,
n
pp. 1549-1571.
Chien, C.-C. and Wu, T.-Y. (2004). An accurate solution to the meshless local Petrov-
Galerkin formulation in elastodynamics, J. Chinese inst. Engineers 27, pp. 463-471.
Chng, E. S., Chen, S. and Mulgrew, B. (1996). Gradient radial basis function networks for
nonlinear and nonstationary time series prediction, IEEE Trans. Neural Networks
7, pp. 190-194.
Chng, E. S., Chen, S., Mulgrew, B. and Gibson, G. J. (1995). Realising the Bayesian
decision boundary for channel equalisation using radial basis function network and
linear equaliser, in Mathematics for Neural Networks and Applications, Oxford.
Chui, C. K. (1988). Multivariate Splines, CBMS-NSF Reg. Conf. Ser. in Appl. Math. 54
(SIAM).
Chui, C. K., Stockier, J. and Ward, J. D. (1996). Analytic wavelets generated by radial
functions, Adv. Comput. Math. 5, pp. 95-123.
Chui, C. K., Ward, J. D. and Jetter, K. (1992). Cardinal interpolation with differences of
tempered functions, Comput. Math. Appl. 24, pp. 35-48.
Chung, K. C. and Yao, T. H. (1977). On lattices admitting unique Lagrange interpolations,
SIAM J. Numer. Anal. 14, pp. 735-743.
Cid-Sueiro, J., Artes-Rodriguez, A. and Figueiras-Vidal, A. R. (1994). Recurrent radial ba-
sis function networks for optimal symbol-by-symbol equalisation, EURASIP Signal
Processing 40, pp. 53-63.
Cid-Sueiro, J. and Figueiras-Vidal, A. R. (1993). Recurrent radial basis function networks
for optimal equalisation, Proc. IEEE Workshop on Neural Networks for Signal Pro-
cessing, pp. 562-571.
Cleveland, W. S. and Loader, C. L. (1996). Smoothing by local regression: Principles and
methods, in Statistical Theory and Computational Aspects of Smoothing, W. Haerdle
and M. G. Schimek (eds.), Springer (New York), pp. 10-49.
Cuppens, R. (1975). Decomposition of Multivariate Probability, Academic Press (New
York).
Curtis, P. C, Jr. (1959) n-parameter families and best approximation, Pacific J. Math. 9,
pp. 1013-1027.
Davidov, O., Sestini, A. and Morandi, R. (2005). Local R B F approximation for scattered
data fitting with bivariate splines, in Trends and Applications in Constructive Ap-
proximation, M. G. de Bruin, D. H. Mache, and J. Szabados (eds.), Birkhauser
(Basel), pp. 91-102.
De Forest, E. L. (1873). On some methods of interpolation applicable to the graduation
of irregular series, Annual Report of the Board of Regents of the Smithsonian Insti-
tution for 1871, pp. 275-339.
De Forest, E. L. (1874). Additions to a memoir on methods of interpolation applicable
to the graduation of irregular series, Annual Report of the Board of Regents of the
Smithsonian Institution for 1873, pp. 319-353.
Delvos, F. J. (1985). Convergence of interpolation by translation, Colloq. Math. Soc. J.
Bolya 49, pp. 273-287.
Delvos, F. J. (1987). Periodic interpolation on uniform meshes, J. Approx. Theory 51,
pp. 71-80.
De Marchi, S., Schaback, R. and Wendland, H. (2005). Near-optimal data-independent

point locations for radial basis function interpolation, Adv. in Comput. Math. 23 3,
pp. 317-330.
Dilts, G. (1996). Equivalence of the SPH method and a space-time Galerkin moving particle
method, Los Alamos National Laboratory.
Dix, J. G. and Ogden, R. D. (1994). An interpolation scheme with radial basis in Sobolev
s n
spaces H (R ), Rocky Mountain J. Math. 24, pp. 1319-1337.
Djokovic, D. and van Damme, R. (1996). Radial splines and moving grids, Technical report,
University of Twente.
Djokovic, D. and van Damme, R. (1997). Moving node methods for PDE's using radial
basis functions and S-splines, Technical report, University of Twente.
Dolbow, J. and Belytschko, T. (1997). An introduction to programming the meshless
element free Galerkin method, Technical report, Northwestern University.
Donoghue, W. F. (1974). Monotone Matrix Functions and Analytic Continuation, Springer
(Berlin).
Driscoll, T. A. and Fornberg, B. (2002). Interpolation in the limit of increasingly flat radial
basis functions, Comput. Math. Appl. 43, pp. 413-422.
Driscoll, T. A. and Heryudono, A. (2006). Adaptive residual subsampling methods for
radial basis function interpolation and collocation problems, Comput. Math. Appl.,
submitted.
Duan, Z.-H. and Krasny, R. (2001). An adaptive treecode for computing nonbonded po-
tential energy in classical molecular systems, J. Comput. Chemistry 22, pp. 184-195.
Duarte, C. A. (1995). A review of some meshless methods to solve partial differential
equations, T I C A M Report 95-06, University of Texas.
Duarte, C. A. and Oden, J. T. (1995a). Hp clouds a meshless method to solve boundary-
value problems, T I C A M Report 95-05, University of Texas.
Duarte, C. A. and Oden, J. T. (1996a). An h-p adaptive method using clouds, Comput.
Meth. Appl. Mech. Engg. 139 1, pp. 237-262.
Duarte, C. A. and Oden, J. T. (1996b). H-p clouds an h-p meshless method, Num.
Meth. for Part. Diff. Eq. 12, pp. 673-705.
Dubai, M. R. (1992). Construction of three-dimensional black-hole initial data via multi-
quadrics, Phys. Rev. D 45, pp. 1178-1187.
Dubai, M. R. (1994). Domain decomposition and local refinement for multiquadric ap-
proximations. I: Second-order equations in one-dimension, J. Appl. Sc. Comp. 1,
pp. 146-171.
Dubai, M. R., Oliveira, S. R. and Matzner, R. A. (1992). Solution of elliptic equations
in numerical relativity using multiquadrics, in Approaches to Numerical Relativity,
R. d'Inverno (ed.), Cambridge University Press (Cambridge), pp. 265-280.
Duchon, J. (1976). Interpolation des fonctions de deux variables suivant le principe de la
flexion des plaques minces, Rev. Francaise Automat. Informat. Rech. Oper., Anal.
Numer. 10, pp. 5-12.
Duchon, J. (1977). Splines minimizing rotation-invariant semi-norms in Sobolev spaces,
in Constructive Theory of Functions of Several Variables, Oberwolfach 1976, W.
Schempp and K. Zeller (eds.), Springer Lecture Notes in Math. 571, Springer-Verlag
(Berlin), pp. 85-100.
Duchon, J. (1978). Sur l'erreur d'interpolation des fonctions de plusieurs variables par
les Z) -splines, Rev.
m
Francaise Automat. Informat. Rech. Oper., Anal. Numer. 12,
pp. 325-334.
Duchon, J. (1980). Fontions splines homogenes a plusiers variables, Universite de Grenoble.
Dyn, N. (1987). Interpolation of scattered data by radial functions, in Topics in Multivari-
Bibliography 461
ate Approximation, C. K. Chui, L. L. Schumaker, and F. Utreras (eds.), Academic

Press (New York), pp. 47-61.
Dyn, N. (1989). Interpolation and approximation by radial and related functions, in Ap-
proximation Theory VI, C. Chui, L. Schumaker, and J. Ward (eds.), Academic Press
(New York), pp. 211-234.
Dyn, N., Goodman, T. and Micchelli, C. A. (1986). Positive powers of certain conditionally
negative definite matrices, Indaga. Math. Proc. Ser.A. 89, pp. 163-178.
Dyn, N., Jackson, I. R. H., Levin, D. and Ron, A. (1992). On multivariate approximation
by the integer translates of a basis function, Israel J. Math. 78, pp. 95-130.
Dyn, N. and Levin, D. (1981). Bell shaped basis functions for surface fitting, in Ap-
proximation Theory and Applications, Z. Ziegler (ed.), Academic Press (New York),
pp. 113-129.
Dyn, N. and Levin, D. (1982). Construction of surface spline interpolants of scattered data
over finite domains, Rev. Francaise Automat. Informat. Rech. Oper., Anal. Numer.
16, pp. 201-209.
Dyn, N. and Levin, D. (1983). Iterative solution of systems originating from integral equa-
tions and surface interpolation, SIAM J. Numer. Anal. 20, pp. 377390.
Dyn, N., Levin, D. and Rippa, S. (1983). Surface interpolation and smoothing by "thin
plate" splines, in Approximation Theory IV, C. Chui, L. Schumaker, and J. Ward
Dyn, N., Levin, D. and Rippa, S. (1986). Numerical procedures for surface fitting of
scattered data by radial functions, SIAM J. Sci. Statist. Comput. 7, pp. 639-659.
Dyn, N., Light, W. A. and Cheney, E. W. (1989). Interpolation by piecewise-linear radial
basis functions, I, J. Approx. Theory 59, pp. 202-223.
Dyn, N. and Micchelli, C. A. (1990). Interpolation by sums of radial functions, Numer.
Math. 58, pp. 1-9.
Dyn, N., Narcowich, F. J. and Ward, J. D. (1997). A framework for interpolation and
approximation on Riemannian manifolds, in Approximation theory and optimization
(Cambridge, 1996), Cambridge Univ. Press (Cambridge), pp. 133-144.
Dyn, N., Narcowich, F. J. and Ward, J. D. (1999). Variational principles and Sobolev-type
estimates for generalized interpolation on a Riemannian manifold, Constr. Approx.
15 2, pp. 175-208.
Dyn, N. and Ron, A. (1995a). Radial basis function approximation: from gridded centers
to scattered centers, Proc. London Math. Soc. 71, pp. 76-108.
Dyn, N. and Ron, A. (1995b). Multiresolution analysis by infinitely differentiable com-
pactly supported functions, Appl. Comput. Harmon. Anal. 2, pp. 15-20.
Evgeniou, T., Pontil, M. and Poggio, T. (2000). Regularization networks and support
vector machines, Adv. Comput. Math. 13 1, pp. 1-50.
Fan, J. and Gijbels, I. (1996). Local Polynomial Modelling and its Applications, Chapman
& Hall (New York).
Farwig, R. (1986). Multivariate interpolation of arbitrarily spaced data by moving least
squares methods, J. Comput. Appl. Math. 16, pp. 79-93.
Farwig, R. (1987). Multivariate interpolation of scattered data by moving least squares
methods, in Algorithms for Approximation, Oxford Univ. Press (New York), pp. 193
211, 1987.
Farwig, R. (1991). Rate of convergence of moving least squares interpolation methods:
the univariate case, in Progress in Approximation Theory, Academic Press (Boston,
MA), pp. 313-327.
Fasshauer, G. E. (1994a). Scattered data interpolation with radial basis functions on the
sphere, Technical Report, Vanderbilt University.
Fasshauer, G. E. (1994b). On the density of certain classes of radial basis functions on

spheres, Technical Report, Vanderbilt University.
Fasshauer, G. E. (1995a). Adaptive least squares fitting with radial basis functions on the
sphere, in Mathematical Methods for Curves and Surfaces, M. Daehlen, T. Lyche,
and L. Schumaker (eds.), Vanderbilt University Press (Nashville), pp. 141-150.
Fasshauer, G. E. (1995b). Radial Basis Functions on Spheres, Ph.D. Dissertation, Vander-
bilt University.
Fasshauer, G. E. (1997). Solving partial differential equations by collocation with radial
basis functions, in Surface Fitting and Multiresolution Methods, A. Le Mehaute, C.
Rabut, and L. L. Schumaker (eds.), Vanderbilt University Press (Nashville, TN),
pp. 131-138.
Fasshauer, G. E. (1999a). On smoothing for multilevel approximation with radial ba-
sis functions, in Approximation Theory IX, Vol.IT. Computational Aspects, Charles
K. Chui, and L. L. Schumaker (eds.), Vanderbilt University Press, pp. 55-62.
Fasshauer, G. E. (1999b). Hermite interpolation with radial basis functions on spheres,
Adv. in Comp. Math. 10, pp. 81-96.
Fasshauer, G. E. (1999c). On the numerical solution of differential equations with radial
basis functions, in Boundary Element Technology XIII, C. S. Chen, C. A. Brebbia,
and D. W. Pepper (eds.), W I T Press, pp. 291-300.
Fasshauer, G. E. (1999d). Solving differential equations with radial basis functions: mul-
tilevel methods and smoothing, Adv. in Comp. Math. 11, pp. 139-159.
Fasshauer, G. E. (2001a). Nonsymmetric multilevel R B F collocation within an opera-
tor Newton framework for nonlinear PDEs, in Trends in Approximation Theory,
K. Kopotun, T. Lyche, and M. Neamtu (eds.), Vanderbilt University Press, pp. 103
112.
Fasshauer, G. E. (2001b). High-order moving least-squares approximation via fast radial
Laguerre transforms, Technical Report, Illinois Institute of Technology.
Fasshauer, G. E. (2002a). Newton iteration with multiquadrics for the solution of nonlinear
PDEs, Comput. Math. Applic. 43, pp. 423-438.
Fasshauer, G. E. (2002b). Matrix-free multilevel moving least-squares methods, in Approxi-
mation Theory X: Wavelets, Splines, and Applications, C. K. Chui, L. L. Schumaker,
and J. Stockier (eds.), Vanderbilt University Press (Nashville), pp. 271-281.
Fasshauer, G. E. (2002c). Approximate moving least-squares approximation with com-
pactly supported weights, in Lecture Notes in Computer Science and Engineer-
ing Vol.26: Meshfree Methods for Partial Differential Equations, M. Griebel and
M. A. Schweitzer (eds.), Springer Verlag (Berlin), pp. 105-116.
Fasshauer, G. E. (2002d). Approximate moving least-squares approximation for time-
dependent PDEs, in WCCM V, Fifth World Congress on Computational Mechanics
(http://wccm.tuwien.ac.at), H. A. Mang, F. G. Rammerstorfer, and J. Eberhard-
steiner (eds.), Vienna University of Technology (Vienna).
Fasshauer, G. E. (2003). Approximate moving least-squares approximation: A fast and
accurate multivariate approximation method, in Curve and Surface Fitting: Saint-
Malo 2002, A. Cohen, J.-L. Merrien, and L. L. Schumaker (eds.), Nashboro Press
(Nashville), pp. 139-148.
Fasshauer, G. E. (2004). Toward approximate moving least squares approximation with
irregularly spaced centers, Computer Methods in Applied Mechanics & Engineering
193, pp. 1231-1243.
Fasshauer, G. E. (2005a). Dual bases and discrete reproducing kernels: a unified framework
for R B F and MLS approximation, Journal of Engineering Analysis with Boundary
Elements 29, pp. 313-325.
Bibliography 463
Fasshauer, G. E. (2005b). R B F collocation methods as pseudospectral methods, in Bound-

ary Elements XXVII, A. Kassab, C. A. Brebbia, E. Divo, and D. Poljak (eds.), WIT
Press (Southampton), pp. 47-56.
Fasshauer, G. E. (2006). Meshfree Methods, in Handbook of Theoretical and Computational
Nanotechnology, Vol. 2, M. Rieth and W. Schommers (eds.), American Scientific
Publishers, pp. 33-97.
Fasshauer, G. E., Gartland, E. C. and Jerome, J. W. (2000a). Algorithms defined by Nash
iteration: some implementations via multilevel collocation and smoothing, J . Comp.
Appl. Math. 119, pp. 161-183.
Fasshauer, G. E., Gartland, E. C. and Jerome, J. W. (2000b). Newton iteration for partial
differential equations and the approximation of the identity, Numerical Algorithms
25, pp. 181-195.
Fasshauer, G. E. and Jerome, J. W. (1999). Multistep approximation algorithms: Improved
convergence rates through postconditioning with smoothing kernels, Adv. in Comput.
Math. 10, pp. 1-27.
Fasshauer, G. E., Khaliq, A. Q. M. and Voss, D. A. (2004). Using meshfree approxima-
tion for multi-asset American option problems, J . Chinese Institute Engineers 27,
pp. 563-571.
Fasshauer, G. E. and Schumaker, L. L. (1998). Scattered data fitting on the sphere,
in Mathematical Methods for Curves and Surfaces II, M. Dshlen, T. Lyche, and
L. L. Schumaker (eds.), Vanderbilt University Press, pp. 117-166.
Fasshauer, G. E. and Zhang, J. G. (2004). Recent results for moving least squares approx-
imation, in Geometric Modeling and Computing: Seattle 2003, M. L. Lucian and M.
Neamtu (eds.), Nashboro Press (Brentwood, TN), pp. 163-176.
Fasshauer, G. E. and Zhang, J. G. (2006). Iterated approximate moving least squares
approximation, submitted.
Faul, A. C. and Powell, M. J. D. (1999). Proof of convergence of an iterative technique for
thin plate spline interpolation in two dimensions, Adv. Comput. Math. 11, pp. 183
192.
Faul, A. C. and Powell, M. J. D. (2000). Krylov subspace methods for radial basis function
interpolation, in Numerical Analysis 1999 (Dundee), Chapman &: Hall/CRC (Boca
Raton, FL), pp. 115-141.
Fedoseyev, A. I., Friedman, M. J. and Kansa, E. J. (2000). Continuation for nonlinear
elliptic partial differential equations discretized by the multiquadric method, Int. J .
Bifurcation and Chaos 10 2, pp. 481-492.
Fedoseyev, A. I., Friedman, M. J. and Kansa, E. J. (2002). Improved multiquadric method
for elliptic partial differential equations via P D E collocation on the boundary, Com-
put. Math. Applic. 43, pp. 439-455.
Feller, W. (1966). An Introduction to Probability Theory and Its Application, Vol. 2, Wiley
&; Sons, New York.
Fenn, M. and Steidl, G. (2006). Robust local approximation of scattered data, in Geometric
Properties from Incomplete Data, R. Klette, R. Kozera, L. Noakes, and J. Weickert
(eds.), Kluwer (Dordrecht), pp. 317-334.
Ferreira, A. J. M. and Fasshauer, G. E. (2006) Computation of natural frequencies of
shear deformable beams and plates by an RBF-pseudospectral method, Comput.
Meth. Appl. Mech. Engng. 196, 134-146.
Ferreira, A. J. M. and Fasshauer, G. E. (2007) Analysis of natural frequencies of composite
plates by an RBF-pseudospectral method, Composite Structures, to appear.
Floater, M. S. and Iske, A. (1996a). Thinning and approximation of large sets of scattered
data, in Advanced Topics in Multivariate Approximation, F. Fontanella, K. Jetter,
and P.-J. Laurent (eds.), World Scientific Publishing (Singapore), pp. 87-96.
-1. 4 /
Floater, M. S. and Iske, A. (1996b). Multistep scattered data interpolation using compactly
supported radial basis functions, J . Comput. Applied Math. 73, pp. 65-78.
Florence, A. G. and van Loan, C. F. (2000). A Kronecker product formulation of the fast
Gauss transform, preprint.
Flyer, N. (2006). Exact polynomial reprodction for oscillatory radial basis functions on
infinite lattices, Comput. Math. Appl. 51 8, pp. 1199-1208.
Foley, T. A. (1987). Interpolation and approximation of 3-D and A-D scattered data,
Foley, T. A. (1990). Interpolation of scattered data on a spherical domain, in Algorithms
for Approximation II, M. G. Cox and J. C. Mason (eds.), Chapman & Hall (London),
pp. 303-310.
Foley, T. A. (1992). The map and blend scattered data interpolant on the sphere, Comput.
Math. Appl. 24, pp. 49-60.
Foley, T. A. (1994). Near optimal parameter selection for multiquadric interpolation, J .
Appl. Sc. Comp. 1, pp. 54-69.
Foley, T. A., Dayanand, S. and Zeckzer, D. (1995). Localized radial basis methods using
rational triangle patches, in Geometric Modelling Dagstuhl 1993, H. Hagen, G.
Farin and H. Noltemeier (eds.), Springer Verlag (Berlin), pp. 163-176.
Foley, T. A., Lane, D. A., Nielson, G. M., Franke, R. and Hagen, H. (1990a). Interpolation
of scattered data on closed surfaces, Comput. Aided Geom. Design 7, pp. 303-312.
Foley, T. A., Lane, D. A., Nielson, G. M. and Ramaraj, R. (1990b). Visualizing functions
over a sphere, Comp. Graphics and Applies. 10, pp. 32-40.
Fornberg, B. (1998). A Practical Guide to Pseudospectral Methods, Cambridge Univ. Press.
Fornberg, B., Driscoll, T. A., Wright, G. and Charles, R. (2002). Observations on the
behavior of radial basis function approximations near boundaries, Comput. Math.
Appl. 43, pp. 473-490.
Fornberg, B. and Flyer, N. (2005). Accuracy of radial basis function interpolation and
derivative approximations on 1-D infinite grids, Adv. Comput. Math. 23 1-2, pp. 5
20.
Fornberg, B., Larsson, E. and Wright, G. (2004). A new class of oscillatory radial basis
functions, Comput. Math. Appl. 51 8, pp. 1209-1222.
Fornberg, B. and Wright, G. (2004). Stable computation of multiquadric interpolants for
all values of the shape parameter, Comput. Math. Appl. 47, pp. 497-523.
Fornberg, B. and Zuev, J. (2006). The Runge phenomenon and spatially variable shape
parameters in R B F interpolation, Comput. Math. Appl., submitted.
Franke, C. and Schaback, R. (1998a). Solving partial differential equations by collocation
using radial basis functions, Appl. Math. Comp. 93, pp. 73-82.
Franke, C. and Schaback, R. (1998b). Convergence orders of meshless collocation methods
using radial basis functions, Adv. in Comput. Math. 8, pp. 381-399.
Franke, R. (1977). Locally determined smooth interpolation at irregularly spaced points
in several variables, J . Inst. Math. Applic. 19, pp. 471-482.
Franke, R. (1982a). Scattered data interpolation: tests of some methods, Math. Comp. 48,
pp. 181-200.
Franke, R. (1982b). Smooth interpolation of scattered data by local thin plate splines,
Franke, R. (1985). Thin plate splines with tension, Comput. Aided Geom. Design 2, pp. 87
95.
Franke, R. (1987). Recent advances in the approximation of surfaces from scattered data, in
Topics in Multivariate Approximation, C. K. Chui, L. L. Schumaker, and F. Utreras
Bibliography 465
Franke, R. (1990). Approximation of scattered data for meteorological applications, in Mul-

tivariate Approximation and Interpolation, ISNM 94, W. Haussman and K. Jetter
(eds.), Birkhauser (Basel), pp. 107-120.
Franke, R. (1995). Paper presented at SIAM Geometric Modeling Conference in Nashville
Nov. 1995.
Franke, R., Hagen, H. and Nielson, G. M. (1994). Least squares surface approximation to
scattered data using multiquadric functions, Adv. in Comput. Math. 2, pp. 81-99.
Franke, R., Hagen, H. and Nielson, G. M. (1995). Repeated knots in least squares mul-
tiquadric functions, in Geometric Modelling Dagstuhl 1993, H. Hagen, G. Farin
and H. Noltemeier (eds.), Springer Verlag (Berlin), pp. 177-187.
Franke, R. and Nielson, G. M. (1991). Scattered data interpolation and applications: A
tutorial and survey, in Geometric Modeling, Methods and Applications, H. Hagen
and D. Roller (eds.), Springer Verlag (Berlin), pp. 131-160.
Franke, R. and Salkauskas, K. (1995). Localization of multivariate interpolation and
smoothing methods, Paper presented at SIAM Geometric Modeling Conference,
Nashville, 1995.
Freeden, W. (1982). Spline methods in geodetic approximation problems, Math. Meth.
Appl. Sci. 4, pp. 382-396.
Freeden, W. (1984). Spherical spline interpolation basic theory and computational
aspects, J . Comput. Appl. Math. 11, pp. 367-375.
Freeden, W. (1987). A spline interpolation method for solving boundary value problems of
potential theory from discretely given data, Num. Meth. Part. Diff. Eq. 3, pp. 375
398.
Freeden, W., Gervens, T. and Schreiner, M. (1998). Constructive Approximation on the
Sphere, Oxford University Press (Oxford).
Freeden, W., Schreiner, M. and Franke, R. (1997). A survey on spherical spline approxi-
mation, Surv. Math. Ind. 7, pp. 29-85.
Freeman, J. A. and Saad, D. (1995). Learning and generalization in radial basis function
Frigo, M. and Johnson, S. G. FFTW: The fastest Fourier transform in the West (C library),
http://www.fftw.org/.
Galperin, E. A. and Zheng, Q. (1993). Solution and control of P D E via global optimization
methods, Comput. Math. Appl. 25, pp. 103-118.
Gaspar, C. (2004). A meshless polyharmonic-type boundary interpolation method for solv-
ing boundary integral equations, Engineering Analysis with Boundary Elements 28,
pp. 1207-1216.
Gasper, G. (1975). Positivity and special functions, in Theory and Application of Special
Functions, R. Askey (ed.), Academic Press (New York), pp. 375-433.
Gel'fand, I. M. and Schilow, G. E. (1960). Verallgemeinerte Funktionen, Vol. 1, Deutscher
Verlag der Wissenschaften (Leipzig).
Gel'fand, I. M. and Vilenkin, N. Ya. (1964). Generalized Functions Vol. 4, Academic Press
(New York).
Ghorbany, M. and Soheili, A. R. (2004). Moving element free Petrov-Galerkin viscous
method, J . Chinese Inst. Engineers 27, pp. 473-479.
Girosi, F. (1992). Some extensions of radial basis functions and their applications in arti-
ficial intelligence, Comput. Math. Appl. 24, pp. 61-80.
Girosi, F. (1998). An equivalence between sparse approximation and support vector ma-
chines, Neural Computation 10, pp. 1455-1480.
Girosi, F. and Anzellotti, G. (1993). Rates of convergence for radial basis functions and
neural networks, in Artificial Networks for Speech and Vision, R. J. Mammone (ed.),
Chapman & Hall, London, 1993, pp. 97-114.
Girosi, F., Jones, M. and Poggio, T. (1993). Regularization theory and neural network
architectures, MIT AI memo 1430.
Gneiting, T. (1999). Correlation functions for atmospheric data analysis, Quart. J. Mete-
orol. Soc. 125, pp. 2449-2464.
Gneiting, T. (2002). Compactly supported correlation functions, J. Multivariate Analysis
83, pp. 493-508.
Golberg, M. A. (1995). A note on the sparse representation of discrete integral operators,
Appl. Math. Comp. 70, pp. 97-118.
Golberg, M. A. (1996). Recent developments in the numerical evaluation of particular
solutions in the boudary element method, Appl. Math. Comp. 75, pp. 91-101.
Golberg, M. A. and Chen, C. S. (1994). The theory of radial basis functions applied to the
BEM for inhomogeneous partial differential equations, Boundary Elements Comm.
5, pp. 57-61.
Golberg, M. A. and Chen, C. S. (1996) A bibliography on radial basis function approxi-
mation, UNLV.
Golberg, M. A. and Chen, C. S. (1997). Discrete Projection Methods for Integral Equations,
Computational Mechanics Publications (Southampton).
Golberg, M. A., Chen, C. S. and Bowman, H. (1999). Some recent results and proposals
for the use of radial basis functions in the BEM, Eng. Anal, with Bound. Elem. 23,
pp. 285-296.
Golberg, M. A., Chen, C. S., Bowman, H. and Power, H. (1998). Some comments on
the use of radial basis functions in the dual reciprocity method, Comput. Mech. 21,
pp. 141-148.
Golberg, M. A., Chen, C. S. and Karur, S. R. (1996). Improved multiquadric approximation
for partial differential equations, Eng. Anal, with Bound. Elem. 18, pp. 9-17.
von Golitschek, M. and Schumaker, L. L. (1990). Data fitting by penalized least squares,
in Algorithms for Approximation II, M. G. Cox and J. C. Mason (eds.), Chapman
& Hall (London), pp. 210-227.
Golomb, M. and Weinberger, H. F. (1959). Optimal approximation and error bounds,
in On Numerical Approximation, R. E. Langer(ed.), University of Wisconsin Press,
pp. 117-190.
Golub, G. H. and Van Loan, C. F. (1989). Matrix Computations, Johns Hopkins University
Press (Baltimore).
Goodsell, G. (1997). A multigrid-type method for thin plate spline interpolation on a circle,
IMA J. Numer. Anal. 17, pp. 321-327.
Gottlieb, D. and Lustman, L. (1983). The spectrum of the Chebyshev collocation operator
for the heat equation, SIAM J. Numer. Anal. 20 5, pp. 909-921.
Gram, J. P. (1883). Uber Entwicklung reeler Functionen in Reihen mittelst der Methode
der kleinsten Quadrate, J. Math. 94, pp. 41-73.
Greengard, L. (1994). Fast algorithms for classical physics, Science 265, pp. 909-914.
Greengard, L. and Rokhlin, V. (1987). A fast algorithm for particle simulations, J. Comput.
Phys. 73 2, pp. 325-348.
Greengard, L. and Strain, J. (1991). The fast Gauss transform, SIAM J. Sci. Statist.
Comput. 12, pp. 79-94.
Greengard, L. F. and Sun, X. (1998). A new version of the fast Gauss transform, Doc.
Math. J. DMV, Extra Volume ICM, pp. 575-584.
Gu, C. and Wahba, G. (1993). Semiparametric analysis of variance with tensor product
thin plate splines, J. Roy. Statist. Soc. Ser. B 55, pp. 353-368.
Guo, K., Hu, S. and Sun, X. (1993a). Conditionally positive definite functions and Laplace-
Stieltjes integrals, J. Approx. Theory 74, pp. 249-265.
Bibliography 467
Guo, K., Hu, S. and Sun, X. (1993b). Cardinal interpolation using linear combinations of
translates of conditionally positive definite functions, Numer. Func. Anal. Optim.
14, pp. 371-381.
Guo, K. and Sun, X. (1991). Scattered data interpolation by linear combinations of trans-
lates of conditionally positive definite functions, Numer. Func. Anal. Optim. 12,
pp. 137-152.
Gutzmer, T. (1996). Interpolation by positive definite functions on locally compact groups
with application to SO(3), Results Math. 29, pp. 69-77.
Hagan, R. E. (1993). Multiquadrics in engineering modeling, in Proceedings ASCE National
Conference on Irrigation and Drainage Engineering, Park City, Utah.
Hagan, R. E. and Kansa, E. J. (1994). Studies of the R parameter in the multiquadric
function applied to ground water pumping, J. Appl. Sci. Comput. 1, pp. 266281.
Hales, S. J. and Levesley, J. (2000). On compactly supported, positive definite, radial basis
functions, Technical Report 2000/30, University of Leicester, UK.
Hales, S. J. and Levesley, J. (2002). Error estimates for multilevel approximation using
polyharmonic splines, Numer. Algorithms 30, pp. 1-10.
Halgamuge, S. K., Poechmueller, W. and Glesner, M. (1995). An alternative approach for
generation of membership functions and fuzzy rules based on radial and cubic basis
function networks, Internat. J. Approx. Reason. 12, pp. 279-298.
Halton, E. J. and Light, W. A. (1993). On local and controlled approximation order, J.
Halton, J. H. (1960). On the efficiency of certain quasi-random sequences of points in
evaluating multi-dimensional integrals, Numer. Math. 2, pp. 84-90.
Handscomb, D. (1993). Local recovery of a solenoidal vector field by an extension of the
thin-plate spline technique, Numer. Algorithms 5 1-4, pp. 121-129.
Hangelbroek, T. (2006). Error estimates for thin plate spline approximation in the disc,
Constr. Approx., to appear.
Harder, R. L. and Desmarais, R. N. (1972). Interpolation using surface splines, J. Aircraft
9, pp. 189-191.
Hardy, R. L. (1971). Multiquadric equations of topography and other irregular surfaces,
J. Geophys. Res. 76, pp. 1905-1915.
Hardy, R. L. (1972). Geodetic applications of multiquadric analysis, AVN Allg. Vermess.
Nachr. 79, pp. 389-406.
Hardy, R. L. (1975). Research results in the application of multiquadric equations to
surveying and mapping problems, Survg. Mapp. 35, pp. 321-332.
Hardy, R. L. (1977). Least squares prediction, Photogramm. Eng. and Remote Sensing 43,
pp. 475-492.
Hardy, R. L. (1990). Theory and applications of the multiquadric-biharmonic method,
Hardy, R. L. (1992a). Comments on the discovery and evolution of the multiquadric
method, Comput. Math. Appl. 24, pp. xi-xii.
Hardy, R. L. (1992b). A contribution of the multiquadric method: interpolation of potential
inside the earth, Comput. Math. Appl. 24, pp. 81-97.
Hardy, R. L. and Gopfert, W. M. (1975). Least squares prediction of gravity anomalies,
geodial undulations, and deflections of the vertical with multiquadric harmonic func-
tions, Geophys. Res. Letters 2, pp. 423-426.
Hardy, R. L. and Nelson, S. A. (1986). A multiquadric-biharmonic representation and
approximation of disturbing potential, Geophys. Res. Letters 13, pp. 18-21.
Hardy, R. L. and Siranyone, S. (1989). The multiquadric-biharmonic method of a three
dimensional mapping inside ore deposits, in Proceedings of Multinational Conference

on Mine Planning and Design, University of Kentucky.
Hartman, E. J., Keeler, J. D. and Kowalski, J. M. (1990). Layered neural networks with
Gaussian hidden units as universal approximations, Neural Computations 2, pp. 210
215.
Hartmann, S. (1998). Multilevel-Fehlerabschatzung bei der Interpolation mit radialen Ba-
sisfunktionen, Ph.D. Dissertation, Universitat Gottingen.
Heiss, M. and Kampl, S. (1996). Multiplication-free radial basis function network, IEEE
Trans. Neural Networks 7, pp. 1461-1464.
Hollig, K. (2003). Finite Element Methods With B-splines, SIAM Frontiers in Applied
Mathematics no. 26 (Philadelphia).
Hon, Y. C. (1999). Multiquadric collocation method with adaptive technique for problems
with boundary layer, Appl. Sci. Comput. 6 3, pp. 173-184.
Hon, Y. C. and Mao, X. Z. (1997). A multiquadric interpolation method for solving initial
value problems, J . Sci. Comput. 12 1, pp. 51-55.
Hon, Y. C. and Mao, X. Z. (1998). An efficient numerical scheme for Burgers' equation,
Appl. Math. Comput. 95, pp. 37-50.
Hon, Y. C. and Mao, X. Z. (1999). A radial basis function method for solving options
pricing model, Financial Engineering 8, pp. 31-49.
Hon, Y. C. and Schaback, R. (2001). On nonsymmetric collocation by radial basis func-
tions, Appl. Math. Comput. 119, pp. 177-186.
Hon, Y. C., Schaback, R. and Zhou, X. (2003). An adaptive greedy algorithm for solving
large R B F collocation problems, Numer. Algorithms 32, pp. 13-25.
Hon, Y. C. and Wu, Z. (2000). Additive Schwarz domain decomposition with a radial basis
approximation, Int. J . Appl. Math. 4, pp. 81-98.
Horn, R. A. and Johnson, C. R. (1985). Matrix Analysis, Cambridge University Press
(Cambridge).
Horn, R. A. and Johnson, C. R. (1991). Topics in Matrix Analysis, Cambridge University
Press (Cambridge).
Hu, H.-Y., Li, Z.-C. and Cheng, A. H.-D. (2005). Radial basis collocation methods for
elliptic boundary value problems, Comput. Math. Appl. 50, pp. 289-320.
Hubbert, S. and Morton, T. M. (2004a). A Duchon framework for the sphere, J . Approx.
Theory 129, pp. 28-57.
Hubbert, S. and Morton, T. M. (2004b). L -error estimates for radial basis function inter-
p
polation on the sphere, J . Approx. Theory 129, pp. 58-77.

Hunt, K. J., Haas, R. and Murray Smith, R. (1996). Extending the functional equivalence
of radial basis function networks and fuzzy inference systems, IEEE Trans. Neural
Networks 7, pp. 776-781.
Hutchinson, M. F. (1991). The application of thin plate smoothing splines to continent-
wide data assimilation, in BMRC Research Report No.27, Data Assimilation Sys-
tems, J. D. Jasper (ed.), Bureau of Meteorology (Melbourne), pp. 104-113.
Hutchinson, M. F. (1993). On thin plate splines and kriging, in Computing and Science
in Statistics 25, M. E. Tarter and M. D. Lock (eds.), Interface Foundation of North
America, University of California, Berkeley, pp. 55-62.
Hutchinson, M. F. (1995). Interpolating mean rainfall using thin plate smoothing splines,
Int. J . GIS 9, pp. 305-403.
Ingber, M. S., Chen, C. S. and Tanski, J. A. (2004). A mesh free approach using radial basis
functions and parallel domain decomposition for solving three dimensional diffusion
equations, Int. J . Num. Meths. Engng. 60, pp. 2183-2201.
Iske, A. (1994). Charakterisierung bedingt positiv definiter Funktionen fur multivariate
Bibliography 469
Interpolationsmethoden mit radial Basisfunktionen, Ph.D. Dissertation, Universitat

Gottingen.
Iske, A. (1995). Reconstruction of functions from generalized Hermite-Birkhoff data, in
Approximation Theory VIII, Vol. 1: Approximation and Interpolation, C. Chui, and
L. Schumaker (eds.), World Scientific Publishing (Singapore), pp. 257-264.
Iske, A. (1999a). Reconstruction of smooth signals from irregular samples by using radial
basis function approximation, in Proceeding of the 1999 International Workshop on
Sampling Theory and Applications, Y. Lyubaskii (ed.), The Norwegian University of
Science and Technology (Trondheim), pp. 82-87.
Iske, A. (1999b). Perfect centre placement for radial basis function methods, Technical
Report TUM M9809, Technische Universitat Miinchen.
Iske, A. (2001). Hierarchical scattered data filtering for multilevel interpolation schemes, in
Mathematical Methods for Curves and Surfaces: Oslo 2000, T. Lyche and L. L. Schu-
maker (eds.), Vanderbilt University Press, pp. 211-220.
Iske, A. (2002). Multiresolution Methods in Scattered Data Modelling, Habilitation Thesis,
Technische Universitat Miinchen.
Iske, A. (2003a). On the approximation order and numerical stability of local Lagrange
interpolation by polyharmonic splines, in Modern Developments in Multivariate Ap-
proximation, W. Haussmann, K. Jetter, M. Reimer, and J. Stockier (eds.), ISNM
145, Birkhauser Verlag (Basel), pp. 153-165.
Iske, A. (2003b). Radial basis functions: basics, advanced topics and meshfree methods
for transport problems, Rend. Sem. Mat. Univ. Pol. Torino 61, pp. 247-285.
Iske, A. (2004). Multiresolution Methods in Scattered Data Modelling, Lecture Notes in
Computational Science and Engineering 37, Springer Verlag (Berlin).
Iske, A. and Sonar, T. (1996). On the structure of function spaces in optimal recovery
of point functionals for ENO-schemes by radial basis functions, Numer. Math. 74,
pp. 177-201.
Ivanov, T., Maz'ya, V. and Schmidt, G. (1999). Boundary layer approximate approxima-
tions and cubature of potentials in domains, Adv. in Comput. Math. 10, pp. 311-342.
Jackson, I. R. H. (1987). Approximation to continuous functions using radial basis func-
tions, in The Mathematics of Surfaces II, R. R. Martin (ed.), Clarendon Press (Ox-
ford), pp. 115-135.
Jackson, I. R. H. (1988a). Radial basis function methods from multivariate approximation,
Ph.D. Dissertation, University of Cambridge.
Jackson, I. R. H. (1988b). Convergence properties of radial basis functions, Constr. Approx.
4, pp. 243-264.
Jackson, I. R. H. (1989a). An order of convergence for some radial basis functions, IMA
J. Numer. Anal. 9, pp. 567-587.
Jackson, I. R. H. (1989b). Radial basis functions a survey and new results, in Mathemat-
ics of Surfaces III, D. C. Handscomb (ed.), Clarendon Press (Oxford), pp. 115-133.
Jekeli, C. (1994). Hardy's multiquadric-biharmonic method for gravity field predictions,
Jetter, K. (1993a). Multivariate approximation from the cardinal interpolation point of
view, in Approximation Theory VII, E. W. Cheney, C. Chui, and L. Schumaker
Jetter, K. (1993b). Riesz bounds in scattered data interpolation and /^-approximation, in
Multivariate Approximation: From CAGD to Wavelets, Kurt Jetter and Florencio
Utreras (eds.), World Scientific Publishing (Singapore), pp. 167-177.
Jetter, K. (1994). Conditionally lower Riesz bounds in scattered data interpolation, in
Wavelets, Images, and Surface Fitting, P.-J. Laurent, A. Le Mehaute, and L. L.
Schumaker (eds.), A. K. Peters (Wellesley, MA), pp. 295-302.
Jetter, K. and Stockier, J. (1991). Algorithms for cardinal interpolation using box splines
and radial basis functions, Numer. Math. 6 0 , pp. 97-114.
Jetter, K. and Stockier, J. (1997). Topics in scattered data interpolation and non-uniform
sampling, in Surface Fitting and Multiresolution, A. Le Mehaute, C. Rabut, and
Jia, R. Q. and Lei, J. J. (1991). Approximation by multiinteger translates of functions
having global support, J . Approx. Theory 7 2 , pp. 2-23.
Jia, R. Q. and Lei, J. J. (1993). A new version of the Strang-Fix conditions, J . Approx.
Theory 74, pp. 221-225.
Johnson, M. J. (1998). A bound on the approximation order of surface splines, Constr.
Approx. 1 4 3, pp. 429-438.
Johnson, M. J. (2000a). Overcoming the boundary effects in surface spline interpolation,
IMA J. Numer. Anal. 2 0 3, pp. 405-422.
d
Johnson, M. J. (2000b). Approximation in L ( R ) from spaces spanned by the perturbed
p
integer translates of a radial function, J . Approx. Theory 1 0 7 2, pp. 163-203.

Johnson, M. J. (2000c). An improved order of approximation for thin-plate spline inter-
polation in the unit disc, Numer. Math. 8 4 3, pp. 451-474.
Johnson, M. J. (2001a). On the error in surface spline interpolation of a compactly sup-
ported function, Kuwait J . Sci. Engrg. 2 8 1, pp. 37-54.
Johnson, M. J. (2001b). The Z/22-approximation order of surface spline interpolation, Math.
Comp. 7 0 234, pp. 719-737.
Johnson, M. J. (2004a). An error analysis for radial basis function interpolation, Numer.
Math. 9 8 4, pp. 675-694.
Johnson, M. J. (2004b). The L -approximation order of surface spline interpolation for
p
1 < P < 2, Constr. Approx. 2 0 2, pp. 303-324.

Johnson, M. J. (2006). A note on the limited stability of surface spline interpolation, J .
Approx. Theory 1 4 1 2, pp. 182-188.
Jumarhon, B., Amini, S. and Chen, K. (2000). The Hermite collocation method using radial
basis functions, Engineering Analysis with Boundary Elements 2 4 7-8, pp. 607-611.
Kaczmarz, S. (1937). Approximate solution of systems of linear equations (originally pub-
lished as: Angenaherte Auflosung von Systemen linearer Gleichungen, Bulletin In-
ternational de l'Academie Polonaise des Sciences, Lett. A, 1937, pp. 355-357), Int.
J . Control s, pp. 1269-1271.
Kansa, E. J. (1986). Application of Hardy's multiquadric interpolation to hydrodynamics,
Proc. 1986 Simul. Conf. 4, pp. 111-117.
Kansa, E. J. (1990a). Multiquadrics A scattered data approximation scheme with ap-
plications to computational fluid-dynamics I: Surface approximations and partial
derivative estimates, Comput. Math. Appl. 1 9 , pp. 127-145.
Kansa, E. J. (1990b). Multiquadrics A scattered data approximation scheme with ap-
plications to computational fluid-dynamics II: Solutions to parabolic, hyperbolic
and elliptic partial differential equations, Comput. Math. Appl. 1 9 , pp. 147-161.
Kansa, E. J. (1992). A strictly conservative spatial approximation scheme for the gov-
erning engineering and physics equations over irregular regions and inhomogeneous
scattered nodes, Comput. Math. Appl. 2 4 , pp. 169-190.
Kansa, E. J. and Carlson, R. E. (1992). Improved accuracy of multiquadric interpolation
using variable shape parameters, Comput. Math. Appl. 2 4 , pp. 99-120.
Kansa, E. J. and Carlson, R. E. (1995). Radial basis functions: a class of grid free scattered
data approximations, Computational Fluid Dynamics J . 3, pp. 479-496.
Kansa, E. J., Fasshauer, G. E., Power, H. and Ling, L. (2004). A volumetric integral
radial basis function method for time-dependent partial differential equations: I.
Formulation, Engineering Analysis with Boundary Elements 2 8 , pp. 1191-1206.
Bibliography 471
Kansa, E. J. and Hon, Y. C. (2000). Circumventing the ill-conditioning problem with mul-
tiquadric radial basis functions: Applications to elliptic partial differential equations,
Comput. Math. Applic. 39, pp. 123-137.
Karlin, S. (1968). Total Positivity, Stanford University Press (Stanford).
Karur, S. R. and Ramachandran, P. A. (1994). Radial basis function approximation in the
dual reciprocity method, Math. Comput. Modell. 20, pp. 59-70.
Karur, S. R. and Ramachandran, P. A. (1995). Augmented thin plate spline approximation
in DRM, Boundary Elements Comm. 6, pp. 55-58.
Kent, J. T. and Mardia, K. V. (1994). The link between kriging and thin-plate splines,
Probability, statistics and optimisation, Wiley Ser. Probab. Math. Statist., Wiley
(Chichester), pp. 325-339.
Kimeldorf, G. and Wahba, G. (1971). Some results on Tchebycheffian spline functions, J.
Math. Anal. Applic. 33, pp. 82-95.
Kincaid, D. and Cheney, W. (2002). Numerical Analysis: Mathematics of Scientific Com-
puting (3rd ed.), Brooks/Cole (Pacific Grove, CA).
Komargodsky, Z. and Levin, D. (2006). Hermite type moving-least-squares approximation,
Comput. Math. Appl. 51 8, pp. 1223-1232.
Kounchev, O. (2001). Multivariate Polysplines: Applications to Numerical and Wavelet
Analysis, Academic Press (New York).
Krzyzak, A., Linder, T. and Lugosi, G. (1996). Nonparametric estimation and classification
using radial basis function nets and empirical risk minimization, IEEE Trans. Neural
Networks 7, pp. 475-487.
Kunis, S. and Potts, D. (2002). NFFT, Softwarepackage (C-library), Universitat Liibeck,
http://www.math.uni-luebeck.de/potts/nfft/.
Kunis, S., Potts, D. and Steidl, G. (2002). Fast Fourier transforms at non-
equispaced knots: A user's guide to a C-library, Universitat Liibeck,
http://www.math.uni-luebeck.de/potts/nfft/.
Kurdila, A. J., Narcowich, F. J. and Ward, J. D. (1995). Persistency of excitation in
identification using radial basis function approximants, SIAM J. Control Optim. 33,
pp. 625-642.
Kybic, J., Blu, T. and Unser, M. (2002a). Generalized sampling: A variational approach
Part I: Theory, IEEE Trans. Signal Proc. 50, pp. 1965-1976.
Kybic, J., Blu, T. and Unser, M. (2002b). Generalized sampling: A variational approach
Part II: Applications, IEEE Trans. Signal Proc. 50, pp. 1977-1985.
Lancaster, P. and Salkauskas, K. (1981). Surfaces generated by moving least squares meth-
ods, Math. Comp. 37, pp. 141-158.
Lancaster, P. and Salkauskas, K. (1986). Curve and Surface Fitting, Academic Press (New
York).
Lanzara, F., Maz'ya, V. and Schmidt, G. (2006). Approximate Approximations from scat-
tered data, J. Approx. Theory, in press.
La Rocca, A., Hernandez Rosales, A. and Power, H. (2005). Radial basis function Her-
mite collocation approach for the solution of time dependent convection-diffusion
problems, Engineering Analysis with Boundary Elements 29, pp. 359-370.
Larsson, E. and Fornberg, B. (2003). A numerical study of some radial basis function based
solution methods for elliptic PDEs, Comput. Math. Appl. 46, pp. 891-902.
Larsson, E. and Fornberg, B. (2005). Theoretical and computational aspects of multivariate
interpolation with increasingly flat radial basis functions, Comput. Math. Appl. 49,
pp. 103-130.
Lee, D. and Shiau, J. H. (1994). Thin plate splines with discontinuities and fast algorithms
for their computation, SIAM J. Scient. Computing 15, pp. 1311-1330.
Lee, S. and Pan, J. C. J. (1996). Unconstrained handwritten numeral recognition based

on radial basis competitive and cooperative networks with spatiotemporal feature
representation, IEEE Trans. Neural Networks 7, pp. 455-474.
Lee, Y. (1991). Handwritten digit recognition using K nearest-neighbor, radial-basis func-
tion, and backpropagation neural networks, Neural Computation 3, pp. 440-449.
Le Gia, Q. T. (2004). Galerkin approximation for elliptic PDEs on spheres, J. Approx.
Theory 130, pp. 125-149.
Leitao, V. M. A. (2001). A meshless method for Kirchhoff plate bending problems, Int. J.
Numer. Meth. Engng. 52, pp. 1107-1130.
Leitao, V. M. A.- (2004). RBF-based meshless methods for 2D elastostatic problems, E n -
gineering Analysis with Boundary Elements 28, pp. 1271-1281.
Leitao, V. M. A. (2006). Application of radial basis functions to linear and non-linear
structural analysis problems, Comput. Math. Appl. 51 8, pp. 1311-1334.
Le Mehaute, A. (1995). Knot removal for scattered data, in Approximation Theory,
Wavelets and Applications, S. P. Singh (ed.), Kluwer (Dordrecht), pp. 197-213.
Le Mehaute, A. (1999). Inf-Convolution and radial basis functions, in New Developments in
Approximation Theory, M. W. Miiller, M. D. Buhmann, D. H. Mache and M. Felten
(eds.), Birkhauser (Basel), pp. 95-108.
Leonard, J. A., Kramer, M. A. and Ungar, J. H. (1992). Using radial basis functions to
approximate a function and its error bounds, I E E E Trans, on Neural Networks 3,
pp. 624-627.
Levesley, J. (1994). Convolution kernels for approximation by radial basis functions, in
Wavelets, Images, and Surface Fitting, P.-J. Laurent, A. Le Mehaute, and L. L.
Schumaker (eds.), A. K. Peters (Wellesley, MA), pp. 343-350.
Levesley, J. (1995a). Pointwise estimates for multivariate interpolation using conditionally
positive definite functions, in Approximation Theory, Wavelets and Applications, S.
P. Singh (ed.), Kluwer (Dordrecht), pp. 381-401.
Levesley, J. (1995b). Convolution kernels based on thin plate splines, Numer. Algorithms
10, pp. 401-419.
Levesley, J. and Kushpel, A. K. (1995). Interpolation on compact Abelian groups using
generalised sk-splies, in Approximation Theory VIII, Vol. 1: Approximation and
Interpolation, C. Chui, and L. Schumaker (eds.), World Scientific Publishing (Sin-
gapore), pp. 317-324.
Levesley, J. and Kushpel, A. K. (1999). Generalised sk-spline interpolation on compact
Abelian groups, J. Approx. Theory 97 2, 311-333.
Levesley, J., Light, W., Ragozin, D. and Sun, X. (1999). A simple approach to the vari-
ational theory for interpolation on spheres, in New Developments in Approxima-
tion Theory, M. W. Miiller, M. D. Buhmann, D. H. Mache and M. Felten (eds.),
Birkhauser (Basel), pp. 117-144.
Levesley, J. and Ragozin, D. (2002). Positive definite kernel interpolation on manifolds:
convergence rates, in Approximation Theory X: Abstract and Classical Analysis,
C. K. Chui, L. L. Schumaker, and J. Stockier (eds.), Vanderbilt University Press,
Nashville, 2002, pp. 277-285.
Levesley, J. and Roach, M. (1995). Quasi-interpolation on compact domains, in Approxi-
mation Theory, Wavelets and Applications, S. P. Singh (ed.), Kluwer (Dordrecht),
pp. 557-566.
Levesley, J. and Sun, X. (1995). Scattered Hermite interpolation by ridge functions, Numer.
Func. Anal. Optim. 16, pp. 989-1001.
Levesley, J. and Sun, X. (2005). Approximation in rough native spaces by shifts of smooth
kernels on spheres, J. Approx. Theory 133, pp. 269-283.
Bibliography 473
Levesley, J . a n d S u n , X . (2006). C o r r i g e n d u m to a n d two open questions arising from the

article " A p p r o x i m a t i o n in rough native spaces b y shifts of s m o o t h kernels on spheres"
[J. A p p r o x . T h e o r y 133 (2005) 269-283], J. Approx. Theory 1 3 8 1, pp. 124-127.
Levesley, J . , X u , Y . , L i g h t , W . a n d C h e n e y , W . (1996). C o n v o l u t i o n operators for r a d i a l
basis approximation, SIAM J. Math. Anal. 2 7 , pp. 286-304.
L e v i n , D . (1998). T h e a p p r o x i m a t i o n power of m o v i n g least-squares, Math. Comp. 6 7 ,
pp. 1517-1531.
L i , J . (2005). M i x e d methods for fourth-order elliptic a n d p a r a b o l i c problems using r a d i a l
basis functions, Adv. in Comput. Math. 2 3 , pp. 21-30.
L i , J . , C h e n g , A . H . - D . a n d C h e n , C - S . (2003). O n the efficiency a n d exponential con-
vergence of m u l t i q u a d r i c collocation m e t h o d c o m p a r e d to finite element m e t h o d ,
Engineering Analysis with Boundary Elements 2 7 3, pp. 251257.
L i , J . a n d H o n , Y . C . (2004). D o m a i n decomposition for radial basis meshless methods,
Numerical Methods for Partial Differential Equations 2 0 3, pp. 450-462.
L i , S. a n d L i u , W . K . (1996). M o v i n g least-square r e p r o d u c i n g kernel m e t h o d P a r t I I :
Fourier analysis, Comp. Meth. Appl. Mech. Eng. 1 3 9 , pp. 159-193.
L i , S. a n d L i u , W . K . (2002). Meshfree a n d particle methods a n d their applications, Applied
Mechanics Review 5 5 , pp. 1-34.
L i , X . (1998). O n simultaneous a p p r o x i m a t i o n s by r a d i a l basis function n e u r a l networks,
Appl. Math. Comput. 9 5 , pp. 75-89.
L i , X . a n d C h e n , C . S. (2004). A m e s h free m e t h o d using hyperinterpolation a n d
fast F o u r i e r transform for solving differential equations, Engineering Analysis with
Boundary Elements 2 8 , pp. 1253-1260.
L i , X . , H o , C . H . a n d C h e n , C . S. (2002). C o m p u t a t i o n a l test of a p p r o x i m a t i o n of functions
a n d their derivatives by r a d i a l basis functions, Neural Parallel Sci. Comput. 10,
pp. 25-46.
L i , X . a n d M i c c h e l l i , C . A . (2000). A p p r o x i m a t i o n by radial bases a n d n e u r a l networks,
Numer. Algorithms 2 5 , pp. 241-262.
L i n d s a y , K . a n d K r a s n y , R . (2001). A particle m e t h o d a n d a d a p t i v e treecode for vortex
sheet motion i n three-dimensional flow, J. Comput. Physics 1 7 2 , pp. 879-907.
L i g h t , W . A . (1991). R e c e n t developments in the S t r a n g - F i x theory for a p p r o x i m a t i o n
orders, in Curves and Surfaces, P . - J . L a u r e n t , A . L e M e h a u t e , a n d L . L . S c h u m a k e r
(eds.), A c a d e m i c P r e s s ( N e w Y o r k ) , pp. 285-292.
L i g h t , W . A . (1992). Some aspects of r a d i a l basis function a p p r o x i m a t i o n , in Approxi-
mation Theory, Spline Functions and Applications, S. P . S i n g h (ed.), K l u w e r ( D o r -
drecht), pp. 163-190.
L i g h t , W . A . (1994). U s i n g r a d i a l functions on compact domains, in Wavelets, Images,
and Surface Fitting, P . - J . L a u r e n t , A . L e M e h a u t e , and L . L . S c h u m a k e r (eds.), A .
K . Peters (Wellesley, M A ) , pp. 351-370.
L i g h t , W . A . (1996). V a r i a t i o n a l error b o u n d s for r a d i a l basis functions, i n Numerical
Analysis 1995, D . F . Griffiths a n d G . A . W a t s o n (eds.), L o n g m a n ( H a r l o w ) , pp. 94
106.
L i g h t , W . A . a n d C h e n e y , E . W . (1991). Interpolation by piecewise-linear r a d i a l basis
functions, I I , J. Approx. Theory 6 4 , pp. 38-54.
L i g h t , W . A . a n d C h e n e y , E . W . (1992a). I n t e r p o l a t i o n by periodic radial basis functions,
J. Math. Anal. Appl. 1 6 8 , pp. 111-130.
L i g h t , W . A . a n d C h e n e y , E . W . (1992b). Q u a s i - i n t e r p o l a t i o n w i t h translates of a function
h a v i n g noncompact s u p p o r t , Constr. Approx. 8, pp. 35-48.
L i g h t , W . A . a n d W a y n e , H . (1995). E r r o r estimates for a p p r o x i m a t i o n by r a d i a l basis
f u n c t i o n s , i n Approximation Theory, Wavelets and Applications, S. P. S i n g h ( e d . ) ,

K l u w e r (Dordrecht), pp. 215-246.
L i g h t , W . A . and W a y n e , H . (1998). O n power functions a n d error estimates for r a d i a l
basis f u n c t i o n i n t e r p o l a t i o n , J. Approx. Theory 9 2 , p p . 2 4 5 - 2 6 6 .
L i g h t , W . A . a n d W a y n e , H . ( 1 9 9 9 ) . Spaces o f d i s t r i b u t i o n s a n d i n t e r p o l a t i o n b y t r a n s l a t e s
o f a basis f u n c t i o n , Numer. Math. 8 1 , p p . 4 1 5 - 4 5 0 .
L i n g , L . a n d H o n , Y . C . (2005). I m p r o v e d n u m e r i c a l solver for K a n s a ' s m e t h o d based
o n affine space d e c o m p o s i t i o n , Engineering Analysis with Boundary Elements 29,
pp. 1077-1085.
L i n g , L . a n d K a n s a , E . J . ( 2 0 0 4 ) . P r e c o n d i t i o n i n g for r a d i a l basis f u n c t i o n s w i t h d o m a i n
d e c o m p o s i t i o n m e t h o d s , Math, and Comput. Modelling 4 0 , p p . 1 4 1 3 - 1 4 2 7 .
L i n g , L . a n d K a n s a , E . J. ( 2 0 0 5 ) . A least-squares p r e c o n d i t i o n e r for r a d i a l basis f u n c t i o n s
c o l l o c a t i o n m e t h o d s , Adv. in Comput. Math. 2 3 , p p . 3 1 - 5 4 .
L i n g , L . , O p f e r , R . a n d S c h a b a c k , R . ( 2 0 0 6 ) . R e s u l t s o n meshless c o l l o c a t i o n t e c h n i q u e s ,
Engineering Analysis with Boundary Elements 30, p p . 247-253.
L i u , G . R . ( 2 0 0 2 ) . Mesh Free Methods: Moving beyond the Finite Element Method, CRC
Press ( B o c a R a t o n , F L ) .
L i u , W . K . , C h e n , Y . , A z i z U r a s , R . a n d C h a n g , C . T . ( 1 9 9 6 ) . G e n e r a l i z e d m u l t i p l e scale
r e p r o d u c i n g k e r n e l p a r t i c l e m e t h o d s , Comp. Meth. Appl. Mech. Eng. 1 3 9 , p p . 9 1
157.
L i u , W . K . , L i , S. a n d B e l y t s c h k o , T . ( 1 9 9 7 ) . M o v i n g l e a s t s q u a r e r e p r o d u c i n g k e r n e l
m e t h o d s ( I ) M e t h o d o l o g y a n d c o n v e r g e n c e , Comp. Meth. Appl. Mech. Eng. 1 4 3 ,
p p . 113-154.
L o c h e r , F . ( 1 9 8 1 ) . I n t e r p o l a t i o n o n u n i f o r m meshes b y t h e t r a n s l a t e s o f one f u n c t i o n a n d
r e l a t e d a t t e n u a t i o n f a c t o r s , Math. Comp. 3 7 , p p . 4 0 3 - 4 1 6 .
L o r e n t z , R . , N a r c o w i c h , F . J. a n d W a r d , J . D . ( 2 0 0 3 ) . C o l l o c a t i o n d i s c r e t i z a t i o n s o f t h e
t r a n s p o r t e q u a t i o n w i t h r a d i a l basis f u n c t i o n s , Applied Mathematics and Computa-
tion 1 4 5 1, p p . 9 7 - 1 1 6 .
L o w e , D . ( 1 9 9 3 ) . N o v e l ' t o p o g r a p h i c ' n o n l i n e a r f e a t u r e e x t r a c t i o n u s i n g r a d i a l basis f u n c -
t i o n s for c o n c e n t r a t i o n c o d i n g i n t h e ' a r t i f i c i a l nose', i n 3rd IEE International Con-
ference on Artificial Neural Networks, I E E ( L o n d o n ) .
Lowe, D . a n d M a t t h e w s , R . (1995). Shakespeare vs. Fletcher: a stylistic analysis b y r a d i a l
basis f u n c t i o n s , Computers and Humanities 29, pp. 449-461.
L o w e , D . a n d M c L a c h l a n , A . ( 1 9 9 5 ) . M o d e l l i n g o f n o n s t a t i o n a r y processes u s i n g r a d i a l
basis f u n c t i o n n e t w o r k s , i n Artificial Neural Networks, 1995, 3 0 0 - 3 0 5 .
L o w e , D . a n d T i p p i n g , M . E . ( 1 9 9 5 ) . A n o v e l n e u r a l n e t w o r k t e c h n i q u e for e x p l o r a t o r y
d a t a a n a l y s i s , i n Proceedings of ICANN '95, V o l . 1, E C 2 & C i e ( P a r i s ) , p p . 3 3 9 - 3 4 4 .
Lowe, D . and T i p p i n g , M . E . (1996). Feed-forward neural networks and t o p o g r a p h i c m a p -
p i n g s for e x p l o r a t o r y d a t a a n a l y s i s , Neural Computing and Applications 4, p p . 83-95.
L o w i t z s c h , S. ( 2 0 0 2 ) . A p p r o x i m a t i o n a n d I n t e r p o l a t i o n E m p l o y i n g D i v e r g e n c e - F r e e R a d i a l
Basis F u n c t i o n s w i t h A p p l i c a t i o n s , P h . D . D i s s e r t a t i o n , T e x a s A & M U n i v e r s i t y .
L o w i t z s c h , S. ( 2 0 0 5 ) . M a t r i x - v a l u e d r a d i a l basis f u n c t i o n s : s t a b i l i t y e s t i m a t e s a n d a p p l i -
c a t i o n s , Adv. in Comput. Math. 2 3 3, p p . 2 9 9 - 3 1 5 .
L u k a c s , E . ( 1 9 7 0 ) . Characteristic Functions, Griffin (London).
L y c h e , T . ( 1 9 9 2 ) . K n o t R e m o v a l for S p l i n e C u r v e s a n d Surfaces, i n Approximation Theory
VII, E . W . C h e n e y , C . C h u i , a n d L . S c h u m a k e r ( e d s . ) , A c a d e m i c Press ( N e w Y o r k ) ,
pp. 207-226.
M a d y c h , W . R . ( 1 9 8 9 ) . C a r d i n a l i n t e r p o l a t i o n w i t h p o l y h a r m o n i c splines, i n Multivariate
Approximation Theory IV, I S N M 90, C . C h u i , W . S c h e m p p , a n d K . Z e l l e r (eds.),
Birkhauser Verlag (Basel), pp. 241-248.
Bibliography 475
Madych, W. R. (1990). Polyharmonic splines, multiscale analysis, and entire functions,

in Multivariate Approximation and Interpolation, ISNM 94, W. Haussman and K.
Jetter (eds.), Birkhauser (Basel), pp. 205-216.
Madych, W. R. (1991) Error estimates for interpolation by generalized splines, in Curves
and Surfaces, P.-J. Laurent, A. Le Mehaute, and L. L. Schumaker (eds.), Academic
Madych, W. R. (1992). Miscellaneous error bounds for multiquadric and related interpo-
lators, Comput. Math. Appl. 24, pp. 121-138.
Madych, W. R. and Nelson, S. A. (1983). Multivariate interpolation: a variational theory,
manuscript.
Madych, W. R. and Nelson, S. A. (1988). Multivariate interpolation and conditionally
positive definite functions, Approx. Theory Appl. 4, pp. 77-89.
Madych, W. R. and Nelson, S. A. (1989). Error bounds for multiquadric interpolation, in
Approximation Theory VI, C. Chui, L. Schumaker, and J. Ward (eds.), Academic
Madych, W. R. and Nelson, S. A. (1990a). Multivariate interpolation and conditionally
positive definite functions, II, Math. Comp. 54, pp. 211-230.
Madych, W. R. and Nelson, S. A. (1990b). Polyharmonic cardinal splines, J. Approx.
Theory 60, pp. 141-156.
Madych, W. R. and Nelson, S. A. (1990c). Polyharmonic cardinal splines: A minimization
property, J. Approx. Theory 63, pp. 303-320.
Madych, W. R. and Nelson, S. A. (1992). Bounds on multivariate polynomials and expo-
nential error estimates for multiquadric interpolation, J. Approx. Theory 70, pp. 94
114.
Mai-Duy, N. and Tran-Cong, T. (2001a). Numerical solution of differential equations using
multiquadric radial basis function networks, Neural Networks 14, pp. 185-199.
Mai-Duy, N. and Tran-Cong, T. (2001b). Numerical solution of Navier-Stokes equations
using radial basis function networks, Int. J. Numer. Meth. Fluids 37, pp. 65-86.
Maiorov, V. (2005). On lower bounds in radial basis approximation, Adv. in Comp. Math.
22, pp. 103-113.
Mairhuber, J. C. (1956). On Haar's theorem concerning Chebyshev approximation prob-
lems having unique solutions, Proc. Am. Math. Soc. 7, pp. 609-615.
Makroglu, A. (1992). Radial basis functions in the numerical solution of Fredholm and
integro-differential equations, in Advances in Computer Methods for Partial Differ-
ential Equations - VII, R. Vichnevetsky, D. Knight, and G. Richter (eds.), IMACS
(New Brunswick), pp. 478-484.
Makroglu, A. (1994a). Radial basis functions in the numerical solution of nonlinear Volterra
integral equations, J. Appl. Sci. Comput. 1, pp. 33-53.
Makroglu, A. (1994b). Multiquadric collocation methods in the numerical solution of
Volterra integral and integro-differential equations, in Mathematics of Computation
1943-1993, Proc. Sympos. Appl. Math., 48, Amer. Math. Soc. (Providence, RI),
pp. 337-341.
Makroglu, A. and Kansa, E. J. (1993). Multiquadric collocation methods in the numer-
ical solution of Volterra integral equations with weakly singular kernels, Lawrence
Livermore National Laboratory.
Marcozzi, M. D., Choi, S. and Chen, C. S. (1999). R B F and optimal stopping problems: an
application to the pricing of vanilla options on one risky asset, in Boundary Element
Technology XIII, C. S. Chen, C. A. Brebbia, and D. W. Pepper (eds.), WIT Press,
pp. 345-354.
Marcus, M. and Mine, H. (1965). Introduction to Linear Algebra, Dover Publications (New
York), republished 1988.
Matern, B. (1986). Spatial variation (Second ed.), Lecture Notes in Statistics, 36, Springer-
Verlag (Berlin).
Matheron, G. (1965). Les variables regionalise.es et leur estimation, Masson (Paris).
Matheron, G. (1973). The intrinsic random functions and their applications, Adv. Appl.
Prob. 5, pp. 439-468.
Mathias, M. (1923). Uber positive Fourier-Integrale, Math. Zeit. 16, pp. 103-125.
M A T L A B Central File Exchange, available online at http://www.mathworks.com/-
matlabcentral/fileexchange/.
Maz'ya, V. (1991). A new approximation method and its applications to the calcula-
tion of volume potentials. Boundary point method, in DFG-Kolloquium des DFG-
Forschungsschwerpunktes "Randelementmethoden".
Maz'ya, V. (1994). Approximate approximations, in The Mathematics of Finite Elements
and Applications, J. R. Whiteman (ed.), Wiley (Chichester), pp. 77-104.
Maz'ya, V. and Schmidt, G. (1996). On approximate approximations using Gaussian ker-
nels, IMA J. Numer. Anal. 16, pp. 13-29.
Maz'ya, V. and Schmidt, G. (2001). On quasi-interpolation with non-uniformly distributed
centers on domains and manifolds, J. Approx. Theory 110, pp. 125-145.
McCormick, S. F. (1992). Multilevel Projection Methods for Partial Differential Equa-
tions, CBMS-NSF Regional Conference Series in Applied Mathematics 62, SIAM
(Philadelphia).
McLain, D. H. (1974). Drawing contours from arbitrary data points, Comput. J. 17,
pp. 318-324.
McMahon, J. (1986). Knot selection for least squares approximation using thin plate
splines, M.S. Thesis, Naval Postgraduate School.
McMahon, J. and Franke, R. (1992). Knot selection for least squares thin plate splines,
SIAM J. Sci. Statist. Comput. 13, pp. 484-498.
Meinguet, J. (1979a). Multivariate interpolation at arbitrary points made simple, Z.
Angew. Math. Phys. 30, pp. 292-304.
Meinguet, J. (1979b). An intrinsic approach to multivariate spline interpolation at arbi-
trary points, in Polynomial and Spline Approximations, N. B. Sahney (ed.), Reidel
(Dordrecht), pp. 163-190.
Meinguet, J. (1979c). Basic mathematical aspects of surface spline interpolation, in Nu-
merische Integration, G. Hammerlin (ed.), Birkhauser (Basel), pp. 211-220.
Meinguet, J. (1984). Surface spline interpolation: basic theory and computational aspects,
in Approximation Theory and Spline Functions, S. P. Singh, J. H. W. Burry, and B.
Watson (eds.), Reidel (Dordrecht), pp. 127-142.
Melenk, J. M. and Babuska, I. (1996). The partition of unity finite element method: basic
theory and applications, Comput. Methods Appl. Mech. Engrg. 139, pp. 289-314.
Menegatto, V. A. (1992). Interpolation on Spherical Spaces, Ph.D. Dissertation, University
of Texas Austin.
Menegatto, V. A. (1994a). Interpolation on spherical domains, Analysis 14, pp. 415-424.
Menegatto, V. A. (1994b). Strictly positive definite kernels on the Hilbert sphere, Appl.
Anal. 55, pp. 91-101.
Menegatto, V. A. (1995a). Interpolation on the complex Hilbert sphere, Approx. Theory
Appl. 11, pp. 1-9.
Menegatto, V. A. (1995b). Strictly positive definite kernels on the circle, Rocky Mountain
J. Math. 25, pp. 1149-1163.
Menegatto, V. A. (1997). Interpolation on the complex Hilbert sphere using positive defi-
nite and conditionally negative definite functions, Acta Mathematica Hungarica 75
3, pp. 215-225.
Bibliography 477
Menegatto, V. A., Oliveira, C. P. and Peron, A. P. (2006) Strictly positive definite kernels
on subsets of the complex plane, Comput. Math. Appl. 51 8, pp. 1233-1250.
Menegatto, V. A. and Peron, A. P. (2004). Conditionally positive definite kernels on Eu-
clidean domains, J. Math. Anal. Appl. 294 1, pp. 345-359.
Menger, K. (1928). Die Metrik des Hilbertschen Raumes, Anzeiger der Akad. der Wis-
senschaften in Wien, Nat. Kl. 65, pp. 159-160.
Meyer, C. D. (2000). Matrix Analysis and Applied Linear Algebra, SIAM (Philadelphia).
Mhaskar, H. N. (1990). Weighted polynomials, radial basis functions and potentials on
locally compact spaces, Numer. Func. Anal. Optim. 11, pp. 987-1017.
Mhaskar, H. N. and Micchelli, C. A. (1992). Approximation by superposition of sigmoidal
and radial basis functions, Adv. in Appl. Math. 13, pp. 350-373.
Micchelli, C. A. (1986). Interpolation of scattered data: distance matrices and conditionally
positive definite functions, Constr. Approx. 2, pp. 11-22.
Micchelli, C. A. and Rivlin, T. J. (1977). A survey of optimal recovery, in Optimal E s -
timation in Approximation Theory (Proc. Internat. Sympos., Freudenstadt, 1976),
C. A. Micchelli and T. J. Rivlin (eds.), Plenum Press (New York), pp. 1-54.
Micchelli, C. A. and Rivlin, T. J. (1980). Optimal recovery of best approximations, Resul-
tate Math. 3 1, pp. 25-32.
Micchelli, C. A. and Rivlin, T. J. (1985). Lectures on optimal recovery, in Numerical
analysis, Lancaster 1984, P- R- Turner (ed.), Lecture Notes in Math., 1129, Springer
(Berlin), pp. 21-93.
Micchelli, C. A., Rivlin, T. J. and Winograd, S. (1976). The optimal recovery of smooth
functions, Numer. Math. 26 2, pp. 191-200.
Miranda, J. (2004). Incorporating R-functions into the Theory of Positive Definite Func-
tions to Solve Elliptic Partial Differential Equations, Ph.D. Dissertation, Illinois
Institute of Technology.
Monaghan, J. J. (1988). An introduction to SPH, Comp. Phys. Comm. 48, pp. 89-96.
Montes, P. (1991). Local kriging interpolation: Application to scattered data on the sphere,
in Curves and Surfaces, P.-J. Laurent, A. Le Mehaute, and L. L. Schumaker (eds.),
Montes, P. (1994). Smoothing noisy data by kriging with nugget effects, in Wavelets,
Images, and Surface Fitting, P.-J. Laurent, A. Le Mehaute, and L. L. Schumaker
(eds.), A. K. Peters (Wellesley, MA), pp. 371-378.
Moody, J. and Darken, C. J. (1989). Fast learning in networks of locally-tunes processing
units, Neural Computations 1, pp. 281-294.
Moridis, G. J. and Kansa, E. J. (1992). The method of Laplace transform multiquadrics
for the solution of the groundwater flow equation, in Advances in Computer Methods
for Partial Differential Equations - VII, R. Vichnevetsky, D. Knight, and G. Richter
(eds.), IMACS (New Brunswick), pp. 539-545.
Moridis, G. J. and Kansa, E. J. (1994). The Laplace transform multiquadric method:
A highly accurate scheme for the numerical solution of linear partial differential
equations, J. Appl. Sc. Comp. 1, pp. 375407.
Morse, B., Yoo, T. S., Rheingans, P., Chen, D. T. and Subramanian, K. R. (2001). Interpo-
lating implicit surfaces from scattered surface data using compactly supported radial
basis functions, in 2001 International Conference on Shape Modeling and Applica-
tions (SMI 2001), I E E E Computer Society Press (Los Alamitos, CA), pp. 89-98.
Morton, T. M. (2004). Two approaches to solving pseudodifferential equations on spheres
using zonal kernels, Engineering Analysis with Boundary Elements 28, pp. 1227
1232.
Mouat, C. T. (2001). Fast algorithms and preconditioning techniques for fitting radial
basis functions, Ph.D. Dissertation, University of Canterbury, NZ.
Miiller, C. (1966). Spherical Harmonics, Springer Lecture Notes in Mathematics (Vol. 17).
Mulgrew, B. (1996). Applying radial basis functions, IEEE Signal Proc. Magazine 13,
pp. 50-65.
Musavi, M. T., Ahmed, W., Chan, K. H., Faris, K. B. and Hummels, D. M. (1992). On
the training of radial basis function classifiers, Neural Networks 5, pp. 595-603.
Muttiah, R., Hutchinson, M. and Dyke, P. (1995). Climate surfaces of Texas using thin
plate splines, in Approximation Theory VIII, Vol. 1: Approximation and Interpo-
lation, C. Chui, and L. Schumaker (eds.), World Scientific Publishing (Singapore),
pp. 427-434.
Myers, D. E. (1988). Interpolation with positive definite functions, Sciences de la Terre
28, pp. 252-265.
Myers, D. E. (1992). Kriging, co-Kriging, radial basis functions and the role of positive
definiteness, Comput. Math. Appl. 24, pp. 139-148.
Myers, D. E. (1993). Selection of a radial basis function for interpolation, in Advances
in Computer Methods for Partial Differential Equations - VII, R. Vichnevetsky,
D. Knight, and G. Richter (eds.), IMACS (New Brunswick), pp. 553-558.
Nadaraya, E- A. (1964). On estimating regression, Theor. Probab. Appl. 9, pp. 141-142.
Narcowich, F. J. (1995). Generalized Hermite interpolation and positive definite kernels
on a Riemannian manifold, J . Math. Anal. Appl. 190, pp. 165-193.
Narcowich, F. J., Schaback, R. and Ward, J. D. (1999). Multilevel interpolation and ap-
proximation, Appl. Comput. Harmon. Anal. 7, pp. 243-261.
Narcowich, F. J., Sivakumar, N. and Ward, J. D. (1994). On condition numbers associated
with radial-function interpolation, J . Math. Anal. Appl. 186, pp. 457-485.
Narcowich, F. J., Smith, P. and Ward, J. D. (1995). Density of translates of radial functions
on compact sets, in Approximation Theory VIII, Vol. 1: Approximation and Inter-
polation, C. Chui, and L. Schumaker (eds.), World Scientific Publishing (Singapore),
pp. 435-442.
Narcowich, F. J. and Ward, J. D. (1991a). Norms of inverses and condition numbers for
matrices associated with scattered data, J . Approx. Theory 64, pp. 69-94.
Narcowich, F. J. and Ward, J. D. (1991b). Norms of inverses for matrices associated with
scattered data, in Curves and Surfaces, P.-J. Laurent, A. Le Mehaute, and L. L.
Schumaker (eds.), Academic Press (New York), pp. 341-348.
Narcowich, F. J. and Ward, J. D. (1992). Norm estimates for the inverses of a general
class of scattered-data radial-function interpolation matrices, J . Approx. Theory 69,
pp. 84-109.
Narcowich, F. J. and Ward, J. D. (1994a). Generalized Hermite interpolation via matrix-
valued conditionally positive definite functions, Math. Comp. 63, pp. 661687.
Narcowich, F. J. and Ward, J. D. (1994b). Wavelets associated with periodic basis func-
tions, CAT #340, Texas A&M University.
Narcowich, F. J. and Ward, J. D. (1995). Nonstationary spherical wavelets for scattered
data, in Approximation Theory VIII, Vol. 2: Wavelets and Multilevel Approximation,
C. Chui, and L. Schumaker (eds.), World Scientific Publishing (Singapore), pp. 301
308.
Narcowich, F. J. and Ward, J. D. (1996). Nonstationary wavelets on the m-sphere for
scattered data, Appl. Comput. Harmon. Anal. 3 4, pp. 324-336.
Narcowich, F. J. and Ward, J. D. (2002). Scattered-data interpolation on spheres: error
estimates and locally supported basis functions, SIAM J . Math. Anal. 33 6, pp. 1393
1410.
Bibliography 479
n
Narcowich, F. J. and Ward, J. D. (2004). Scattered-data interpolation on M. : error es-
timates for radial basis and band-limited functions, SIAM J. Math. Anal. 36 1,
pp. 284-300.
Narcowich, F. J., Ward, J. D. and Wendland, H. (2003). Refined error estimates for radial
basis function interpolation, Constr. Approx. 19 4, pp. 541-564.
Narcowich, F. J., Ward, J. D. and Wendland, H. (2005). Sobolev bounds on functions
with scattered zeros, with applications to radial basis function surface fitting, Math.
Comp. 74, pp. 743-763.
Narcowich, F. J., Ward, J. D. and Wendland, H. (2006). Sobolev error estimates and
a Bernstein inequality for scattered data interpolation via radial basis functions,
Constr. Approx. 24 2, pp. 175-186.
Neyman, J. and Pearson, E. S. (1936). Contributions to the Theory of Testing Statistical
Hypotheses, Statistical Research Memoirs 1, pp. 1-37.
Nielson, G. M. (1987). Coordinate free scattered data interpolation, in Topics in Multivari-
ate Approximation, C. K. Chui, L. L. Schumaker, and F. Utreras (eds.), Academic
Nielson, G. M. and Foley, T. A. (1989). A survey of applications of an affine invariant
norm, in Mathematical Methods in Computer Aided Geometric Design, T. Lyche
and L. Schumaker (eds.), Academic Press (New York), pp. 445-467.
Nieslony, A., Potts, D. and Steidl, G. (2004). Rapid evaluation of radial functions by fast
Fourier transforms at nonequispaced knots, Numer. Math. 98, pp. 329-351.
Nuss, W. A. and Titley, D. W. (1994). Use of multiquadric interpolation for meteorological
objective analysis, Monthly Weather Review 12, pp. 1011-1031.
Ohtake, Y., Belyaev, A. and Seidel, H. P. (2003a). A multi-scale approach to 3D scattered
data interpolation with compactly supported basis functions, in Shape Modeling
International, 2003, pp. 153-161.
Ohtake, Y., Belyaev, A., Alexa, M., Turk, G. and Seidel, H. P. (2003b). Multi-level par-
tition of unity implicits, ACM Trans, on Graphics (Proc. SIGGRAPH 2003) 22 3
pp. 463-470.
Opfer, R. (2004). Multiscale Kernels, Ph.D. Dissertation, Universitat Gottingen.
Opfer, R. (2006). Multiscale Kernels, Adv. in Comput. Math. 25 4, pp. 357-380.
Orr, M. J. L. (1995). Regularization in the selection of radial basis function centres, Neural
Computation 7, pp. 606-623.
Orr, M. J. L. (1996). Introduction to radial basis function networks, Cen-
tre for Cognitive Sciences, University of Edinburgh, available online at
http://www.anc.ed.ac.uk/rbf/intro/intro.html.
Orr, M., Hallam, J., Takezawa, K., Murray, A., Ninomiya, S., Oide, M. and Leonard, T.
(2000). Combining regression trees and radial basis function networks, Int. J. Neural
Systems 10, pp. 453-465.
Pahner, U. and Hameyer, K. (2000). Adaptive coupling of differential evolution and mul-
tiquadrics approximation for the tuning of the optimization process, I E E E Trans.
Magn. 36 1, pp. 1047-1051.
Park, J. and Sandberg, I. W. (1991). Universal approximation using radial bais function
Park, J. and Sandberg, I. W. (1993). Approximation and radial basis function networks,
Neural Computation 5, pp. 305-316.
Parzen, E. (1962). On estimation of a probability density function and mode, Ann. Math.
Statist. 33, pp. 1065-1076.
Pegram, C. and Pegram, D. S. (1993). Integration of rainfall via multiquadric surfaces over
polygons, J. Hydraulic Eng. 114, pp. 151-163.
Pinkus, A. (2004). Strictly positive definite functions on a real inner product space, Adv.
in Comput. Math. 20, pp. 263-271.
Platte, R. B. and Driscoll, T. A. (2004). Computing eigenmodes of elliptic operators using
radial basis functions, Comput. Math. Appl. 48, pp. 561-576.
Platte, R. B. and Driscoll, T. A. (2005). Polynomials and potential theory for Gaussian
radial basis function interpolation, SIAM J . Numer. Anal., 43, pp. 750-766.
Platte, R. B. and Driscoll, T. A. (2006). Eigenvalue stability of radial basis function
discretizations for time-dependent problems, Comput. Math. Appl. 51 8, pp. 1251
1268.
Poggio, T. and Girosi, F. (1990). Networks for approximation and learning, Proc. IEEE
78, pp. 1481-1497.
Potter, E. H. (1981). Multivariate polyharmonic spline interpolation, Ph.D. Dissertation,
Iowa State University.
Pottmann, H. and Eck, M. (1990). Modified multiquadric methods for scattered data
interpolation over a sphere, Comput. Aided Geom. Design 7, pp. 313-321.
Pottmann, M. and Jorgl, H. P. (1995). Radial basis function networks for internal model
control, Appl. Math. Comput. 70, pp. 283-298.
Potts, D. and Steidl, G. (2003). Fast summation at nonequispaced knots by NFFTs, SIAM
J . Sci. Comput. 24, pp. 2013-2037.
Powell, M. J. D. (1987). Radial basis functions for multivariable interpolation: a review,
in Algorithms for the Approximation of Functions and Data, J. C. Mason and M. G.
Cox (eds.), Oxford Univ. Press (Oxford), pp. 143-167.
Powell, M. J. D. (1988). Radial basis function approximations to polynomials, in Numerical
Analysis 1987, D. F. Griffiths and G. A. Watson (eds.), Longman Scientific and
Technical (Essex), pp. 223-241.
Powell, M. J. D. (1990). Univariate multiquadric approximation: reproduction of linear
polynomials, in Multivariate Approximation and Interpolation, ISNM 94, W. Hauss-
man and K. Jetter (eds.), Birkhauser (Basel), pp. 227-240.
Powell, M. J. D. (1991). Univariate multivariate interpolation: some recent results, in
Curves and Surfaces, P.-J. Laurent, A. Le Mehaute, and L. L. Schumaker (eds.),
Powell, M. J. D. (1992a). The theory of radial basis functions in 1990, in Advances in
Numerical Analysis II: Wavelets, Subdivision, and Radial Basis Functions, W. Light
(ed.), Oxford University Press (Oxford), pp. 105-210.
Powell, M. J. D. (1992b). Tabulation of thin-plate splines on a very fine two-dimensional
grid, in Numerical Methods in Approximation Theory, ISNM 105, D. Braess, L. L.
Schumaker (ed.), Birkhauser (Basel), pp. 221-244.
Powell, M. J. D. (1993). Truncated Laurent expansions for the fast evaluation of thin
plate splines, in Algorithms for Approximation III, J. C. Mason and M. G. Cox
(eds.), Chapman and Hall (London), pp. 99-120.
Powell, M. J. D. (1994a). Some algorithms for thin plate spline interpolation to functions
of two variables, in Advances in Computational Mathematics (New Delhi, 1993),
H. P. Dikshit and C. A. Micchelli (eds.), World Sci. Publishing, pp. 303-319.
Powell, M. J. D. (1994b). The uniform convergence of thin plate spline interpolation in
two dimensions, Numer. Math. 68, pp. 107-128.
Powell, M. J. D. (1997). A new iterative algorithm for thin plate spline interpolation in
two dimensions, Annals of Numerical Mathematics 4, pp. 519-528.
Powell, M. J. D. (1999). Recent research at Cambridge on radial basis functions, in New De-
velopments in Approximation Theory, M. W. Miiller, M. D. Buhmann, D. H. Mache
and M. Felten (eds.), Birkhauser (Basel), pp. 215-232.
Bibliography 481
Power, H. and Barraco, V. (2002). A comparison analysis between unsymmetric and sym-
metric radial basis function collocation methods for the numerical solution of partial
differential equations, Comput. Math. Appl. 4 3 , pp. 551-583.
Qian, L.-F. and Ching, H.-K. (2004). Static and dynamic analysis of 2-D functionally
graded elasticity by using meshless local Petrov-Galerkin method, J. Chinese Inst.
Engineers 2 7 , pp. 491-503.
Quak, E., Sivakumar, N. and Ward, J. D. (1993). Least squares approximation by radial
functions, SIAM J. Numer. Anal. 2 4 , 1043-1066.
Rabut, C. (1989). Fast quasi-interpolation of surfaces with generalized fi-splines on reg-
ular nets, in Mathematics of Surfaces III, D. C. Handscomb (ed.), Clarendon Press
(Oxford), pp. 429-449.
Rabut, C. (1990). 5-splines polyharmoniques cardinales: Interpolation, quasi-interpo-
lation, filtrage, Ph.D. Dissertation, University of Toulouse.
Rabut, C. (1992a). An introduction to Schoenberg's approximation, Comput. Math. Appl.
24, pp. 149-175.
Rabut, C. (1992b). Elementary m-harmonic cardinal B-splines, Numer. Algorithms 2,
pp. 39-67.
Rabut, C. (1992c). High level m-harmonic cardinal B-splines, Numer. Algorithms 2,
pp. 68-84.
Ragozin, D. L. and Levesley, J. (1996). Zonal kernels, approximations and positive defi-
niteness on spheres and compact homogeneous spaces, in Curves and Surfaces with
Applications in CAGD, A. Le Mehaute, C. Rabut, and L. L. Schumaker (eds.), Van-
derbilt University Press (Nashville, TN), pp. 371-378.
Reinsch, C. H. (1967). Smoothing by spline functions, Numer. Math. 10, pp. 177-183.
Riesz, F. and Sz.-Nagy, B. (1955). Functional Analysis, Dover Publications (New York),
republished 1990.
Rippa, S. (1984). Interpolation and smoothing of scattered data by radial basis functions,
M.S. Thesis, Tel Aviv University.
Rippa, S. (1999). An algorithm for selecting a good value for the parameter c in radial
basis function interpolation, Adv. in Comput. Math. 11, pp. 193-210.
Ron, A. (1992). The L2-approximation orders of principal shift-invariant spaces generated
by a radial basis function, in Numerical Methods in Approximation Theory, ISNM
105, D. Braess, L. L. Schumaker (ed.), Birkhauser (Basel), pp. 245-268.
Ron, A. and Sun, X. (1996). Strictly positive definite functions on spheres, Math. Comp.
65 216, pp. 1513-1530.
Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function,
Ann. Math. Statist. 2 7 , pp. 832-837.
Rosenblum, M. and Davis, L. S. (1996). An improved radial basis function network for
visual autonomous road following, IEEE Trans. Neural Networks 7, pp. 1111-1120.
Rosenblum, M., Yacoob, Y. and Davis, L. S. (1996). Human expression recognition from
motion using a radial basis function network architecture, IEEE Trans. Neural Net-
works 7, pp. 1121-1138.
Rudin, W. (1973). Functional Analysis, McGraw-Hill (New York).
Saad, Y. and Schultz, M. H. (1986). GMRES: a generalized minimal residual algorithm for
solving nonsymmetric linear equations, SIAM J. Sci. Statist. Comput. 7 , pp. 856
869.
Saha, A., Wu, C. L., and Tang, D. S. (1993). Approximation, dimension reduction and
nonconvex optimization using linear superpositions of Gaussians, IEEE Trans, on
Computers 4 2 , pp. 1222-1233.
Salkauskas, K. (1992). Moving least squares interpolation with thin-plate splines and radial
basis functions, Comput. Math. Appl. 2 4 , pp. 177-185.
Salkauskas, K. and Bos, L. (1992). Weighted splines as optimal interpolants, Rocky Moun-
tain J . Math. 2 2 , pp. 205-217.
Sanner, R. M. and Slotine, J. J. E. (1992). Gaussian networks for direct adaptive control,
IEEE Trans. Neural Networks 3, pp. 837-863.
Sarler, B. and Vertnik, R. (2006). Meshfree explicit local radial basis function collocation
method for diffusion problems, Comput. Math. Appl. 5 1 8, pp. 1163-1170.
Sarra, S. A. (2005). Adaptive radial basis function methods for time dependent partial
differential equations, Appl. Numer. Math. 5 4 , pp. 79-94.
Sarra, S. A. (2006). Integrated multiquadric radial basis function approximation methods,
Comput. Math. Appl. 5 1 8, pp. 1283-1296.
Saundersen, H. C. (1992). Multiquadric interpolation of fluid speeds in a natural river
channel, Comput. Math. Appl. 2 4 , pp. 187-193.
Schaback, R. (1993). Comparison of radial basis function interpolants, in Multivariate
Approximation: From CAGD to Wavelets, Kurt Jetter and Florencio Utreras (eds.),
World Scientific Publishing (Singapore), pp. 293-305.
Schaback, R. (1994a). Approximation of polynomials by radial basis functions, in Wavelets,
Images, and Surface Fitting, P.-J. Laurent, A. Le Mehaute, and L. L. Schumaker
(eds.), A. K. Peters (Wellesley, MA), pp. 459-466.
Schaback, R. (1994b). Lower bounds for norms of inverses of interpolation matrices for
radial basis functions, J . Approx. Theory 79, pp. 287-306.
Schaback, R. (1995a). Creating surfaces from scattered data using radial basis functions,
in Mathematical Methods for Curves and Surfaces, M. Daahlen, T. Lyche, and L.
Schumaker (eds.), Vanderbilt University Press (Nashville), pp. 477-496.
Schaback, R. (1995b). Error estimates and condition numbers for radial basis function
interpolation, Adv. in Comput. Math. 3, pp. 251-264.
Schaback, R. (1995c). Multivariate interpolation and approximation by translates of a basis
function, in Approximation Theory VIII, Vol. 1: Approximation and Interpolation,
C. Chui, and L. Schumaker (eds.), World Scientific Publishing (Singapore), pp. 491
514.
Schaback, R. (1996). Approximation by radial basis functions with finitely many centers,
Constr. Approx. 1 2 , pp. 331-340.
Schaback, R. (1997a). Optimal recovery in translation-invariant spaces of functions, Annals
of Numerical Mathematics 4, pp. 547-556.
Schaback, R. (1997b). On the efficiency of interpolation by radial basis functions, in Surface
Fitting and Multiresolution Methods, A. Le Mehaute, C. Rabut, and L. L. Schumaker
(eds.), Vanderbilt University Press (Nashville, TN), pp. 309-318.
Schaback, R. (1999a). Native Hilbert spaces for radial basis functions I, in New Develop-
ments in Approximation Theory, M. W. Miiller, M. D. Buhmann, D. H. Mache and
M. Felten (eds.), Birkhauser (Basel), pp. 255-282.
Schaback, R. (1999b). Improved error bounds for scattered data interpolation by radial
basis functions, Math. Comp. 68 225, pp. 201-216.
Schaback, R. (2000a). A unified theory of radial basis functions. Native Hilbert spaces for
radial basis functions II, J . Comput. Appl. Math. 1 2 1 , pp. 165-177.
Schaback, R. (2000b). Remarks on meshless local construction of surfaces, in The Mathe-
matics of Surfaces, IX, R. Cipolla and R. Martin (eds.), Springer, pp. 34-58.
Schaback, R. (2002) Stability of radial basis function interpolants, in Approximation The-
ory X, C. K. Chui, L. L. Schumaker, and J. Stdckler (eds.), Vanderbilt Univ. Press
(Nashville, TN), pp. 433-440.
Schaback, R. (2003). On the versatility of meshless kernel methods, in Advances in Com-
Bibliography 483
putational and Experimental Engineering and Sciences, S. N. Atluri, D. E. Beskos,

and D. Polyzos (eds.), CD ROM, I C C E S proceedings paper #428.
Schaback, R. (2005). Multivariate interpolation by polynomials and radial basis functions,
Schaback, R. (2006a) Convergence of unsymmetric kernel-based meshless collocation meth-
ods, SIAM J. Numer. Anal., to appear.
Schaback, R. (2006b) Limit problems for interpolation by analytic radial basis functions,
J. Comp. Appl. Math., to appear.
Schaback, R. and Wendland, H. (1999). Using compactly supported radial basis func-
tions to solve partial differential equations, in Boundary Element Technology XIII,
C. S. Chen, C. A. Brebbia, and D. W. Pepper (eds.), W I T Press, pp. 311-324.
Schaback, R. and Wendland, H. (2000a). Numerical techniques based on radial basis func-
tions, in Curve and Surface Fitting: Saint-Malo 1999, A. Cohen, C. Rabut, and
L. L. Schumaker (eds.), Vanderbilt University Press (Nashville, TN), 359-374.
Schaback, R. and Wendland, H. (2000b). Adaptive greedy techniques for approximate
solution of large R B F systems, Numer. Algorithms 24, pp. 239-254.
Schaback, R. and Wendland, H. (2001). Characterization and construction of radial basis
functions, in Multivariate Approximation and Applications, N. Dyn, D. Leviatan,
D. Levin, and A. Pinkus (eds.), Cambridge Univ. Press (Cambridge), pp. 1-24.
Schaback, R. and Wendland, H. (2006). Kernel techniques: From machine learning to
meshless methods, Acta Numerica, 15, pp. 543-639.
Schaback, R. and Werner, J. (2006). Linearly constrained reconstruction of functions by
kernels with applications to machine learning, Adv. in Comput. Math. 25, pp. 237
258.
Schaback, R. and Wu, Z. (1996). Operators on radial functions, J. Comput. Appl. Math.
73, pp. 257-270.
Schagen, I. P. (1984). Sequential exploration of unknown multidimensional functions as an
aid to optimization, IMA J. Numer. Anal. 4, pp. 337-347.
Schimming, R. and Belger, M. (1991). Polyharmonic radial functions on a Riemannian
manifold, Math. Nachr. 153, pp. 207-216.
Schiro, R. and Williams, G. (1984). An adaptive application of multiquadric interpolants
for numerically modeling large numbers of irregularly spaced hydrographic data,
Survg. Mapp. 44, pp. 365-381.
Scholkopf, B. and Smola, A. J. (2002). Learning with Kernels: Support Vector Machines,
Regularization, Optimization, and Beyond, MIT Press (Cambridge, MA).
Schoenberg, I. J. (1937). On certain metric spaces arising from Euclidean spaces by a
change of metric and their imbedding in Hilbert space, Ann. of Math. 38, pp. 787
793.
Schoenberg, I. J. (1938a). Metric spaces and completely monotone functions, Ann. of Math.
39, pp. 811-841.
Schoenberg, I. J. (1938b). Metric spaces and positive definite functions, Trans. Amer.
Math. Soc. 44, pp. 522-536.
Schoenberg, I. J. (1942). Positive definite functions on spheres, Duke Math. J. 9, pp. 96
108.
Schoenberg, I. J. (1964). Spline functions and the problem of graduation, Proc. Nat. Acad.
Sci. 52, pp. 947-950.
Schreiner, M. (1997). On a new condition for strictly positive definite functions on spheres,
Proc. Amer. Math. Soc. 125 2, pp. 531-539.
Schumaker, L. L. (1981). Spline Functions: Basic Theory, John Wiley & Sons (New York),
reprinted by Krieger Publishing 1993.
Schweitzer, M. A. (2003). A Parallel Multilevel Partition of Unity Method for Elliptic Par-
tial Differential Equations, Lecture Notes in Computational Science and Engineering,
Vol. 29, Springer Verlag (Berlin).
Sethuraman, R. and Reddy, C. S. (2004). Pseudo elastic analysis of material non-linear
problems using element free Galerkin method, J. Chinese Inst. Engineers 27,
pp. 505-516.
Shannon, C. (1949). Communication in the presence of noise, Proc. I R E 37, pp. 10-21.
Sharan, M., Kansa, E. J. and Gupta, S. (1997). Application of the multiquadric method
for numerical solution of elliptic partial differential equations, Appl. Math. Comp.,
84 2-3, pp. 275-302.
Shaw, E. W. and Lynn, P. P. (1972). Area rainfall evaluation using two surface fitting
techniques, Bull. Int. Ass. Hydrol. Sci. 17, pp. 419-433.
Shepard, D. (1968). A two dimensional interpolation function for irregularly spaced data,
Proc. 23rd Nat. Conf. ACM, pp. 517-524.
Shepherd, T. J. and Broomhead, D. S. (1990). Nonlinear signal processing using radial
basis functions, SPE Advanced Signal Processing Algorithms, Architectures, and Im-
plementations 1348, pp. 51-61.
Sherstinsky, A. and Picard, R. W. (1996). On the efficiency of the orthogonal least squares
training method for radial basis function networks, I E E E Trans. Neural Networks
7, pp. 195-200.
Shu, C , Ding, H. and Yeo, K. S. (2003). Local radial basis function-based differen-
tial quadrature method and its application to solve two-dimensional incompressible
Navier-Stokes equations, Comput. Methods Appl. Mech. Engrg. 192, pp. 941-954.
Shu, C , Ding, H. and Yeo, K. S. (2004). Solution of partial differential equations by
global radial basis function-based differential quadrature method, Engineering Anal-
ysis with Boundary Elements 28, pp. 1217-1226.
Shu, C , Ding, H. and Zhao, N. (2006). Numerical comparison of least square-based finite-
difference (LSFD) and radial basis function-based finite-difference ( R B F F D ) meth-
ods, Comput. Math. Appl. 51 8, pp. 1297-1310.
Sibson, R. and Stone, G. (1991). Computation of thin-plate splines, SIAM J. Sci. Statist.
Comput. 12, pp. 1304-1313.
Sivakumar, N. and Ward, J. D. (1993). On the least squares fit by radial functions to
multidimensional scattered data, Numer. Math. 65, pp. 219-243.
Smith, B. F., Bj0rstad, P. and Gropp, W. D. (1996). Domain Decomposition. Parallel
Multilevel Methods for Elliptic Partial Differential Equations, Cambridge University
Press (Cambridge).
Smith, K. T., Solmon, D. C. and Wagner, S. L. (1977). Practical and mathematical aspects
of the problem of reconstructing objects from radiographs, Bull. Amer. Math. Soc.
83, pp. 1227-1270.
Sneddon, I. H. (1972). The Use of Integral Transforms, McGraw-Hill (New York).
Sonar, T. (1995). Optimal recovery using thin plate splines in finite volume methods for
the numerical solution of hyperbolic conservation laws, IMA J. Numer. Anal. 16,
pp. 549-581.
Stead, S. (1984). Estimation of gradients from scattered data, Rocky Mountain J. Math.
14, pp. 265-279.
Steele, N. C , Reeves, C. R., Nicholas, M. and King, P. J. (1995). Radial basis func-
tion artificial neural networks for the inference process in fuzzy logic based control,
Computing 54, pp. 99-117.
Stein, E. M. and Weiss, G. (1971). Introduction to Fourier Analysis in Euclidean Spaces,
Princeton University Press (Princeton).
Bibliography 485
Stein, M. L. (1999). Interpolation of spatial data. Some theory for Kriging, Springer Series
in Statistics, Springer-Verlag (New York).
Stewart, J. (1976). Positive definite functions and generalizations, an historical survey,
Rocky Mountain J. Math. 6, pp. 409-434.
Strain, J. (1991). The fast Gauss transform with variable scales, SIAM J. Sci. Statist.
Comput. 1 2 , pp. 1131-1139.
Sun, X. (1989). On the solvability of radial function interpolation, in Approximation Theory
VI, C. Chui, L. Schumaker, and J. Ward (eds.), Academic Press (New York), pp. 643
646.
Sun, X. (1990). Multivariate interpolation using ridge or related functions, Ph.D. Disser-
tation, University of Texas.
Sun, X. (1992a). Norm estimates for inverses of Euclidean distance matrices, J. Approx.
Theory 7 0 , pp. 339-347.
Sun, X. (1992b). Cardinal and scattered-cardinal interpolation by functions having non-
compact support, Comput. Math. Appl. 2 4 , pp. 195-200.
Sun, X. (1993a). Solvability of multivariate interpolation by radial or related functions, J.
Sun, X. (1993b). Conditionally positive definite functions and their application to multi-
variate interpolation, J. Approx. Theory 7 4 , pp. 159-180.
Sun, X. (1994a). Scattered Hermite interpolation using radial basis functions, Linear Al-
gebra Appl. 2 0 7 , pp. 135-146.
Sun, X. (1994b). The fundamentality of translates of a continuous function on spheres,
Numerical Algorithms 8, pp. 131-134.
Sun, X. (1994c). Cardinal Hermite interpolation using positive definite functions, Numer-
ical Algorithms 7, pp. 253-268.
Sun, X. (1995). Conditional positive definiteness and complete monotonicity, in Approx-
imation Theory VIII, Vol. 1: Approximation and Interpolation, C. Chui, and L.
Schumaker (eds.), World Scientific Publishing (Singapore), pp. 537-540.
Sun, X. and Cheney, E. W. (1997). Fundamental sets of continuous functions on spheres,
Constr. Approx. 1 3 2, pp. 245-250.
Suykens, J. A. K., De Brabanter, J., Lukas, L. and Vandewalle, J. (2002). Weighted least
squares support vector machines: robustness and sparse approximation, Neurocom-
puting 4 8 1-4, pp. 85-105.
Szego, G. (1959). Orthogonal Polynomials, Amer. Math. Soc. Coll. Publ. Vol. X X I I I (Prov-
idence) .
Tan, S., Hao, J. and Vandewalle, J. (1995). Multivariable nonlinear system identification
by radial basis function neural networks, Arch. Control Sci. 4, pp. 55-67.
Tarwater, A. E. (1985). A parameter study of Hardy's multiquadric method for scattered
data interpolation, Lawrence Livermore National Laboratory, T R UCRL-563670.
Temlyakov, V. N. (1998). The best m-term approximation and greedy algorithms, Adv. in
Comp. Math. 8, pp. 249-265.
Tiago, C. M. and Leitao, V. M. A. (2006). Application of radial basis functions to linear
and nonlinear structural analysis problems, Comput. Math. Appl. 5 1 8, pp. 1311
1334.
Trefethen, L. N. (2000). Spectral Methods in M A T L A B , SIAM (Philadelphia, PA).
Trefethen, L. N. and Bau, D. (1997). Numerical Linear Algebra, SIAM (Philadelphia, PA).
Treinish, L. A. (1995). Visualization of Scattered Meteorological Data, I E E E Computer
Graphics & Applications 1 5 , pp. 20-26.
Tsukanov, I. and Shapiro, V. (2005). Meshfree modeling and analysis of physicalfieldsin
heterogeneous media, Adv. in Comput. Math. 2 3 1-2, pp. 95-124.
Turk, G. and O'Brien, J. F. (2002). Modelling with implicit surfaces that interpolate, ACM
Transactions on Graphics 21, pp. 855-873.
Unser, M. (2000). Sampling 50 years after Shannon, Proc. IEEE 88, pp. 569-587.
Utreras, F. I. (1985). Positive thin plate splines, Approx. Theory Appl. 1, pp. 77-108.
Utreras, F. I. (1993). Multiresolution and pre-wavelets using radial basis functions, in
Multivariate Approximation: From CAGD to Wavelets, Kurt Jetter and Florencio
Utreras (eds.), World Scientific Publishing (Singapore), pp. 321-333.
2
Utreras, F. I. and Vargas, M. L. (1991). Monotone interpolations of scattered data in IR ,
Villalobos, M. A. and Wahba, G. (1983). Multivariate thin plate spline estimates for
the posterior probabilities in the classification problem, Comm. Statist. A Theory
Methods 12, pp. 1449-1479.
Vrankar, L., Turk, G. and Runovc, F. (2004). modelling of radionuclide migration through
the geosphere with radial basis function method and geostatistics, J . Chinese Inst.
Engineers 27, pp. 455-462.
Wahba, G. (1979). Convergence rate of "thin plate" smoothing splines when the data are
noisy (preliminary report), Springer Lecture Notes in Math. 757, pp. 233-245.
Wahba, G. (1981). Spline Interpolation and smoothing on the sphere, SIAM J . Sci. Statist.
Comput. 2, pp. 5-16.
Wahba, G. (1982). Erratum: Spline interpolation and smoothing on the sphere, SIAM J .
Sci. Statist. Comput. 3, pp. 385-386.
Wahba, G. (1986). Multivariate thin plate spline smoothing with positivity and other
linear inequality constraints, in Statistical image processing and graphics (Luray,
Va., 1983), Statist.: Textbooks Monographs, 72, Dekker (New York), pp. 275-289.
Wahba, G. (1990a). Multivariate model building with additive interaction and tensor prod-
uct thin plate splines, in Curves and Surfaces, P.-J. Laurent, A. Le Mehaute, and L.
L. Schumaker (eds.), Academic Press (New York), pp. 491-504.
Wahba, G. (1990b). Spline Models for Observational Data, CBMS-NSF Regional Confer-
ence Series in Applied Mathematics 59, SIAM (Philadelphia).
Wahba, G. and Luo, Z. (1997). Smoothing spline ANOVA fits for very large, nearly regular
data sets, with application to historical global climate data, in The Heritage of
P. L . Chebyshev: a Festschrift in honor of the 70th birthday of T. J . Rivlin, Ann.
Numer. Math. 4 1-4, pp. 579-597.
Wahba, G. and Wendelberger, J. (1980). Some new mathematical methods for variational
objective analysis using splines and cross validation, Monthly Weather Review 108,
pp. 1122-1143.
Wang, S. and Wang, M. Y. (2006). Radial basis functions and level set method for struc-
tural topology optimization, Int. J . Numer. Meth. Engng. 65 12, pp. 2060-2090.
Ward, J. D. (2004). Least squares approximation using radial basis functions: an up-
date, in Advances in Constructive Approximation: Vanderbilt 2003, M. Neamtu and
E. B. Saff (eds.), Nashboro Press (Brentwood, TN), pp. 499-508.
Watson, G. S. (1964). Smooth regression analysis, Sankhya, Ser. A 26, pp. 359-372.
Webb, A. R. (1995). Multidimensional scaling by iterative majorisation using radial basis
functions, Pattern Recognition 28, pp. 753-759.
Weinrich, M. (1994). Charakterisierung von Funktionenraumen bei der Interpolationn mit
radialen Basisfunktionen, Ph.D. Dissertation, Universitat Gottingen.
Wells, J. H. and Williams, R. L. (1975). Embeddings and Extensions in Analysis, Springer
(Berlin).
Wendland, H. (1994). Ein Beitrag zur Interpolation mit radialen Basisfunktionen, Diplo-
marbeit, Universitat Gottingen.
Bibliography 487
Wendland, H. (1995). Piecewise polynomial, positive definite and compactly supported

radial functions of minimal degree, Adv. in Comput. Math. 4, pp. 389-396.
Wendland, H. (1997). Sobolev-type error estimates for interpolation by radial basis func-
tions, in Surface Fitting and Multiresolution Methods, A. Le Mehaute, C. Rabut, and
L. L. Schumaker (eds.), Vanderbilt University Press (Nashville, TN), pp. 337-344.
Wendland, H. (1998). Error estimates for interpolation by compactly supported radial
basis functions of minimal degree, J. Approx. Theory 93, pp. 258-272.
Wendland, H. (1999a). Meshless Galerkin approximation using radial basis functions,
Math. Comp. 68, pp. 1521-1531.
Wendland, H. (1999b). Numerical solution of variational problems by radial basis functions,
in Approximation Theory IX, Vol.11: Computational Aspects, Charles K. Chui, and
Wendland, H. (2001a). Local polynomial reproduction and moving least squares approxi-
mation, IMA J. Numer. Anal. 21 1, pp. 285-300.
Wendland, H. (2001b). Gaussian interpolation revisited, in Trends in Approximation The-
ory, K. Kopotun, T. Lyche, and M. Neamtu (eds.), Vanderbilt University Press,
pp. 417-426.
Wendland, H. (2001c). Moving least squares approximation on the sphere, in Mathematical
Methods for Curves and Surfaces: Oslo 2000, T. Lyche and L. L. Schumaker (eds.),
Vanderbilt University Press, pp. 517-526.
Wendland, H. (2002a). Fast evaluation of radial basis functions: Methods based on par-
tition of unity, in Approximation Theory X: Wavelets, Splines, and Applications,
C. K. Chui, L. L. Schumaker, and J. Stockier (eds.), Vanderbilt University Press
(Nashville), 473-483.
Wendland, H. (2002b). Surface reconstruction from unorganized points,
http://www.num.math.uni-goettingen.de/wendland/Forschung/recon-
html/reconhtml.html.
Wendland, H. (2004). Solving large generalized interpolation problems efficiently, in Ad-
vances in Constructive Approximation: Vanderbilt 2003, M. Neamtu and E. B. Saff
(eds.), Nashboro Press, Brentwood, TN, pp. 509-518.
Wendland, H. (2005a). Scattered Data Approximation, Cambridge University Press (Cam-
bridge).
Wendland, H. (2005b). On the convergence of a general class of finite volume methods,
SIAM J. Numer. Anal. 43, pp. 987-1002.
Wendland, H. (2005c). Private communications.
Wendland, H. and Rieger, C. (2005). Approximate interpolation with applications to se-
lecting smoothing parameters, Numer. Math. 101 4, pp. 729-748.
Wertz, J., Kansa, E. J. and Ling, L. (2006). The role of the multiquadric shape parameters
in solving elliptic partial differential equations, Comput. Math. Appl. 51 8, pp. 1335
1348.
Wetterschereck, D. and Dietterich, T. (1992). Improving the performance of radial basis
function networks by learning center locations, in Advances in Neural Information
Processing Systems 4- Proceedings of the 1991 Conference, J. E. Moody, S. J. Hanson,
and R. P. Lippmann (eds.), Morgan Kaufmann (San Mateo, CA), pp. 1133-1140.
Whittaker, J. M. (1915). On the functions which are represented by expansions of the
interpolation theory, Proc. Roy. Soc. Edinburgh 35, pp. 181-194.
Whittaker, E. T. (1923). On a new method of graduation, Proc. Edinburgh Math. Soc. 41,
pp. 63-75.
Whitehead, B. A. (1996). Genetic evolution of radial basis function coverage using orthog-
onal niches, IEEE Trans. Neural Networks 7, pp. 1525-1528.
Whitehead, B. A. and Choate, T. D. (1996). Cooperative-competitive genetic evolution

of radial basis function centers and widths for time series prediction, IEEE Trans.
Neural Networks 7, pp. 869-880.
Widder, D. V. (1941). The Laplace Transform, Princeton University Press (Princeton).
Williamson, R. E. (1956). Multiply monotone functions and their Laplace transform, Duke
Math. J. 23, pp. 189-207.
Wong, S. M., Hon, Y. C. and Li, T. S. (1999). Radial basis functions with compactly
support and multizone decomposition: applications to environmental modelling, in
Boundary Element Technology XIII, C. S. Chen, C. A. Brebbia, and D. W. Pepper
(eds.), W I T Press, pp. 355-364.
Wong, T.-T., Luk, W.-S. and Heng, P.-A. (1997). Sampling with Hammersley and Halton
points, J. Graphics Tools 2, pp. 9-24.
Woolhouse, W. S. B. (1870). Explanation of a new method of adjusting mortality tables,
with some observations upon Mr. Makeham's modification of Gompertz's theory, J.
Inst. Act. 15, pp. 389-410.
Wright, G. B. (2003). Radial Basis Function Interpolation: Numerical and Analytical
Developments, Ph.D. Dissertation, University of Colorado at Boulder.
Wright, G. and Fornberg, B. (2006). Scattered node compact finite difference-type formulas
generated from radial basis functions, J. Comput. Physics 212 1, pp. 99-123.
Wu, Z. (1986). Die Kriging-Methode zur Losung mehrdimensionaler Interpolationsprob-
leme, Ph.D. Dissertation, Universitat Gottingen.
Wu, Z. (1990). A class of convexity-preserving bases for radial basis interpolation (in
Chinese), Math. Appl. 3, pp. 33-37.
Wu, Z. (1991). A convergence analysis for a class of radial basis interpolations (in Chinese),
Gaoxiao Yingyong Shuxue Xuebao 6, pp. 331-336.
Wu, Z. (1992). Hermite-Birkhoff interpolation of scattered data by radial basis functions,
Approx. Theory Appl. 8, pp. 1-10.
Wu, Z. (1993a). On the convergence of the interpolation with radial basis function, Chinese
J. Contemp. Math. 14, pp. 269-277.
Wu, Z. (1993b). Convergence of interpolation by radial basis functions (in Chinese), Chi-
nese Ann. Math. Ser. A 14, pp. 480-486.
Wu, Z. (1995a). Characterization of positive definite radial functions, in Mathematical
Methods for Curves and Surfaces, M. Daehlen, T. Lyche, and L. Schumaker (eds.),
Vanderbilt University Press (Nashville), pp. 573-578.
Wu, Z. (1995b). Compactly supported positive definite radial functions, Adv. in Comput.
Math. 4, pp. 283-292.
Wu, Z. (2005). Dynamical knot and shape parameter setting for simulating shock wave
by using multi-quadric quasi-interpolation, Engineering Analysis with Boundary El-
ements 29, pp. 354-358.
Wu, Z. M. and Liu, J. P. (2005). Generalized Strang-Fix condition for scattered data
quasi-interpolation, Adv. in Comput. Math. 23, pp. 201-214.
Wu, Z. and Schaback, R. (1993). Local error estimates for radial basis function interpola-
tion of scattered data, IMA J. Numer. Anal. 13, pp. 13-27.
Wu, Z. and Schaback, R. (1994). Shape preserving properties and convergence of uni-
variate multiquadric quasi-interpolation, Acta Math. Appl. Sinica (English Ser.) 10,
pp. 441-446.
Xu, L., Kryzak, A. and Yuille, A. (1994). On radial basis function nets and kernel regression
statistical consistency, convergence rates, and receptive field sizes, Neural Networks
7, pp. 609-628.
Bibliography 489
Xu, Y. and Cheney, E. W. (1992a). Interpolation by periodic radial functions, Comput.

Math. Appl. 24, pp. 201-215.
Xu, Y. and Cheney, E. W. (1992b). Strictly positive definite functions on spheres, Proc.
Amer. Math. Soc. 116, pp. 977-981.
Xu, Y., Light, W. A. and Cheney, E. W. (1993). Constructive methods of approximation
by ridge functions and radial functions, Numer. Algorithms 4, pp. 205-223.
Yamada, T. and Wrobel, L. C. (1993). Properties of Gaussian radial basis functions in the
dual reciprocity boundary element method, Z. Angew. Math. Phys. 44, pp. 1054
1067.
Yang, Y. and Barron, A. (1999). Information-theoretic determination of minimax rates of
convergence, Ann. Statist. 27, pp. 1564-1599.
Yee, P. V. and Haykin, S. (2001). Regularized Radial Basis Function Networks: Theory
and Applications, Wiley-Interscience.
Yoon, J. (2001). Interpolation by radial basis functions on Sobolev space, J . Approx.
Theory 112, pp. 1-15.
Yoon, J. (2003). L -error estimates for "shifted" surface spline interpolation on Sobolev
p
space, Math. Comp. 72, pp. 1349-1367.

Yoon, Y.-C, Lee, S.-H. and Belytschko, T. (2006). Enriched meshfree collocation method
with diffuse derivatives for elastic fracture, Comput. Math. Appl. 51 8, pp. 1349
1366.
Young, D. L., Jane, S. C, Lin, C. Y., Chiu, C. L. and Chen, K. C. (2004). Solutions
of 2D and 3D Stokes laws using multiquadrics method, Engineering Analysis with
Boundary Elements 28, pp. 1233-1243.
Zastavnyi, V. P. (1993). Positive definite functions depending on the norm, Russian J .
Math. Phys. 1, pp. 511-522.
Zeckzer, D. (1992). Dreiecks-basierte lokale Scattered Data Interpolation unter Verwen-
dung radialer Basismethoden, Diplomarbeit, Universitat Kaiserslautern.
Index
adaptive greedy algorithm, 291 best, 159, 163, 178, 192, 193, 254, 300
adaptive least squares approximation, best K-term, 293
181, 184 greedy, 293
additive Schwarz, 331 non-stationary, 22, 99-101, 126, 128,
algorithm 131, 139, 140, 153-155, 211,
adaptive greedy, 291 267, 378, 421
Contour-Pade, 133, 151, 405 saturated, 130, 156, 231, 237, 240, 246,
domain decomposition, 332 385
fast Gauss transform, 325 sparse, 293
Faul-Powell, 298, 301 stationary, 22, 99-101, 130-132, 140,
F F T evaluation, 245 155-157, 211, 216, 227, 237,
fixed level iteration, 267 279, 378, 380, 421
greedy one-point, 293 stationary multilevel, 277, 283
iterative refinement, 265 approximation order, 111, see also error
knot insertion, 181 estimates
knot removal, 184 L , 112
p
multilevel Galerkin, 421 and moment conditions, 230, 232

nested multilevel Galerkin, 421 exact, 128, 132
Rippa's leave-one-out cross validation, exact L , 112
p
146, 150 exponential, 126

stationary multilevel collocation, 380 L , 179
2
stationary multilevel interpolation, 277 lower bounds, 130

surface reconstruction, 256 numerical evidence, 141-157
Allen-Cahn equation, 409 of Laguerre-Gaussians, 237
almost negative definite, 81 of MLS approximation, 196, 225
applications, 1, 13, 351, 416, 422 of partition of unity, 250
approximate approximation, 99, 131, 156, of Shepard's method, 211, 226
203, 230, 270, 385 spectral, 126, 153, 288
numerical experiments, 237 squaring of, 127
approximate cardinal functions, 298, 301, super-spectral, 126, 153
309, 310 augmented system, 56, 59, 60, 64, 305
approximate inverse, 265, 304 auxiliary data, 256
approximation
approximate, 99, 131, 156, 203, 230, S-spline, 92, 93
270, 385 Backus-Gilbert method, 194
numerical experiments, 237 band-limited functions, 40, 110, 126
491
basic function, 6, 18 non-symmetric, 335, 345, 353, 392

basis preconditioned, 310
bi-orthogonal, 200 symmetric, 348, 365
change of, 311 with CSRBFs, 375
dual, 200 collocation approach, 345
basis function, 3, 6 collocation points, 346
Beppo-Levi space, 109, 129, 163, 164 collocation solution, 392
(semi-)norm, 109, 163 compactly supported radial basis
Bessel function functions, 15, 76, 85-101, 109, 127, 138,
modified of the second kind, 41, 67 140, 149, 234, 240, 241, 251, 260,
modified of the third kind, 41 277-289, 345, 375-385, 421
of the first kind, 34, 39, 85, 432 compactly supported strictly positive
Bessel kernels, 41 definite functions, 35, 42
best K-tevva approximation, 293 completely monotone functions, 14, 38,
best approximation, 159, 163, 178, 192, 42, 47, 47-49, 51, 52, 73-75
193, 254, 300 properties, 47
bi-orthogonal basis, 200, 206 computer graphics, 2, 13, 255
bilinear form, 106, 107, 419 condition number, 23, 135, 137, 139, 142,
Bochner's theorem, 28, 31, 33, 65, 82 172, 273, 303-314, 343, 356, 372, 425
Borel measure, 31, 32, 34, 35, 48, 50, 431 conditionally positive definite functions,
Borel cr-algebra, 431 52, 63-65, 123, 162, 299
boundary centers, 354, 368, 375 conditionally positive definite of order m
boundary conditions, 390 on W, 63, 64
and boundary centers, 354 conditionally positive definite of order
change of, 346 one, 60
Dirichlet, 346, 348, 353, 358, 365, 378, conditionally positive definite radial
381, 390-392, 394, 397, 420 functions, 73-78, 81
essential, 420 conditionally positive semi-definite of
for multilevel interpolation, 278 order one, 60
for native space, 127 constrained optimization, 165, 192, 197,
homogeneous, 391 202, 420
implementation, 361, 372, 391, 393, 396 containment query, 428
mixed, 361 continuous moment conditions, 231, 232
natural, 419, 421, 423 Contour-Pade algorithm, 133, 151, 405
Neumann, 419 convergence
piecewise defined, 370, 415 rate of, 99
time-dependent, 409 correction function, 230
Buhmann's functions, 93, 94, 345 Coulomb potential, 44
Courant-Fischer theorem, 76, 136
cardinal basis functions, 112, 113-115, covariance, 312
151, 164, 195, 196, 199, 311 covering radius, 22
approximate, 298, 301, 309, 310 cross validation, 14, 146-150, 167
local approximate, 309 leave-one-out, 146, 148, 150, 185, 401
polynomials, 314 CSRBF, see compactly supported radial
carrier, 32, 431 basis function, function
centers, 6, 17, 334, 346 cutoff function, 43, 88, 128
change of basis, 311
clustered data, 178, 339 data files, 20
collocation, 333, 335, 388, 420, 422 data mining, 2
multilevel, 380 data sites, 2
Index 493
data values, 2 generic, in terms of power function, 117

Delaunay triangulation, 294, 306, 439 improvements, 127
derivative matrix, see differentiation with respect to shape parameter, 132
matrix essential boundary conditions, 420
derivatives Euclid's hat, 92, 93
of RBFs, 444-449 Euclidean norm, 17
higher-order, 407 evaluation matrix, 8, 20, 95, 144, 212, 214,
of Gaussian, 365 223, 238, 366, 381, 389, 395
of generic radial function, 338, 443 evaluation points, 10
of IMQ, 358, 361, 365 exact Lp-approximation order k, 112
of MQ, 340, 365 exact approximation order, 132
6
of Wendland C CSRBF, 375, 381
descente, 85 fast Fourier transform, 151, 245
differentiation matrix, 387-391, 401-403, fast Gauss transform, 322, 325
412 fast multipole method, 321
higher-order, 407 fast tree codes, 327
dimension walk, 85, 89, 90 Faul-Powell algorithm, 298, 301
Dirichlet boundary conditions, 346, 348, feature, 159
353, 358, 365, 378, 381, 390-392, 394, F F T evaluation algorithm, 245
397, 420 fill distance, 2 2 , 111
Dirichlet tesselation, 306 fixed level iteration algorithm, 267
discrete Gauss transform, 322 Fourier transform, 31-33, 110, 231, 4 3 2
discrete moment conditions, 203, 230 fast, 151, 245
discrete moments, 229 fast inverse, for non-uniformly spaced
discrete weighted least squares, 191 points, 244
distance matrix, 2, 6 fast, for non-uniformly spaced points,
domain decomposition, 331, 332, 350, 422 243, 244
algorithm, 332 generalized, 65, 4 3 3
dual, 104, 159 of generalized MQ, 67
dual basis, 200, 206, 222 of radial powers, 69
dual representation, 200, 201, 206 of thin plate splines, 70
inverse, 4 3 2
eigenfunctions, 107, 201 inverse discrete, 244
energy split, 161, 291 of a measure, 432
error estimate, 111-123, 125-133 of a radial function, 51, 85, 432
for approximate approximation, 231 of Gaussians, 37
for derivatives, 123 of Laguerre-Gaussians, 38
for fast Gauss transform, 325 of Matern functions, 41
for Gaussians, 126 of Poisson radial functions, 40
for least squares approximation, 179 of truncated power functions, 43
for Matern functions, 126 Fourier transform characterization, 34, 65
for MLS approximation, 226 Franke's function, 20
for multiquadrics, 125 Franke-type function, 142, 241, 246, 283
for radial powers, 127 function
for R B F Galerkin method, 420 approximate cardinal, 298, 301, 309,
for rough functions, 129, 131 310
for symmetric P D E collocation, 349 band-limited, 40, 110, 126
for thin plate splines, 127 basic, 6, 18
for Wendland CSRBF, 127 basis, 3, 6
generic, in terms of fill distance, 121 Bessel of the first kind, 34, 39, 85, 432
Buhmann CSRBF, 93, 94, 345 modified Bessel of the second kind, 41,
cardinal basis, 112, 113-115, 151, 164, 67
195, 196, 199, 311 modified Bessel of the third kind, 41
compactly supported RBF, 15, 76, multiply monotone, 49-52, 75-76
85-101, 109, 127, 138, 140, 149, multiquadric, 13, 68, 77, 110, 113, 126,
234, 240, 241, 251, 260, 131-133, 139, 140, 153, 243, 288,
277-289, 345, 375-385, 421 306, 308, 310, 326, 339, 345, 346
completely monotone, 14, 38, 42, 47, multivariate, 17
47-49, 51, 52, 73-75 multivariate Hermite, 322
conditionally positive definite, 52, optimal basis, 164
63-65, 123, 162, 299 Poisson radial, 39
conditionally positive definite and polyharmonic spline, 14, 71, 129, 130,
radial, 73-78, 81 278, 306, 313
conditionally positive definite of order positive definite, 27-35
s
s
m o n K , 63, 64 positive definite on R , 28
construct strictly positive definite, 33 positive definite radial, 33-35
correction, 230 R, 350, 420
cutoff, 43, 88, 128 radial, 17, 33
eigen, 107 radial basis, 6, 17
Euclid's hat, 92, 93 radial power, 69, 74, 109, 127-129, 131,
Franke's, 20 132, 138, 155, 278, 306
Franke-type, 246 rapidly decreasing, 433
Shepard, 13, 205
fundamental positive definite, 32
slowly increasing, 433
Gaussian, 17, 19, 22-24, 35, 37, 40, 49,
Sobolev spline, 41, 109
52, 55, 82, 101, 110, 113, 117,
strictly conditionally positive definite of
123, 126, 130, 132, 133, 137, 8
order m o n R , 63, 64
140, 148, 153, 156, 188, 211,
strictly positive definite, 28, 32
243, 253, 259, 270, 296, 309,
strictly positive definite and radial for
322, 327, 345, 356, 357, 363,
all s, 34
403, 405, 412, 415 s
strictly positive definite on R , 28
generalized inverse multiquadric, 41, 49,
surface spline, 14, 70
67, 123, 138 shifted, 131
generalized multiquadric, 67, 74, 77, thin plate spline, 14, 70, 74, 109,
138 127-129, 131, 132, 138, 155,
generating, 195, 232 163, 164, 167, 170, 278, 306,
Gneiting CSRBF, 91, 345 308, 310, 314, 316, 326, 437
Green's, 164 tri-cube, 227
inverse multiquadric, 37, 154, 296, 306, truncated power, 43, 50
345, 353, 356, 357 univariate, 17
inverse quadratic, 42 vector-valued, 2, 83
A;-times monotone, 50, 50, 51, 75, 76 weight, 193, 196, 199, 201, 202, 206,
kriging, 116 212, 214, 216, 226, 227, 229,
Laguerre-Gaussian, 38, 52, 123, 126, 230, 233, 234, 274, 285, 287
131, 222, 233, 237, 238, 241, Wendland CSRBF, 87, 87-88, 91, 92,
245, 246, 269, 322 98, 99, 109, 127, 129, 131, 138,
Lebesgue, 226 154, 179, 211, 234, 240, 241,
local approximate cardinal, 309 251, 253, 260, 279, 345, 375,
MacDonald's, 41 388, 413, 423
Matern, 41, 109, 123, 126, 129, 388, 410 Whittaker-M, 44
Index 495
Wu CSRBF, 89, 88-90, 345 homogeneous, 278, 311, 317

functional homogeneous kernel, 312, 313, 319
dual, 104 homogeneous Sobolev spaces of order k,
information, 159 109
functions
generating, 197 ill-posed problem, 390
fundamental positive definite function, 32 implicit surfaces, 255
information functional, 159
Galerkin system, 419, 421 inner points, 331
Gaussian, 17, 19, 22-24, 35, 37, 40, 49, 52, inner product, 28, 103, 106-108, 168, 177,
55, 82, 101, 110, 113, 117, 123, 126, 130, 191, 192, 200
132, 133, 137, 140, 148, 153, 156, 188, integral characterization, 31-33
211, 243, 253, 259, 270, 296, 309, 322, integral operator, 50, 76, 86
327, 345, 356, 357, 363, 403, 405, 412, integral transforms, 432
415 integrally positive definite, 107
generalized covariance, 312 interaction region, 323
generalized derivative, 109 interior cone condition, 120
generalized Fourier transform, 65, 433 interpolation, 2
generalized Hermite interpolation, 333, generalized Hermite, 333, 348, 390
348, 390 scattered data, 2
generalized interpolation conditions, 334 interpolation matrix, 3, 8, 20
generalized inverse multiquadric, 49 sparse, 95-98
generalized inverse multiquadrics, 41, 67, interpolation theorem, 49, 64, 77
123, 138 inverse Fourier transform, 244, 432
generalized Laguerre polynomials, 38, 233 inverse multiquadric, 37, 154, 296, 306,
generalized multiquadrics, 67, 74, 77, 138 345, 353, 356, 357
generating functions, 195, 197 inverse quadratic, 42
construction of, 232 iterative refinement algorithm, 265
global variable, 193
GMRES, 309, 350 Jacobi polynomials, 234
Gneiting's functions, 91, 345
Gram matrix, 177, 178, 194, 197, 207, 220 fc-times monotone, 50, 50, 51, 75, 76
greedy algorithm, 184 Kansa's matrix, 346, 347, 393, 394,
greedy approximation algorithms, 293 396-398
greedy one-point algorithm, 293 Kansa's method, 335, 345, 353, 392
Green's functions, 164 kd-tree, 95, 98, 216, 428
kernel
Haar space, 4 Bessel, 41
Halton points, 5, 427 covariance, 312
Hammersley points, 428 generalized covariance, 312
Hankel inversion theorem, 41, 432 homogeneous, 312, 313, 319
Hankel transform, 432 integrally positive definite, 107
Hausdorff-Bernstein-Widder Theorem, 48 multiscale, 278
Helmholtz equation, 411, 419, 423 reproducing, 103, 103-108, 311
Hermite interpolation kernel method, 205
generalized, 333, 348, 390 knot insertion algorithm, 181
Hermite polynomials (multivariate), 322 knot removal algorithm, 184
Hermite-based collocation, 348, 365 kriging, 312
high-order method, 229, 291, 419 kriging function, 116
history of meshfree approximation, 13 Kronecker tensor-product, 412
i lST
Lagrange form, 112 evaluation, 20, 389

Lagrange multipliers, 165, 166, 197, Gram, 177, 178, 194, 197, 207, 220
200-202, 208, 209, 216, 217, 222, 420 higher-order differentiation, 407
Laguerre-Gaussians, 38, 52, 123, 126, 131, interpolation, 3, 8, 20
222, 233, 237, 238, 241, 245, 246, 269, sparse, 95-98
322 Kansa's, 346, 347, 393, 394, 396-398
Laplace transform, 48, 432 negative definite, 60
of a measure, 433 positive definite, 27
Laplace's equation, 415 positive semi-definite, 27
Laurent series, 151 stiffness, 420, 423
learning theory, 2, 13, 163 measure, 431
least squares Borel, 31, 32, 34, 35, 48, 50, 431
adaptive, 181, 184 carrier, 32, 431
approximation, 168, 177 Mercer's theorem, 107
discrete weighted, 191 meshfree, 1, 12
moving, 14, 191, 192, 194, 198, 207, 216 meshless, 12
properties, 198 meshsize, 22
nonlinear, 187 Micchelli's theorem, 51
penalized, 167 minimal eigenvalue
regularized, 166 upper bound, 137
smoothing, 170 for Gaussian, 137
leave-one-out cross validation, 146, 148, for generalized multiquadrics, 138
150, 185, 401 for radial powers, 138
Lebesgue constant, 120 for thin plate splines, 138
Lebesgue function, 226, 227, 269 for Wendland CSRBF, 138
local approximate cardinal functions, 309 minimum norm interpolant, 162, 166
local polynomial regression, 202 mixed boundary conditions, 361
local polynomial reproduction, 120 MLS, s e e moving least squares
local variable, 193 modified Bessel function of the second
Lp-approximation order k, 112 kind, 41
modified Bessel function of the third kind,
m-unisolvent, 53 41
MacDonald's function, 41 moment conditions, 203
Mairhuber-Curtis theorem, 3 and approximation order, 230, 232
manifolds, 83, 255 continuous, 231, 232
Maple programs, s e e program discrete, 203, 230
marching cube, 257 moments, 207, 323, 328
Matern functions, 41, 109, 123, 126, 129, discrete, 229
388, 410 montee, 85
mathematical finance, 2, 13 moving least squares approximation, 14,
M A T L A B programs, see program 191, 207, 216
matrix Backus-Gilbert approach, 194
almost negative definite, 81 equivalence of formulations, 198
augmented, 56, 59, 60, 64, 305 iterated approximate, 270
conditionally positive definite of order properties, 198
one, 60 standard interpretation, 192
conditionally positive semi-definite of multigrid algorithm, 332
order one, 60 multilevel Galerkin algorithm, 421
differentiation, 387-391, 401, 403, 412 multilevel interpolation, 277
distance, 2, 6 multiplicative Schwarz, 331
Index 497
multiply monotone functions, 49-52, montee, 85

75-76 turning bands, 91
multiquadric, 13, 68, 77, 110, 113, 126, optimal basis functions, 164
131-133, 139, 140, 153, 243, 288, 306, optimal recovery, 159, 165
308, 310, 326, 339, 345 optimality properties, 160
multiquadric method, 346 optimality theorem I, 162
multiscale kernels, 278 optimality theorem II, 163
multivariate, 17 optimality theorem III, 164
multivariate Hermite functions, 322 optimization, 2
constrained, 165, 192, 197, 420
native space, 103, 105, 106 constrained quadratic, 202
native space norm, 119, 166, 278 orthogonal projection, 163
native space semi-norm, 123 oscillatory strictly positive definite
natural boundary conditions, 419, 421, functions, 35
423 overlapping domains, 331
nearest neighbor, 227, 298, 310, 323, 428
negative definite, 60 p-norms, 79-82
nested multilevel Galerkin algorithm, 421 packing radius, 136
neural networks, 2, 13, 172 partial differential equation
NFFT, see Fourier transform Allen-Cahn, 409
noisy data, 14, 165, 170-175, 212 elliptic with variable coefficients, 358
non-stationary approximation, 22, 99-101, Helmholtz, 411, 419, 423
126, 128, 131, 139, 140, 153-155, 211, Laplace, 415
267, 378, 421 linear elliptic, 346, 390
non-symmetric method, 345 nonlinear reaction diffusion, 409
non-symmetric pseudospectral method, solution of, 1
391 transport equation, 387, 403, 405
non-uniform sampling, 2 partition of unity, 206, 229, 249
nonlinear least squares, 187 patch test, 55
nonlinear reaction-diffusion equation, 409 PDE, see partial differential equation
norm, 17 penalized least squares, 167
basic function, 156, 313 piecewise defined boundary conditions,
equivalence, 293 370
Euclidean, 17 piecewise linear spline, 5
native space, 119, 166, 278 plane wave, 325
Sobolev space, 109 point cloud data, 255
weighted, 191 point cloud modeling, 257, 260
(p-)norm distance matrix, 8 Poisson problem, 353, 361, 365, 378, 381
(semi-) norm Poisson radial functions, 39
Beppo-Levi space, 109, 163 polyharmonic splines, 14, 71, 129, 130,
norm invariance, 18 278, 306, 313
normal equations, 177, 192, 194 polynomial
(multivariate) Hermite, 322
off-surface points, 255 generalized Laguerre, 38, 233
operator Jacobi, 234
descente, 85 polynomial precision, 119
differential, 86 polynomial reproduction, 55-59, 64, 195,
discrete differential, 398 203, 207, 216, 305
for radial functions, 85 local, 120
integral, 50, 76, 86 positive definite function, 27-35
criterion to check, 33 PU2D_CS.m, 251

properties, 29 RBFApproximation2D.m, 169
positive definite matrix, 27 RBFApproximation2Dlinear.m, 171
s
positive definite on M. , 28 RBFCardinalFunction.m, 114
positive definite radial function, 33-35 RBFGalerkin2D.m, 424
positive semi-definite matrix, 27 RBFGreedyOnePoint2D.m, 294
power function, 115, 116, 119, 121, 128, RBFHermite_2D.m, 341
139, 143 RBFInterpolation2D.m, 21
preconditioned conjugate gradient, 98, RBFInterpolation2Dlinear.m, 56
303, 309, 379 RBFInterpolation2DtpsH.m, 317
preconditioning, 303-313 RBFKnotInsert2D.m, 182
program RBFKnotRemove2D.m, 185
ApproxMLSApproxlD.m, 239 Shepard2D.m, 212
CostEpsilon.m, 150 Shepard_CS.m, 215
CostEpsilonDRBF.m, 402 sinc.m, 435
D2RBF.m, 409 testfunction.m, 435
DifferenceMatrix.m, 342 Thin.m, 439
DistanceMatrix.m, 7 tps.m, 437
DistanceMatrixCSRBF.m, 96 TPS_RidgeRegression2D.m, 173
DistanceMatrixFit.m, 8 tpsK.m, 314
DRBF.m, 402 TransportDRBF.m, 403
HermiteLaplace_2D.m, 366 properties of positive definite functions, 29
HermiteLaplace_2D_CSRBF.m, 376 pseudospectral method
HermiteLaplaceMixedBCTref_2D.m, non-symmetric, 391
370 symmetric, 394
IteratedJVILSApproxApprox2D.m, 271
KansaEllipticVC_2D.m, 358 quadratic form, 27, 165
kansaLaplace_2D.m, 355 quasi-interpolant, 194, 201, 206, 208, 225,
kansaLaplaceMixedBC_2D.m, 361 229, 230, 267, 322
LinearMLS2D_CS.m, 219 evaluation of, 243, 327
LinearMLS2D_GramSolve.m, 220 optimal, 164
LinearScaling2D_CS.m, 217
LOOCV2D.m, 147 .R-functions, 350, 420
LOOCV2Dmin.m, 150 radial, 33
LRBF.m, 414 radial basis function, 6, 17
MakeSDGrid.m, 436 radial function, 17
ML_CSRBF3D.m, 279 radial powers, 69, 74, 109, 127-129, 131,
MLJrIermiteLaplaceCSRBF2D.m, 381 132, 138, 155, 278, 306
MLSDualBases.mws, 440 range search, 428
pl7.m, 412 rapidly decreasing test functions, 433
pl7_2D.m, 414 rate of convergence, 99
p35.m, 409 ray tracing, 257
PlotError2D.m, 437 Rayleigh quotient, 136
PlotErrorSlices.m, 439 RBF, see radial basis function, function
Plotlsosurf.m, 438 reaction-diffusion equation, 409
PlotSlices.m, 438 regression
PlotSurf.m, 437
local, 212, 227
PointCloud2D.m, 257
local polynomial, 202
PointCloud3D_PUCS.m, 261
ridge, 167
Powerfunction2D.m, 144
rigde, 257
' ^ ^ ^ ^ ^
Index 499
regression spline, 170 spectral approximation order, 126, 153,

regularization, 390 288
regularization theory, 167 spline
reproducing kernel, 103, 106-108, 311 B-, 92, 93
for conditionally positive definite basic piecewise linear, 5
function, 311 polyharmonic, 14, 71, 129, 130, 278,
properties, 104 306, 313
reproducing kernel Hilbert space, 103, regression, 170
103-105, 107 shifted surface, 131
residual iteration, 268 smoothing, 167
ridge regression, 167, 257 Sobolev, 41, 109
RKHS, see reproducing kernel Hilbert surface, 14, 70
space, 103 thin plate, 14, 70, 74, 109, 127-129,
RMS-error, see root-mean-square error 131, 132, 138, 155, 163, 164,
root-mean-square error, 10 167, 170, 278, 306, 308, 310,
314, 316, 326, 437
sampling theory, 13, 110, 164 web-, 350, 420
saturation, 130, 156, 237, 240, 246, 385 spread, 195
saturation error, 231 stability, 23, 131, 135-140, 193, 303, 331,
scattered data interpolation, 2 347, 350
scattered data modeling, 1 stationary approximation, 22, 99-101,
Schoenberg-Menger Theorem, 81 130-132, 140, 155-157, 211, 216, 227,
Schwartz space, 433 237, 277, 279, 378, 421
(semi-)norm stationary multilevel collocation
algorithm, 380
Beppo-Levi space, 109, 163
stationary multilevel interpolation
Shannon sampling theorem, 110
algorithm, 277
shape parameter, 17, 37
stiffness matrix, 420, 423
choice of, 141-150
strictly conditionally positive definite
convergence with respect to, 132-133
function
shape parameter free, 69
Fourier transform characterization, 65
Shepard function, 13, 205
strictly conditionally positive definite of
Shepard's method, 205, 211 5
order rnonl , 63, 64
high-order, 229 strictly positive definite and radial for all
iterated, 274 s, 34, 49
shifted surface splines, 131 Schoenberg's characterization, 35
cr-algebra, 431 strictly positive definite function, 28, 32
slowly increasing functions, 433 compactly supported, 35, 42
smoothing, 167, 211, 240, 257, 297, 308, Fourier transform characterization, 34
385 oscillatory, 35, 39, 90
of noisy data, 170-175 s
strictly positive definite on R , 28
smoothing splines, 167 strong form solution, 345
Sobolev space, 108, 128, 130-132, 179, 228 super-spectral approximation order, 126,
homogeneous of orderfc,109 153
norm, 109 surface reconstruction algorithm, 256
Sobolev splines, 41, 109 surface splines, 14, 70
sources, 323 symmetric formulation, 335
sparse approximation, 293 symmetric pseudospectral method, 394
special point, 317
special points, 310, 313 targets, 323
I
Taylor expansion, 119, 120, 193, 226, 323, tri-cube, 227

327-331 trial and error, 142
theorem truncated power functions, 43, 50
Bochner, 28, 31, 33, 65, 82 turning bands operator, 91
Courant-Fischer, 76, 136
Hankel inversion, 41, 432 uncertainty principle, 24, 139, 277
Hausdorff-Bernstein-Widder, 48 univariate, 17
interpolation, 49, 64, 77 upper bound for A i , 137
m n
Mairhuber-Curtis, 3
Mercer, 107 van der Corput sequence, 427
Micchelli, 51 vector-valued functions, 2, 83
optimality I, 162 Voronoi diagram, 306
optimality II, 163
optimality III, 164 web-spline, 350, 420
Schoenberg-Menger, 81 weight functions, 193, 196, 199, 201, 202,
Shannon Sampling, 110 206, 212, 214, 216, 226, 227, 229, 230,
Williamson, 50 233, 234, 274, 285, 287
zeros, 128 weighted norm, 191
thin plate spline, 308 well-posed, 3
thin plate splines, 14, 70, 74, 109, Wendland CSRBF, 87, 87-88, 91, 92, 98,
127-129, 131, 132, 138, 155, 163, 164, 99, 109, 127, 129, 131, 138, 154, 179,
167, 170, 278, 306, 310, 314, 316, 326, 211, 234, 240, 241, 251, 253, 260, 279,
437 345, 375, 388, 413, 423
thinning algorithm, 187, 282, 439 Whittaker-M function, 44
time-dependent boundary conditions, 409 Williamson's theorem, 50
trade-off principle, 24, 100, 138-140, 277 Wu CSRBF, 89, 88-90, 345
translation invariant, 29, 106
transport equation, 387, 403, 405 zeros theorem, 128
tree codes, 325

Meshfree Approximation Methods With MATLAB

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Meshfree Approximation Methods With MATLAB

Caricato da

Copyright:

Formati disponibili

.

Series Editor: Jinqiao Duan (Illinois Inst, of Tech., USA)

V o l . 4: Mathematical Theory of Adaptive Control

Vol. 6: Meshfree A p p r o x i m a t i o n Methods w i t h MATLAB

Illinois Institute of Technology, U S A - ^

British Library Cataloguing-in-Publication Data

Copyright 2007 by World Scientific Publishing Co. Pte. Ltd.

Printed by Mainland Press Pte Ltd

T r a d i t i o n a l n u m e r i c a l m e t h o d s , s u c h as finite e l e m e n t , finite difference, o r finite v o l -

3. Positive Definite Functions 27

3.1 Positive Definite Matrices and Functions 27

4. Examples of Strictly Positive Definite Radial Functions 37

5. Completely Monotone and M u l t i p l y Monotone Functions 47

5.1 Completely Monotone Functions 47

6.1 Interpolation w i t h Multivariate Polynomials 53

7. Conditionally Positive Definite Functions 63

7.1 Conditionally Positive Definite Functions Defined 63

8. Examples of Conditionally Positive Definite Functions 67

9. Conditionally Positive Definite Radial Functions 73

9.1 Conditionally Positive Definite R a d i a l Functions and Completely

10. Miscellaneous T h e o r y : Other N o r m s a n d Scattered D a t a F i t t i n g

11.3 Wu's Compactly Supported Functions 88

12. Interpolation w i t h Compactly Supported RBFs in M A T L A B 95

13. R e p r o d u c i n g K e r n e l H i l b e r t Spaces a n d N a t i v e Spaces for

13.1 R e p r o d u c i n g K e r n e l H i l b e r t Spaces 103

14. T h e P o w e r F u n c t i o n a n d N a t i v e Space E r r o r E s t i m a t e s 111

14.1 F i l l Distance and A p p r o x i m a t i o n Orders I l l

15. Refined and I m p r o v e d E r r o r B o u n d s 125

15.1 N a t i v e Space E r r o r B o u n d s f o r Specific B a s i s F u n c t i o n s 125

16. Stability and Trade-Off Principles 135

16.1 S t a b i l i t y and C o n d i t i o n i n g o f R a d i a l Basis F u n c t i o n I n t e r p o l a n t s . 135

17. N u m e r i c a l E v i d e n c e for A p p r o x i m a t i o n O r d e r R e s u l t s 141

17.1 I n t e r p o l a t i o n for e > 0 141

18. The O p t i m a l i t y of R B F Interpolation 159

18.1 T h e Connection to O p t i m a l Recovery 159

19. L e a s t Squares R B F A p p r o x i m a t i o n w i t h M A T L A B 165

19.1 O p t i m a l Recovery Revisited 165

20. T h e o r y for L e a s t Squares A p p r o x i m a t i o n 177

20.1 Well-Posedness o f R B F L e a s t Squares A p p r o x i m a t i o n 177

21. A d a p t i v e L e a s t Squares A p p r o x i m a t i o n 181

21.1 A d a p t i v e L e a s t Squares u s i n g K n o t I n s e r t i o n 181

22. M o v i n g L e a s t Squares A p p r o x i m a t i o n 191

22.1 Discrete W e i g h t e d Least Squares A p p r o x i m a t i o n 191

23. Examples of M L S Generating Functions 205

23.1 Shepard's M e t h o d 205

24. MLS Approximation with M A T L A B 211

24.1 A p p r o x i m a t i o n w i t h Shepard's M e t h o d 211

25. E r r o r B o u n d s for M o v i n g L e a s t Squares A p p r o x i m a t i o n 225

25.1 A p p r o x i m a t i o n O r d e r o f M o v i n g Least Squares 225

26. A p p r o x i m a t e M o v i n g Least Squares A p p r o x i m a t i o n 229

26.1 High-order Shepard M e t h o d s via M o m e n t Conditions 229

27. N u m e r i c a l E x p e r i m e n t s for A p p r o x i m a t e M L S A p p r o x i m a t i o n 237

27.1 Univariate Experiments 237

28. Fast F o u r i e r T r a n s f o r m s 243

28.1 NFFT 243

29. Partition of Unity Methods 249

29.1 Theory 249

30. Approximation of Point Cloud D a t a i n 3D 255

30.1 A G e n e r a l A p p r o a c h v i a I m p l i c i t Surfaces 255

31. Fixed Level Residual Iteration 265

31.1 Iterative Refinement 265

32. Multilevel Iteration 277

32.1 Stationary Multilevel Interpolation 277

33. Adaptive Iteration 291

11 Xi\\2 111 352112 1*1 - X \\