Curve Fitting

We are given a set of measurements (x i , yi ), i N where x i may be the times or positions of

measurements and yi ' s are the measured values corrupted with undesired but unavoidable noise. It
may be known that the relation between y and x is a parametric function such as y f ( x, ) where
is the set of parameters in vector form. We wish to estimate the values of the parameters, or in
other words, we wish to find the set of parameters which provide the best fit to the given
N
measurements. For example we may choose to minimize ( y f ( x ; ))
i 1
i i
2
. We may also

N
express the problem as * ArgMin

( y f ( x ; ))
i 1
i i
2
where * is the value of the parameters

where the function is minimized.

In some cases the problem may be reduced to the solution of a system of linear equations:

Suppose f ( x; ) is a polynomial in x such as f ( x; ) n xn n1 xn1 ... 1 x 0 .

Let ei be the fit error at xi : ei yi (n x n n1 x n1 ... 1 x 0 ) . Then,

e1 y1 1 x1 x1 ... x1 0
2 n

. . 2 n
1 x2 x2 ... x2 . , or e = y - X .
. . ... .

eN yN 1 xN xN ... xN n
2 n

Min y - X .
2 2
Hence the optimization problem is Min e

Foa given value of , X is a linear combination of the columns of X . Thus by choosing the correct
(optimal) value of we want to find the vector in the column space of X which is closest to y . For
this optimal value the error vector e must be orthogonal to the column space of X :

XT e 0 XT (y - X) 0 XT X XT y (called the normal equations).

This last equation is always consistent, and hence can be solved to find the optimal parameter set.

Example: Say the measurement values corresponding to x = 0, 1, 2, 3 are = 1, -1, 3, 2 respectively. If

we want to fit a second degree polynomial to the given set of data, then we have three parameters
to estimate, namely 0 , 1 , and 2 of the polynomial y 2 x 2 1 x 0 . Thus we have
1 0 1 0 0
1 1 1 1 1
y , x , and X .
3 2 1 2 4

2 3 1 3 9

The normal equations become

 T T
1 0 0 1 0 0 1 0 0 1
1 0 4 6 14 0 5
1 1 1
1
1 1 1 1 1
6 14 36 1 11
1
1 2 4 1 2 4 1 2 4 3

2 29
9 2 1
14 36 98
1 3 9 1 3 3 9 2

0 9

and the solution is 1
1
20 1 .
2 5