Sei sulla pagina 1di 20

# Models Lineals

2-The L2 models

Pere Puig
Department de Matemtiques
Universitat Autnoma de Barcelona
ppuig@mat.uab.cat

Models Lineals

## The size of the Earth

On 19 March 1791 the meter
was theoretically defined by
the French Academy of Science
as being equal to the ten
millionth part of one quarter
of the terrestrial meridian.
However, the length of the
meridian still had to be set up.
The persons in charge of this
mission were Pierre-Franois
Mechain and Jean-Baptiste
Delambre. The length of the
meridian would be determined
by triangulation between
Dunkirk and Barcelona.

Models Lineals

Models Lineals

## Measurements of the French meridian arc, made in 1795

between Barcelona and Dunkirk:

## Using this data set, Legendre in 1805 solved an

overdetermined system of equations by the method of least
squares.

Models Lineals

The L2 solutions
A general problem: Consider an overdetermined system of
equations
, where A is a nxm matrix, n>m, and
The L2 solution of the system is a vector
which minimizes
the quadratic difference between b and Ax:
where ai is the i-th row of the matrix A.
Theorem: Any solution of the mxm system of equations
(normal equations)
solves the L2 minimization problem.

Models Lineals

## Proof: b is a vector in Rn and the columns of A define a vector

space C where Ax is an arbitrary vector in C. The minimum
distance b Ax is attained when b-Ax is orthogonal to Ax.

b Ax

C
Ax

Models Lineals

## Example 1: fitting a simple line (simple linear regression).

Suppose there are n data points {yi, xi}, where i = 1, 2, , n.
The goal is to find the equation of the straight line
which would provide a "best" (L2) fit for the data points.
Here,

Models Lineals

## They are usually expressed as,

where Sxy is the sample covariance of x and y, and S2x is the
sample variance of x.

Models Lineals

## Example 2: fitting a simple line without intercept.

Now there are n data points {yi, xi}, where i = 1, 2, , n, and
the goal is to find the equation of the line passing through the
origin
which would provide a "best" (L2) fit.

Models Lineals

## Example 3: the two variables predicting model.

Now there are n data points {zi, yi, xi}, where i = 1, 2, , n, and
the goal is to find the equation of the plane
which would provide a "best" (L2) fit for the data points.
Here,

Models Lineals

Models Lineals

## Note that the L2 solution has been expressed as,

but is

always invertible?

Models Lineals

is invertible

## Proof: Remember that a square matrix B is invertible

Remember also that rank(A) + dim(Ker(A)) = m, and therefore
rank(A) =m is equivalent to say that
The proof will be finished if we prove that
-

Models Lineals

## Property: Consider an overdetermined system of equations

, where A is a nxm matrix, n>m, rank(A)=m, with a first column of
1s. Then, the L2 solution
, satisfies
where indicates the mean of the coefficients of the i-th column
of the matrix A, and is the mean of the components of vector b.
Proof: (Exercise)

Models Lineals

## An application: the best local polynomial fit

Consider 2k+1 consecutive values of a time series, yt-k,...,yt,..., yt+k.
A local polynomial estimator of order p<2k+1 is the minimizer of
the least squares function,

## Then, choosing u=0 we obtain a predictor of the central

observation yt . The local polynomial approach consists in
replacing yt by y*t=0

Models Lineals

## The normal equations of this multiple regression

model are determined by,

## However we are only interested in the value of 0

because we want to evaluate the polynomial at
u=0.

Models Lineals

Notice that,

17

Models Lineals

## Example: Finding the local polynomial smoothing with p=2

and k=2 (5 consecutive points).
1
1
1
1
1

-2
-1
0
1
2

4
1
0
1
4

1
-2
4

1
-1
1

1
0
0

1
1
1

Matrix At
Matrix A
5
0
10

0
10
0

10
0
34

0,48571
0
-0,14286

At*A

0 -0,14286
0,1
0
0 0,071429
inv(At*A)

## Les multiplicacions de matrius es fan amb MMULT

L'inversi d'una matriu es fa amb MINVERSA
-0,08571 0,3428571 0,48571 0,34286 -0,08571
-0,2
-0,1
0
0,1
0,2
0,142857 -0,071429 -0,14286 -0,0714 0,142857
inv(At*A)*At

18

1
2
4

Models Lineals

## This linear filter is,

yt*=(-3 yt-2 + 12 yt-1 + 17 yt + 12 yt+1 -3 yt+2 )/35
For instance, it can be applied to the following time
series,

December 1979.

19

Models Lineals

## It can also be compared with the centered (simple) 5 points

moving average CMA(5).
1200
1000
800
600
400
200

en
e74
ju
l-7
4
en
e75
ju
l-7
5
en
e76
ju
l-7
6
en
e77
ju
l-7
7
en
e78
ju
l-7
8
en
e79
ju
l-7
9

Data

CMA(5)

20

LP2(5)