OLS Estimator

Modelos Lineares
Propriedades do MQO em Pequenas Amostras
Cristine Campos de Xavier Pinto
CEDEPLAR/UFMG
Maio/2010
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Linear Models
Y
i
= X
i
+
i
where X
i
= [1, X

i
] .
A1: (Y
i
, X
i
) are i.i.d sample.
A2:
_

N
i =1
X
0
i
X
i
_
has full rank (It is nonsigular)
A3: E[
i
j X
i
] = 0
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Linear Models
We can write the model in the matrix form
y = X+
where
y = [y
1
, y
2
, ..., y
N
]
0
, = [
1
,
2
, ..,
N
]
0
, X = [X
1
, X
2
, ..., X
N
]
0
X
NxK
=
_

_
1 x
11
x
1K1
1 x
21
x
2K1
.
.
.
.
.
.
.
.
.
.
.
.
1 x
N1
x
NK1
_

_
A1: (Y
i
, X
i
) are i.i.d sample.
A2: X has full rank
A3: E[ j X] = 0
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Under the A2,

=
_
X
0
X
_
1
X
0
Y
=
_
N

i =1
X
0
i
X
i
_
1
_
N

i =1
X
0
i
Y
i
_
Using the matrix notation, recall that
Y = X

+
= P
X
Y+ (I P
X
) Y
Notice that one property of OLS is that
0
X = 0
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Interpretations
1 Orthogonal projection onto subspace Col (X), and decompose
Y into two components: a linear combination of columns of X
and a component that is orthogonal to X.
2 Sample analog of the coecient of the regression population
function. When the CEF is linear, this coecients is the one
that minimizes the MSE.
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Expected Value
Using the population model, we have

=
_
X
0
X
_
1
X
0
Y
= +
_
X
0
X
_
1
X
0

Taking the conditional expectation of

,
E
_

X
_
= +
_
X
0
X
_
1
X
0
E[ j X]
. .
=0 by A3
At the end E
_

X
_
= .
Under A1-A3,

is an unbiased estimator.
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Variance
To calculate the "exact" variance of the OLS estimator, we
need to add the following assumption:
A4: E
_

X
_
=
2
I
N
Under A1-A4,
Var
_

X
_
= E
_
_

_0

X
_0
= E
_
_
X
0
X
_
1
X
0

0
X
_
X
0
X
_
1

X
_
=
2

_
X
0
X
_
1
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Variance
We can use the least squares residuals to get an estimator for

2
,

2
=
1
N

Using the geometric approach,

= (I P
X
) Y
= (I P
X
) X
. .
=0
+ (I P
X
)
= (I P
X
)
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Variance
Using the properties of orthogonal projectors,
E
_

2

X
_
=
1
N
E
_

0
(I P
X
)

X
_
=
1
N
E
_
tr (I P
X
)
0

X
_
=
1
N
tr E
_
(I P
X
)
0

X
_
=
1
N
tr (I P
X
)E
_

X
_
=
2
1
N
tr (I P
X
)
=
2
N K
N
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Variance
An unbiased estimator of
2

2
=
1
N K

And an unbiased estimator for Var

_

X
_
\
Var
_

X
_
=
2

_
X
0
X
_
1
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Best Linear Unbiased Estimator
Class of linear estimators of : estimators of the form A
0
y
for any NxK constant matrix.
Lets restrict the class of linear functions by imposing the
unbiasedness condition
E
_
A
0
y

X
_
= for all
The OLS estimator is the best member of this class (best of
all linear unbiased estimator).
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Best Linear Unbiased Estimator
Theorem
(Gauss-Markov): Let

= A
0
y for any NxK constant matrix such
that E
_
A
0
y

X
_
= . Let assumptions A1-A4 hold, then
Var
_
c
0

X
_
Var
_
c
0

X
_
for all c 2 R
K
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Best Linear Unbiased Estimator
Proof.
Lets use the tted vector X. Using the fact that both estimators
are unbiased, we have
X = E
_
X

X
_
= E
_
XA
0
y

X
_
= XE
_
A
0
y

X
_
= XA
0
X
so
X = XA
0
X
In this case,A
0
X = I , or the projector onto Col (X).
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
Best Linear Unbiased Estimator
Proof.
Under this restriction, we will show that Cov
_

_
= 0.
Cov
_

X
_
= Cov
_
A
0
y,
_
X
0
X
_
1
X
0
y

X
_

2
_
X
0
X
_
1
= A
0
Var [ yj X] X
_
X
0
X
_
1

2
_
X
0
X
_
1
=
2
A
0
X
_
X
0
X
_
1

2
_
X
0
X
_
1
= 0
which implies that
Var
_

X
_
= Var
_

X
_
Var
_

X
_
0
Cristine Campos de Xavier Pinto Institute
Modelos Lineares
OLS Estimator
References
Amemya: 1
Golderberg: 14 e 15
Hayashi: 1
Rudd: 6-9
Cristine Campos de Xavier Pinto Institute
Modelos Lineares