Review of OLS & Introduction to Pooled

Cross Sections
EMET 8002
Lecture 1
July 23, 2008
Outline
Review the key assumptions of OLS regression
Chapter 3 in the text
Motivation for course topics
Introduce regression analysis with time series data
Chapter 10 in the text
OLS assumptions
MLR.1: The model is linear in parameters
MLR.2: Random sampling
MLR.3: No perfect multicollinearity
None of the independent variables are constant and
there are no exact linear relationships among the
independent variables
MLR.4: Zero conditional mean
0 1 1
...
k k
y x x u = + + + +
( )
1
| ,..., 0
k
E u x x =
Unbiasedness

of OLS
Theorem 3.1: Under assumptions MLR.1 to MLR.4
For any values of the population parameter
j
In other words, the OLS estimators are unbiased
estimators of the population parameters
This is a property of the estimator, not any specific
estimate

i.e., we have no reason to believe that our estimate is either too big or too small
( )

, 0,1,...,
j j
E j k = =
Review: The Variance of OLS
MLR.5: Homoskedasticity
Theorem: Given MLR.1 through MLR.5, we can show that:
Where SST
j
is the total sample variation of x
j
:
And is the R-squared from regressing x
j
on all the other
regressors in the model.
var u x
1
, x
2
,..., x
k
( )
=
2
var

j
( )
=

2
SST
j
1 R
j
2
( )
SST
j
= x
ij
x
j
( )
2
i=1
n

R
j
2
Review: Standard Errors for OLS
Unless we know , we need an unbiased estimator in order
estimate standard errors.
We obtain:
Thus, the formula for the standard error is given by:

2
=

u
i
2
i=1
n

n k 1
=
SSR
n k 1
, E

2
X

=
2
se

j
( )
=

SST
j
1 R
j
2
( )

1
2
Review: Gauss-Markov Theorem
Theorem: Under assumptions MLR.1 through MLR.5, the OLS
estimators
are the best linear unbiased estimators (BLUE) of
i.e., For any linear estimator,
OLS has a lower variance, i.e.,

0
,

1
,...,

0
,
1
,...,
k

j
= w
ij
y
i
i=1
n

var

j
( )
var

j
( )
Review: Variance-Covariance Matrix

var

( )
=
var

0
( )
cov

0
,

1
( )
cov

0
,

k
( )
cov

1
,

0
( )
var

1
( )

cov

i
,

j
( )
var

i
( )

var

k
( )

For testing hypotheses involving more than one coefficient, we may need to
know covariances between coefficient estimators:
What is the sampling distribution?
Knowing the mean and variance of the OLS estimator is
insufficient information to permit hypothesis testing.
Under what conditions will the OLS estimator be normally
distributed?
Exact, small (finite) sample assumptions
Large sample (asymptotic) assumptions
Asymptotic Normality
Theorem: Under the Gauss-Markov Assumption (MLR.1
through MLR.5, also permitting MLR.4):
Where
And
Which allows us:
n

j

j
( )
~
a
N 0,
o
2
a
j
2

_
,

a
j
2
= plim n
1

r
ij
2
i=1
n

> 0
plim

2
=
2

j

j
se

j
( )
~
a
N 0,1 ( )
Motivation for course
During the lecture component of the course we are
going to study circumstances in which some of the
OLS assumptions fail
Caution: a little econometric technique in the wrong
hands can be a dangerous thing
See critique by Angus Deaton (2009) Instruments of
Development
The nature of time series data
Two key differences between time series and cross-
sectional data:
Temporal ordering
Interpretation of randomness
When we collect a time series data set, we obtain one
possible outcome, or realization, but if certain conditions
in the past had been different, we generally obtain a
different realization for the stochastic process
The set of all possible realizations of a time series
process plays the role of the population in a cross-
sectional analysis
Simple examples: A finite distributed
lag model
A finite distributed lag (FDL) model:
This is model is an FDL of order 2
How do we interpret the coefficients in the context of
a temporary, one-period increase in the value of z?

0
shows the immediate impact of a one-unit increase
in z at time t impact propensity or impact
multiplier
We can graph
j
as a function of j to obtain the lag
distribution
0 0 1 1 2 2 t t t t t
y z z z u

= + + + +
Finite distributed lag model
We are also interested in the change in y due to a
permanent one-unit increase in z
At the time of the increase y increases by
0
After one period, y has increased by
0
+
1
After two periods, y has increased by
0
+
1
+
2
There are no further changes in y after two periods
This shows that the long-run change in y given a
permanent increase in z is the sum of the coefficients
on the current and lagged values of z
This is known as the long-run propensity or long-
run multiplier
Unbiasedness of the OLS estimator
As in cross-section data, we require a set of
assumptions in order for the OLS estimator to be
unbiased for time series models
Assumption TS.1: The stochastic process {(x
t1
, x
t2
,
, x
tk
, y
t
): t=1,2,,n} follows the linear model
where {u
t
: t=1,2,,n} is the sequence of errors or
disturbances and n is the number of observations
(time periods)
0 1 1
...
t t k kt t
y x x u = + + + +
Unbiasedness of the OLS estimator
Assumption TS.1 is essentially the same as
assumption MLR.1 for cross-sectional models
Assumption TS.2: No perfect collinearity no
independent variable is constant nor a perfect linear
combination of the others
Assumption TS.2 is the same as the corresponding
assumption for cross-sectional data
Unbiasedness of the OLS estimator
A little further notation:
Let x
t
=(x
t1
, x
t2
, , x
tk
) denote the set of al
independent variables in the equation at time t
Let Xdenote the collection of all independent variables
for all time periods where the rows correspond the
observations for each time period
Assumption TS.3: (Zero conditional mean) For
each t, the expected value of the error u
t
, given the
explanatory variables for all time periods, is 0:
( )
| 0, 1, 2,...,
t
E u t n = = X
Unbiasedness of the OLS estimator
Assumption TS.3 implies that the error term, u
t
, is
uncorrelated with each explanatory variable in every
time period
If u
t
is independent of Xand E(u
t
)=0 then TS.3 is
automatically satisfied
Given our assumption for cross-sectional analysis
(MLR.4) it is not surprising that we require u
t
to be
uncorrelated with the explanatory variables in period
t. Stated in terms of conditional expectations:
E(u
t
|x
t
)=0
Unbiasedness of the OLS estimator
When E(u
t
|x
t
)=0 holds, we say that the x
tj
are
contemporaneously exogenous
However, TS.3 is stronger than simply requiring
contemporaneous exogeneity. The error term u
t
must
be uncorrelated with x
sj
even when st.
When TS.3 holds, we say that the explanatory
variables are strictly exogenous
Unbiasedness of the OLS estimator
For cross-sectional data we did not need to specify
how the error term for, say, person i, u
i
, is related to
the explanatory variables for other persons in the
dataset. Due to random sampling, u
i
is automatically
uncorrelated with the explanatory variables for other
persons
Importantly, TS.3 does not put any restrictions on
the correlation between explanatory variables or on
the correlation of u
t
across time
Unbiasedness of the OLS estimator
Assumption TS.3 can fail for many reasons, such as
measurement error or omitted variable bias, but it can also fail
due to less obvious reasons
For example, consider the simple static model
Assumption TS.3 requires that u
t
is uncorrelated with the
present value as well as past and future values of z
t
. This has
two implications:
z can have no lagged effect on y. If z does have a lagged
effect on y, then we should estimate a distributed lagged
model.
Strict exogeneity excludes the possibility that changes in the
error term today can cause future changes in z. This
effectively rules out feedback from y to future values of z
0 1 t t t
y z u = + +
Unbiasedness of the OLS estimator
As a concrete example, consider
Lets assume, for arguments sake, that u
t
is
uncorrelated with polpc
t
and with past values of
polpc
Suppose, though, that the city adjusts the size of its
police force based on past values of the murder rate
(i.e., a high murder rate in period t leads to an
increase in the size of the police force in period t+1).
Then, u
t
is correlated with polpc
t+1
, violating TS.3
0 1 t t t
mrdrte polpc u = + +
Unbiasedness of the OLS estimator
Explanatory variables that are strictly exogenous cannot react to
what has happened to y in the past
For example, the amount of rainfall in agricultural production
is not influenced by production in the previous year
However, something like labour input for agricultural
production may not be strictly exogenous since the farmer
may adjust the amount based on last years yields
There are lots of reasons to believe that TS.3 will be violated in
many social science contexts as current policies, such as
interest rates, are influenced by previous realizations. However,
it is the simplest way to demonstrate unbiasedness of the OLS
estimator.
Unbiasedness of the OLS estimator
Theorem 10.1: (Unbiasedness of OLS) Under
assumptions TS.1, TS.2 and TS.3, the OLS estimators
are unbiased conditional on X, and therefore
unconditionally as well:
( )

, 0,1,...,
j j
E j k = =
Variance of the OLS estimator
We have seen the conditions necessary for an
unbiased estimator. The next step is to learn how we
can say anything about how precise the estimator is.
Assumption TS.4: (homeskedasticity) Conditional
on X, the variance of u
t
is the same for all t:
( ) ( )
2
var | var , 1, 2,...,
t t
u u t n = = = X
Variance of the OLS estimator
TS.4 requires that the variance cannot depend on X.
It is sufficient that:
u
t
and Xare independent, and
var(u
t
) must be constant over time
When TS.4 does not hold, we say that the errors are
heteroskedastic, just as in the cross-section case
Variance of the OLS estimator
Example of heteroskedasticity:
Consider the following regression of the 3-month T-bill
rate (i3
t
) on the inflation rate (inf
t
) and the federal
deficit as a percentage of GDP (def
t
):
Since policy regime changes are known to affect the
variability of interest rates, it is unlikely that the error
terms are homoskedastic
Furthermore, the variability in interest rates may
depend on the level of inflation or the deficit, also
violating the homoskedasticity assumption
0 1 2
3
t t t t
i inf def u = + + +
Variance of the OLS estimator
Assumption TS.5: (NEW!! No serial correlation)
Conditional on X, the errors in two different time
periods are uncorrelated:
To interpret this condition, it is easier to ignore the
conditioning on X, in which case the condition is:
( )
, | 0,
t s
corr u u t s = X
( )
, 0,
t s
corr u u t s =
Variance of the OLS estimator
If this condition does not hold, then we say the
errors suffer from serial correlation or
autocorrelation
Example of serial correlation:
If u
t-1
>0 then u
t
is on average above 0, then the
correlation is also positive. This turns out to be a
reasonable characterization of error terms in many
times series applications, which we will deal with later
Variance of the OLS estimator
However, TS.5 says nothing about the temporal
correlation of the explanatory variables
For example, inf
t
is almost certainly positively
correlated over time, but this does not violate TS.5
We did not need a similar assumption to TS.5 for
cross-sectional analysis because under random
sampling the error terms are automatically
uncorrelated for two different observations
However, serial correlation will come up in the context
of panel data
Variance of the OLS estimator
Theorem 10.2 (OLS sampling variances) Under the
time series assumptions TS.1 through TS.5, the
variance of the OLS estimator conditional on Xis:
where SST
j
is the total sum of squares of x
tj
and R
j
2
is the R-squared from the regression of x
j
on the
other independent variables
( )
( )
2 2

var | 1 , 1,...,
j j j
SST R j k

= =

X
Variance of the OLS estimator
This is the same variance we derived for cross-
sectional analysis
Theorem 10.3 (unbiased estimation of
2
) Under
assumptions TS.1 through TS.5, the estimator
is an unbiased estimator of
2
where df=n-k-1
2

/ SSR df =
Variance of the OLS estimator
Theorem 10.4: (Gauss-Markov Theorem) Under
assumptions TS.1 through TS.5, the OLS estimators
are the best linear unbiased estimators conditional on
X.
Bottom line: OLS has the same desirable finite
sample properties under TS.1 through TS.5 that is
has under MLR.1 through MLR.5
Inference
In order to use the OLS standard errors, t statistics,
and F statistics, we need to add one more additional
assumption
Assumption TS.6: (normality) The errors u
t
are
independent of Xand are independently and
identically distributed as N(0,
2
)
TS.6 implies assumptions TS.3 through TS.5 but it is
stronger since it assumes independence and
normality
Inference
Theorem 10.5: (normal sampling distributions)
under assumptions TS.1 through TS.6 the OLS
estimators are normally distributed, conditional on X.
Further, under the null hypothesis, each t statistic
has a t distribution and each F statistic has an F
distribution. Confidence intervals are constructed in
the usual way.
Bottom line: Under these assumptions, we can
proceed as usual.
Functional form, dummy variables
and index numbers
Not surprisingly, we can use logarithmic functional
forms and dummy variables just as before (see
Section 10.4 in Wooldridge)
Trends
Many economic time series have a common tendency
of growing over time. If we ignore this underlying
trend we may improperly attribute the correlation
between the two series
One popular way to capture trending behaviour is a
linear time trend:
where et is i.i.d. with E(e
t
)=0 and var(e
t
)=
2
0 1
, 1, 2,...
t t
y t e t = + + =
Trends
A second popular method for capturing trends in
using an exponential trend which holds when a
series has the same average growth rate from period
to period:
Other forms of trends are used in empirical analysis
but linear and exponential trends are the most
common
( )
0 1
log , 1, 2,...
t t
y t e t = + + =
Trends
Nothing about trending variables immediately
contradicts our assumptions, TS.1 through TS.6.
However, it may be that unobserved trending factors
that affect y
t
may also affect some of the explanatory
variables. If we ignore this, we have run a spurious
regression!
Consider the model:
0 1 1 2 2 3 t t t t
y x x t u = + + + +
Trends
If the above model satisfies assumptions TS.1
through TS.3 (those required for the OLS estimator
to be unbiased), then omitting t from the regression
will generally lead to biased estimators of
1
and
2
Example 10.7 provides a simple example of the
impacts of ignoring trends
Trends
R
2
are often very high in times series regressions.
Does this mean they are more informative than
cross-sectional regressions?
Not necessarily
Time series data is often aggregate data such as
average wage levels, meaning individual heterogeneity
has been averaged over
Moreover, the R
2
can be artificially high when the
dependent variable is trending
Trends
Recall the formula for the adjusted R
2
:
where
However, when E(y
t
) follows, say, a linear trend,
then this is no longer an unbiased or consistent
estimator of var(y
t
). The simplest way around this is
to detrend the variables first and calculate the R
2
from a regression using the detrended variables
( )
2 2 2

1
u y
R =
( )
2
2
1

n
y t
t
y y
=
=

Seasonality
If a time series is observed at monthly or quarterly
intervals (or even weekly or daily) it may exhibit
seasonality
e.g., housing starts can be strongly influenced by
weather. If weather patterns are generally worse in,
say, January than June, then housing starts will
generally be lower in January
e.g., retail sales are generally higher in December than
in other months because of the Christmas holiday
Seasonality
One way to address this is to allow the expected
value to vary by month:
Many series that display seasonal patterns are often
seasonally adjusted before being reported for
public use. Essentially, they have had the seasonal
component removed.
0 1 1
1 2 11
...
...
t t k kt
t t t t
y x x
feb mar dec u


= + + + +
+ + + +