Lecture 2

Objectives
• By the end of this lecture students will:
– understand the simple linear regression model
– understand the logic behind the method of ordinary
least squares (OLS) estimation
• CLRM topic is a review of the material covered in
316-206 Quantitative Methods 2
• References:
– Gujarati., chapter 2
• also pp 100-105
 The Model
– True Model (Population Regression Function)
Yi    X i  ui
– Yi is the ith observation of the dependent variable
– Xi is the ith observation of the independent or
explanatory variable
– ui is the ith observation of the error or disturbance term
  and  are unknown parameters (coefficients)
– i = 1,..., n observations
 The Model
• Estimated model (Sample Regression
Function)
Yi  ˆ  ˆX i  uˆi
• where
–
ˆ and ˆ are the estimators of the coefficients
(parameters)
– are the i estimated residuals
ûi
• Objective: estimate the PRF on basis of SRF
 The Model
• In terms of the SRF the observed Yi can be
written as
Yi  Yˆi  uˆi
• in terms of the PRF Yi can be written as
Yi  E (Yi | X i )  ui
• Given the SRF is an approximation to the PRF
can we make this approximation as close as
possible?
 Y
SRF : Yˆi  ˆ  ˆX i
Yi
Yi
ui ûi PRF : E (Y | X i )    X i
Yˆi
Yˆi
E(Y|Xi) E(Y|Xi)

X
Xi
 Ordinary Least Squares (OLS)
• OLS obtains the estimators ̂ and ˆ
minimising the sum of squared residuals
(RSS) with respect to  and  :
N N
RSS   ui   (Yi    X i )
ˆ 2
ˆ ˆ 2

i 1 i 1
• Partially differentiate RSS w.r.t. the
coefficients
 Ordinary Least Squares (OLS)
• The first order conditions are:

RSS N
• (1)  2 (Yi  ˆ  ˆX i )  0
ˆ i 1

RSS N
• (2)  2 (Yi  ˆ  ˆX i )( X i )  0
ˆ i 1
 Ordinary Least Squares (OLS)
• Solving (1) and (2) simultaneously gives
N

(X i  X )(Yi  Y )
̂  i 1
N

 i
( X
i 1
 X ) 2

• and ˆ  Y  ˆX
• where Y and X are the sample means
 Ordinary Least Squares (OLS)
• Example: Melbourne house prices
• Y price of house in A$’000
• X distance of house from GPO in
kilometres
• A=area of house in square metres
• Interpret the results
The EVIEWS regression results are:
=============================================================
LS // Dependent Variable is Y
Date: 02/06/98 Time: 13:45
Sample: 1 27
Included observations: 27
=============================================================
Variable Coefficient Std. Error t-Statistic Prob.
=============================================================
C 272.8357 15.96022 17.09473 0.0000
X -5.303697 1.138680 -4.657759 0.0001
=============================================================
R-squared 0.464608 Mean dependent var 213.1593
Adjusted R-squared 0.443192 S.D. dependent var 66.27232
S.E. of regression 49.45213 Akaike info criterion 7.873197
Sum squared resid 61137.82 Schwarz criterion 7.969185
Log likelihood -142.5995 F-statistic 21.69471
Durbin-Watson stat 0.863751 Prob(F-statistic) 0.000090
=============================================================
H O U S IN G P R IC E S ($ 0 0 0 )
500

400

300

200

100
0 10 20 30 40

DISTANCE FROM GPO

 Ordinary Least Squares (OLS)
• What type of data is this?
• Calculate the missing values of Y-hat and
u-hat in the table
Yˆ  ˆ  ˆX
i i

Yˆi  272.8357  5.303697 X i

Yˆi  272.8357  5.303697(1.4)
Yˆ  265.4105
i
 Ordinary Least Squares (OLS)
Yi  Yˆi  uˆi
uˆ  Y  Yˆ
i i i

uˆi  210.2  265.4

uˆi  55.2
Y2  300.4; Yˆ2  260.6; uˆ2  39.8
Y  340.6; Yˆ  260.6; uˆ  80.0
3 3 3
• CHECK THE ANSWERS YOURSELVES!
 Summary and conclusions
• PRF vs SRF
• OLS methodology
• Next lecture
– Gujarati p65 – 76 (Classical Assumptions)
– Gujarati p107 – 112 (Normality Assumption)