ch1 - Simple Regression - S

CHAPTER 1
SIMPLE REGRESSION
(Lec2-4)
Nguyen Thu Hang- BMNV, FTU CS2 1

Outline
1. Regression analysis
2. Population Regression Function (PRF)
3. The meaning of term linear
4. Sample regression function (SRF)
5. The Method of Ordinary Least Squares (OLS)
6. The assumptions underlying the method of least squares
7. Precision or standard errors of least-square estimates
8. Properties of least squares estimators
9. A measure of “Goodness of fit”
10. Confidence intervals for regression coefficients and variance
11. Hypothesis testing: t-test, Chi square test and F-test
12. The problem of prediction
13. Reporting the results of regression analysis
14. Some key points when reading the results
• Galton’s law of universal regression: how
the average height of sons changes, given
the fathers’ height
• The average height of children born of
parents of given height tended to move or
regress toward the average height in the
population as a whole.


1. Regression analysis?
Is the study of the dependence of one variable,
the dependent variable, on one or more other
variables, the explanatory variables.
• With a view to estimating and/or predicting the

(population) mean or average value of the
dependent variable on the basis of the known
or fixed values of the explanatory variable(s).

A hypothetical example
•
• Table 2.1: a total population of 60 families
and their weekly income (X) and weekly
consumption expenditure (Y). The 60 families
are divided into 10 income groups.

Some findings
• There is considerable variation in weekly
consumption expenditure in each income group.
• On the average, weekly consumption
expenditure increases as income increases. 
The dependence of consumption expenditure
on income.

2. The population regression line (PRL)

• E(Y/Xi)= f(Xi)  conditional expectation function (CEF) or
population regression function (PRF)  How the mean or
average response of Y varies with X.
• The functional form of the PRF is an empirical question. For
example, assume : E(Y | Xi) = β1 + β2Xi.
Our interest is in estimating the unknown beta 1, beta 2 on the

basis of observations on Y and X.
• One independent variable = simple regression

• More than one independent variable = multiple regression


Stochastic specification of PRF
• The deviation of an individual Yi around its
expected value: ui  Yi  E (Y | X i )
Yi  E (Y | X i )  ui  1   2 X i  ui
• E(Y|Xi) = the mean consumption of all

families with the same level of income  the
systematic, or deterministic component
• ui = stochastic disturbance or stochastic error
term, the random, or nonsystematic
component.
The significance of the stochastic error term
The error term is a representative for all those
variables that are omitted from the model but
that collectively affect Y.
• Why don’t we introduce them into the model
explicitly?
 Vagueness of theory
 Unavailability of data
 Core variables versus peripheral variables
 Principle of parsimony
 Poor proxy variables
3. The meaning of term linear
 Linearity in the Variables
• The conditional expectation of Y is a linear function of Xi,
the regression curve in this case is a straight line. But
E(Y | Xi) = β1 + β2X2i is not a linear function
 Linearity in the Parameters
• The conditional expectation of Y, E(Y | Xi), is a linear
function of the parameters, the β’s; it may or may not be
linear in the variable X.
E(Y | Xi) = β1 + β2X2i is a linear (in the parameter)
regression model.
The term “linear” regression will always mean a
regression that is linear in the parameters.
Examples of linear regression models
1
Yi  1   2 ( )  ui
Xi
ln Yi     ln X i  ui
Yi  1   2 X i   3 X  ui i
2

4. The Sample Regression Function (SRF)
A sample of Y values corresponding to some fixed
X’s. Can we estimate the PRF from the sample data?
• We may not be able to estimate the PRF

“accurately” because of sampling fluctuations.
• Suppose we draw another random sample from
the population of Table 2.1, as presented in Table
2.4 and 2.5, we obtain the scatter-gram given in
Figure 2.4.



• Population regression function
E (Y / X i )  f ( X i )  1   2 X i
• Sample regression function Yî  ˆ1  ˆ2 X i
• Yî = estimator of E(Y | Xi)

• βˆ1 = estimator of β1
• βˆ2 = estimator of β2
• An estimator, also known as a (sample) statistic, is simply a

rule or formula or method that tells how to estimate the
population parameter from the information provided by
the sample.
• A particular numerical value obtained by the estimator in
an application is known as an estimate.
• The stochastic form of SRF:
Yi  ˆ1  ˆ2 X i uˆ i
ûi = the (sample) residual term.

• Conceptually ûi is analogous to u and can be i
regarded as an estimate of ui. It is introduced

in the SRF for the same reasons as ui was
introduced in the PRF.

• Our primary objective in regression analysis is
to estimate the PRF
Yi  E(Y | X i )  ui  1   2 X i  ui
 ˆ  ˆ X uˆ

• On the basis of the SRF i
Y 1 2 i i
How should the SRF be constructed so that βˆ1

is as “close” as possible to the true β1 and βˆ2
is as “close” as possible to the true β2 even
though we will never know the true β1 and β2?

The SRF is an approximation of the PRF

LEC 3
5. The method of Ordinary Least Squares (OLS)

• “The method of least squares is the
automobile of modern statistical analysis;
despite its limitations, occasional accidents,
and incidental pollution, it and its numerous
variations, extensions and related
conveyances carry the bulk of statistical
analysis, and are known and valued by all”.
Stephen M. Stigler

• Two-variable PRF: Yi  1   2 X i  ui
• The PRF is not directly observable. We estimate
it from the SRF: Yi  ˆ1  ˆ2 X i uˆ i
• Or uî  Yi  Yî  Yi  ˆ1  ˆ2 X i

• The residuals are the differences between the
actual and estimated Y values.

• Given n pairs of observations on Y and X, we
would like to determine the SRF in such a
manner that it is as close as possible to the
actual Y. To this end, we may adopt the
following criterion:
Choose the SRF in such a way that the sum of
the residuals uî  Yi  Yî or Q is as small as
possible.
n
Q( ˆ1 , ˆ2 )   (Yi  ˆ1  ˆ2 X i ) 2
i 1

5. The Method of Ordinary Least Squares (OLS)
uî 2  (Yi  ˆ1  ˆ2 X i )2 (3.1.2 )
 
n n 2
 i   Yi  ˆ1  ˆ2 X i  min

ˆ
u 2
i 1 i 1
Derivation of least squares estimates: Derivative partially
with respect to βˆ1 and βˆ2 we obtain:
n
 ( uî2 ) n n  
i 1
  2 ui  2 (Yi  1   2 X i )  0
1 i 1 i 1
n
 ( uî2 ) n n  
i 1
  2 ui X i  2 X i (Yi  1   2 X i )  0
 2 i 1 i 1
Setting these equations to zero, we obtain

5. The Method of Ordinary Least Squares
 i 1 2 Xi
Y  nˆ  ˆ (3.1.4)
 i i 1 i 2  i
X Y  ˆ
 X  ˆ
 X 2 (3.1.5)
 Solving the normal equations simultaneously, we obtain:
(3.1.6)
(3.1.7)
5. The Method of Ordinary Least Squares
Review:
X,Y independent: var(X + Y) = var(X) + var(Y)
X,Y dependent: var(X + Y) = var(X) + var(Y) + 2Cov(X,Y)
Covariance
Correlation (to normalize covariance)
n n
(X i  X )(Yi  Y ) Y X i i  n. X .Y
cov( X , Y )
ˆ2  i 1
n
 i 1
n

Nguyeni Thu Hang- BMNV, FTU
Var ( X )
( X  X ) 2
CS2 X i
2
 n.( X ) 2
29
i 1 i 1
̂ 2 
S xy ˆ1  Y  ˆ2 X
S xx
• Where
S xy   ( X i  X )(Yi  Y )   X iY inXY
S xx   ( X i  X )   X  nX
2
i
2 2
S yy   (Yi  Y ) 2  Yi 2  nY 2

Example 1
• The effect of working time on income. Data on income in
dollars and working time of ten workers, X= working time,
Y= income
Obs X Y X2 Y2 XY
1 10 11 100 121 110

2 7 10 49 100 70
3 10 12 100 144 120
4 5 6 25 36 30
5 8 10 64 100 80
6 8 7 64 49 56
7 6 9 36 81 54
8 7 10 49 100 70
9 9 11 81 121 99
10 10 10 100 100 100
Sum 80 96 668 952 789
Example 1
S xx  668  10  8  28 2
S xy  789  10  8  9,6  21
S yy  952  10  9,6 2  30,4
S xy 21
ˆ2    0,75
S xx 28
ˆ1  Y  ˆX  9,6  0,75  8  3,6
• SRF: Yˆ  3,6  0,75 X
Example 1
• Eviews with Quick/Estimate Equation or Object /New
object/ Equation
• Stata: reg Y X
Variable Coefficient Std. Error t-Statistic Prob.
C 3.600000 2.090177 1.722342 0.1233

X 0.750000 0.255738 2.932692 0.0189
R-squared 0.518092 Mean dependent var 9.600000

Adjusted R-squared 0.457854 S.D. dependent var 1.837873
S.E. of regression 1.353237 Akaike info criterion 3.619732
Sum squared resid 14.65000 Schwarz criterion 3.680249
Log likelihood -16.09866 F-statistic 8.600683
Durbin-Watson stat 2.346416 Prob(F-statistic) 0.018920

Example 2
• The effect of rice price on rice demand. Y= demand (ton
per month), X= price (thousand dong per kg)
Stt Xi Yi XiYi X^2

1 1 10 10 1
2 4 6 24 16
3 2 9 18 4
4 5 5 25 25
5 5 4 20 25
6 7 2 14 49
sum 24 36 111 120
• SRF
Yî  11.5 1.375 X i
Interpretation of coefficient estimates
• Relation between working time and income:
Yˆ  3.6  0.75 X
 If working time increases by 1 hour, the estimated increase in
wages is about 75 cents.
• Relation between rice price and demand price:
Yî  11.5 1.375 X i

 If the rice price increases by 1, the estimated decrease in rice
demand is about 1.375
•

6. The assumptions underlying the OLS
Assumptions 1: Linear in Parameters. The
regression model is linear in the parameters.
Yi  1   2 X i  ui
• Keep in mind that the regressand Y and the
regressor X themselves may be nonlinear.

Assumption 2: X values are fixed in repeated
samplings. X is assumed to be non-stochastic.
• Keeping the value of income X fixed, say, at $80,

we draw at random a family and observe its
weekly family consumption expenditure Y as, say,
$60. Still keeping X at $80, we draw at random
another family and observe its Y value as $75. In
each of these drawings (i.e., repeated sampling),
the value of X is fixed at $80.
• Our regression analysis is conditional regression
analysis, that is, conditional on the given values
of the regressor(s) X.
Assumptions 3: Zero mean value of disturbance
E(ui / X i )  0
• Each Y population corresponding to a given X is
distributed around its mean value with some Y
values above the mean and some below it. The
mean value of these deviations corresponding to
any given X should be zero.
• Note that the assumption E(ui | Xi) = 0 implies
that E(Yi | Xi) = β1 + β2Xi.

Assumption 3
E(ui | Xi) = 0

LEC 4
Assumption 4: Homoscedasticity or equal

variance of disturbance
Var (ui | X i )   2
• The variation around the regression line

(which is the line of average relationship
between Y and X) is the same across the X
values; it neither increases or decreases as X
varies
Assumption 5: No autocorrelation between the
disturbance: Cov(ui .u j )  0 , i≠j.
• The disturbances ui and uj are uncorrelated, i.e., no

serial correlation. This means that, given Xi , the
deviations of any two Y values from their mean value
do not exhibit patterns.
Ex: (Yt = β1 + β2Xt + ut) that ut and ut−1 are positively
correlated. Then Yt depends not only on Xt but also on
ut−1 for ut−1 to some extent determines ut.
• We will see how intercorrelations among the
disturbances can brought into the analysis and with
what consequences.

Assumption 6: Zero covariance between ui and Xi
Cov(ui X i )  0
• The disturbance u and explanatory variable X are

uncorrelated. The PRF assumes that X and u (which
may represent the influence of all the omitted variables)
have separate (and additive) influence on Y. But if X and
u are correlated, it is not possible to assess their
individual effects on Y.
•

• Assumption 7: The number of observations n must be
greater than the number of parameters to be estimated.
• Assumption 8: Variability in X values. The X values in a
given sample must not all be the same.

Assumption 9: The regression model is correctly
specified. There is no specification bias or error in the
model used in empirical analysis.
• Some important questions that arise in the

specification of the model include the following:
(1) What variables should be included in the model?

(2) What is the functional form of the model? Is it
linear in the parameters, the variables, or both?
(3) What are the probabilistic assumptions made
about the Yi , the Xi, and the ui entering the model?
An example of assumption 9
• Two models depict the underlying
relationship between the rate of
change of money wages and the
unemployment rate:
Yi = α1 + α2Xi + ui
Yi = β1 + β2 (1/Xi ) + ui
• If the second model is the “correct” or

the “true” model, fitting the first one to
the scatterpoints shown in Figure 3.7
will give us wrong predictions.
Assumption 10: There is no perfect

multicollinearity.
• That is, there are no perfect linear

relationships among the independent
variables.
•  We will discuss in the multiple regression
models.

7. Precision or standard errors of Least-Squares
estimates
• The least-squares estimates are a function of the sample data. But since
the data change from sample to sample, the estimates will change.
Therefore, what is needed is some measure of “reliability” or precision of
the estimators βˆ1 and βˆ2. In statistics the precision of an estimate is
measured by its standard error (se), which can be obtained as follows:
See Appendix 3A.2,

p100-102

7. Precision or standard errors of Least-Squares
estimates
σ2 is the constant or homoscedastic variance of ui
See Ap 3.A5, 102
Where:
• ˆσ2 is the OLS estimator of the true but unknown
σ2 and where the expression n−2 is known as the
number of degrees of freedom (df).
• is the residual sum of squares (RSS).
See 3.5.2, 83

Properties of OLS Estimators under
the Normality Assumption
• With the assumption that ui follow the normal
distribution, the OLS estimators have the following
properties:
• Yi , is itself normally distributed with

• Mean: E (Yi )  1   2 X i
Var (Yi )   2

Example
• Wage and education (WAGE1.dta)
. reg wage educ
Source SS df MS Number of obs = 526

F( 1, 524) = 103.36
Model 1179.73204 1 1179.73204 Prob > F = 0.0000
Residual 5980.68225 524 11.4135158 R-squared = 0.1648
Adj R-squared = 0.1632
Total 7160.41429 525 13.6388844 Root MSE = 3.3784
wage Coef. Std. Err. t P>|t| [95% Conf. Interval]
educ .5413593 .053248 10.17 0.000 .4367534 .6459651

_cons -.9048516 .6849678 -1.32 0.187 -2.250472 .4407687

LEC 5
8. Properties of OLS statistics

• The sum and the sample average of the OLS
n
residuals is zero.  uˆ
i 1
i 0
• The sample covariance between the regressors

n
and the OLS residuals is zero.  x uˆ
i 1
i i 0
• The point ( x , y ) is always on the OLS

regression line.
(See Appendix 3A.1, p100)

8. Properties of least-squares estimators
• Gauss-Markov Theorem: Given the

assumptions of the classical linear regression
model, the least-squares estimators in the class
of unbiased linear estimators, have minimum
variance, that is, they are BLUE. (BLUE- Best
Linear Unbiased Estimator).

The goodness of fit: how “well” the sample
regression line fits the data.
• The coefficient of determination r2 (two-

variable case) or R2 (multiple regression) is a
summary measure that tells how well the
sample regression line fits the data.


• To compute this r2, we proceed as follows: Recall that
Yi  Yî uˆ i
or in the deviation form yi  yˆ i uˆ i
Squaring on both sides and summing over the sample, we obtain
• Since = 0 and yî = βˆ2xi . (p83)

• Total sum of squares (TSS): Total variation of
the actual Y values about their sample mean.
• Explained sum of squares (ESS): The variation

of the estimated Y values about their mean.
• Residual sum of squares (RSS): The residual of

the Y values about the regression line
• TSS = ESS +RSS

The coefficient of
determination


• The coefficient of determination r2 is a measure of the
goodness of fit of a regression line.
r2 measures the proportion or percentage of the total

variation in Y explained by the regression model.
 r2 is a nonnegative quantity.
 0 ≤ r2 ≤ 1
• An r2 of 1 means a perfect fit, that is, Yî = Yi for each i.

On the other hand, an r2 of zero means that there is no
relationship between the regressand and the regressor
whatsoever (i.e., βˆ2 = 0).

Example
• CEO salary and ROE (CEOSAL1.DTA)
. reg salary roe

F( 1, 207) = 2.77
Model 5166419.04 1 5166419.04 Prob > F = 0.0978
Residual 386566563 207 1867471.32 R-squared = 0.0132
Total 391732982 208 1883331.64 Root MSE = 1366.6
salary Coef. Std. Err. t P>|t| [95% Conf. Interval]
roe 18.50119 11.12325 1.66 0.098 -3.428196 40.43057

_cons 963.1913 213.2403 4.52 0.000 542.7902 1383.592

LEC 6
10. Confidence intervals for regression coefficients
ˆ1  1 and ˆ2   2 follow the t distribution with (n-2) df

se( ˆ1 ) se( ˆ2 )
 
Pr ˆ1  t / 2 se(ˆ1 )  1  ˆ1  t / 2 se(ˆ1 )  1  

Pr ˆ2  t / 2 se(ˆ2 )   2  ˆ2  t / 2 se(ˆ2 )  1   
100(1-α)% confidence intervals for the coefficients:
ˆ1  t / 2 se(ˆ1 ) ˆ2  t / 2 se(ˆ2 )

Example
• Consumption expenditure and income:
consump2.dta
. reg cons income

F( 1, 8) = 202.87
Model 8552.72727 1 8552.72727 Prob > F = 0.0000
Total 8890 9 987.777778 Root MSE = 6.493
cons Coef. Std. Err. t P>|t| [95% Conf. Interval]
income .5090909 .0357428 14.24 0.000 .4266678 .591514

_cons 24.45455 6.413817 3.81 0.005 9.664256 39.24483

Example
• df=8, t / 2  t 0.025  2.306
• The 95 confidence interval for beta2:

0.5091  2.306 * 0.0357  2  0.5091  2.306 * 0.0357
0.4268  2  0.5914
•  You can observe in the Stata output.

10. Confidence intervals for variance
We have : ˆ 2 (n  2)
~  2
n2
2
 100(1-α)% confidence intervals for the variance
(n  2)ˆ 2
(n  2)ˆ 2
  2
2
  / 2 ,( n  2 )
2
1 / 2,( n2)

10. Confidence intervals for variance
(Note the skewed characteristic of the chi-square distribution.)

Example
• df=8, ˆ  42.1591
• The 95% confidence intervals for the

variance:
8 * 42.1591 8 * 42.1591
 
2
17.5246 2.1797
19.234   2  154.7336

LEC 7
11. Hypothesis testing

• Hypotheses:
- Two-tail test: H 0 :    0 , H1 :    0
- Right- tail test: H 0 :    0 , H1 :    0
- Left-tail test: H 0 :    0 , H1 :    0

11. Hypothesis testing-t test
• The t Test: The decision to accept or reject
H0 is made on the basis of the value of the
test statistic obtained from the data at hand.
ˆ  
t 
• Recall SE ( ˆ ) follows the t
distribution (n-2) df.

11. Hypothesis testing- t test
• Compute t-statistics : ˆ   0
t0 
se ( ˆ )
• The decision to accept or reject H0

- Two-tail test: |t0|>t(n-2),α/2  reject H0
- Right-tail test : t0 > t(n-2),α  reject H0.
- Left-tail test: t0 <- t(n-2),α  reject H0

Example
consump2.dta
. reg cons income

F( 1, 8) = 202.87
Model 8552.72727 1 8552.72727 Prob > F = 0.0000
Total 8890 9 987.777778 Root MSE = 6.493
income .5090909 .0357428 14.24 0.000 .4266678 .591514

_cons 24.45455 6.413817 3.81 0.005 9.664256 39.24483

Example
• Two tail test H 0 :  2  0 H1 : 2  0
t= 14.24 t / 2  t 0.025  2.306
Reject H0
• Two tail test H 0 :  2  0.3 H1 : 2  0.3
 t=5.857 t / 2  t 0.025  2.306

 Reject H0

Example

Decision rule
• Construct a 100(1   )% confidence interval for
beta. If the beta under H0 falls within this
confidence interval, do not reject H0, but if it
falls outside this interval, reject H0.

Practical aspects- meaning of “accepting” or
“rejecting”
• If on the basis of a test of significance, we
decide to “accept” the null hypothesis, all we
are saying is that on the basis of the sample
evidence we have no reason to reject it, we
are not saying that the null hypothesis is true
beyond and doubt.

Practical aspects: The “Zero” Null Hypothesis
and the “2-t” Rule of Thumb
• A null hypothesis that is commonly tested in empirical
work is H0: β2 = 0, that is, the slope coefficient is zero.
• This null hypothesis can be easily tested by the
confidence interval or the t-test approach discussed in
the preceding sections. But very often such formal
testing can be shortcut by adopting the “2-t” rule of
significance, which may be stated as

Example- One tail test
• Suppose economic theory suggests that the
marginal propensity to consume is greater
than 0.3.
H 0 :  2  0.3 H1 :  2  0.3
• We have t 0.05 (8df )  1.86 , t=5.857  Reject H0.

11. Hypothesis testing- Chi-square test
• The Chi-square test: Testing the significance of
variance H : 2   2, H : 2   2
0 0 1 0
- Two-tail test:
H 0 :  2   02 , H 1 :  2   02
- Right-tail test:
H 0 :  2   02 , H 1 :  2   02
- Left-tail test:
- Compute:
ˆ 2
(n  2)
0 
2
 02

11. Hypothesis Testing- p-value
p-Value Approach to Testing

• Convert Sample Statistic to Test Statistic ( Z or t statistic )
• Obtain the p-value from a table or computer
• Compare the p-value with 
o If p-value <  , reject H0

o If p-value   , do not reject H0

11. Hypothesis testing- F test
• Analysis of Variance – ANOVA
We have
TSS = ESS + RSS.

A study of these components of TSS is known as the
analysis of variance (ANOVA).

Example
• ANOVA - Consumption expenditure and income:
consump2.dta
. reg cons income

F( 1, 8) = 202.87
Model 8552.72727 1 8552.72727 Prob > F = 0.0000
Total 8890 9 987.777778 Root MSE = 6.493
income .5090909 .0357428 14.24 0.000 .4266678 .591514

_cons 24.45455 6.413817 3.81 0.005 9.664256 39.24483

11. Hypothesis testing-F test
• F test: testing the significance of the model
• Hypothesis: H0: R2=0, H1: R2>0
We have: ESS RSS
~  2
~  n2 2
2 2
1
ESS / 1 R 2 (n  2)
Thus F 
RSS /( n  2) 1 R2
follows the F distribution, (1, n-2) df.
or F  (n  2)
2
Step 1: Compute F  ESS / 1 R
1 R2
0
RSS /( n  2)
0
If F0  F (1, n  2)  Reject H0.

Example
consump2.dta
. reg cons income

F( 1, 8) = 202.87 The critical
Model 8552.72727 1 8552.72727 Prob > F = 0.0000 F value = 5.32
Total 8890 9 987.777778 Root MSE = 6.493
income .5090909 .0357428 14.24 0.000 .4266678 .591514

_cons 24.45455 6.413817 3.81 0.005 9.664256 39.24483

LEC 8
12. The problem of prediction

• We obtain: Yî  24.4545  0.5091X i
• What use can be made of this historical regression?
 One use is to predict or forecast the future
consumption expenditure Y corresponding a chosen X,
say X0.
There are two prediction:
• Mean Prediction
• Individual prediction

12. The Prediction- mean prediction
Assume that X=X0
• The point estimate of mean E(Y|X0) prediction as:
Yˆ0  ˆ1  ˆ2 X 0
• The confidence interval of mean E(Y|X0) prediction as:
where

Example-mean prediction
• X0=100
Yˆ0  ˆ1  ˆ2 X 0  24.4545  0.5091*100  75.3645
ˆ  1 (100  170) 2

var(Y0 )  42.159    10.4759
10 33000 
• The 95% confidence interval for true E(Y|X0) :
67.9010  E (Y | X  100)  82.8381

Example- Confidence intervals (bands)
for mean Y
160
140
120
100
80
60
100 150 200 250

income
95% CI Fitted values

12. Prediction: Individual Prediction
Assume that X=X0
• The point estimate of mean E(Y|X0) prediction as:
Yˆ0  ˆ1  ˆ2 X 0
 1 ( X  X ) 2 
• We have Var (Y0  Yˆ0 )   2 1   0 2 
 n
  xi 

• Substitute ˆ 2for unknown 2 Y0  Yˆ0
t ~ t (n  2)
ˆ
se(Y0  Y0 )
• The 95% confidence interval for individual Y.
ˆ1  ˆ2 X 0  t / 2 se(Y0  Yˆ0 )  1  2 X 0  ˆ1  ˆ2 X 0  t / 2 se(Y0  Yˆ0 )

Example-individual prediction
• X0=100
Yˆ0  ˆ1  ˆ2 X 0  24.4545  0.5091*100  75.3645
ˆ 1 ( X 0  X )2 (100  170) 2
se(Y0  Y0 )  ˆ (1  
2
)  42.159 (1  0.1  )  7.255
n  i x 2
33000
• The 95% confidence interval for true Y0|X0 :

58.645  Y0 | X 0  100)  92.0945

Example- Confidence intervals (bands)
for individual Y values
200
150
100
50
100 150 200 250

income
95% CI Fitted values

13. Reporting the results of regression
Variable Coefficient Std. Error t-Statistic Prob.
C 3.600000 2.090177 1.722342 0.1233

X 0.750000 0.255738 2.932692 0.0189
R-squared 0.518092 Mean dependent var 9.600000

Adjusted R-squared 0.457854 S.D. dependent var 1.837873
S.E. of regression 1.353237 Akaike info criterion 3.619732
Sum squared resid 14.65000 Schwarz criterion 3.680249
Log likelihood -16.09866 F-statistic 8.600683
Durbin-Watson stat 2.346416 Prob(F-statistic) 0.018920

13. Reporting the results of regression
• Yˆ  3.6 + 0.75X ; n=10

se = (2.090177) ( 0.255738) ; R2=0.518092
t = (1.722342) (2.932692) ; F0 = 8.600683
p-value = (0.1233) (0.0189) ; 0.018920

13. Reporting the results in a paper

• Test the model: look at F test  If p-value is
smaller than alpha, the model is significant.
• Test the goodness of fit of the model: look at R-
square  It measures the proportion or
percentage of the total variation in Y explained by
the regression model. If the model is significant
but R-square is small, it means that observed
values are widely spread around the regression
line.
• Test that the slope is significantly different from
zero: Look at t-value in the ‘Coefficients’ table
and find p-value.

How “good” is the fitted model? We need some criteria with
which to answer this question.
• First, are the signs of the estimated coefficients in
accordance with theoretical or prior expectations? e.g., the
income consumption model should be positive.
• Second, if theory says that the relationship should also be
statistically significant. The p value of the estimated t value
is extremely small.
• Third, how well does the regression model explain variation
in the consumption expenditure? One can use r2 to answer
this question, which is a very high.
Assignments
• Questions 2.1~2.8 in p52-53, Gurajati.
• Problems 2.1, 2.3, 2.4, 2.5, 2.6 in p.61-63,
Wooldridge.
• Computer Exercises C.2.2, 2.3, 2.4, 2.5, 2.6,
2.7 in p65-6, Wooldridge.
• Questions 5.2, 5.3 in p152, Gurajati.
• Problems 5.8, 5.9 in p153-154, Gurajati.

ch1 - Simple Regression - S

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

ch1 - Simple Regression - S

Caricato da

Copyright:

Formati disponibili

CHAPTER 1

Nguyen Thu Hang- BMNV, FTU CS2 1

Nguyen Thu Hang- BMNV, FTU CS2 3

Nguyen Thu Hang- BMNV, FTU CS2 4

• With a view to estimating and/or predicting the

Nguyen Thu Hang- BMNV, FTU CS2 5

Nguyen Thu Hang- BMNV, FTU CS2 6

Nguyen Thu Hang- BMNV, FTU CS2 8

Nguyen Thu Hang- BMNV, FTU CS2 9

Our interest is in estimating the unknown beta 1, beta 2 on the

• One independent variable = simple regression

Nguyen Thu Hang- BMNV, FTU CS2 10

Nguyen Thu Hang- BMNV, FTU CS2 11

• E(Y|Xi) = the mean consumption of all

Nguyen Thu Hang- BMNV, FTU CS2 15

• We may not be able to estimate the PRF

Nguyen Thu Hang- BMNV, FTU CS2 16

Nguyen Thu Hang- BMNV, FTU CS2 17

Nguyen Thu Hang- BMNV, FTU CS2 18

• Yˆi = estimator of E(Y | Xi)

• An estimator, also known as a (sample) statistic, is simply a

ˆui = the (sample) residual term.

regarded as an estimate of ui. It is introduced

Nguyen Thu Hang- BMNV, FTU CS2 20

How should the SRF be constructed so that βˆ1

Nguyen Thu Hang- BMNV, FTU CS2 21

Nguyen Thu Hang- BMNV, FTU CS2 22

5. The method of Ordinary Least Squares (OLS)

Nguyen Thu Hang- BMNV, FTU CS2 23

• Or uˆi  Yi  Yˆi  Yi  ˆ1  ˆ2 X i

Nguyen Thu Hang- BMNV, FTU CS2 24

Nguyen Thu Hang- BMNV, FTU CS2 26

uˆi 2  (Yi  ˆ1  ˆ2 X i )2 (3.1.2 )

 i   Yi  ˆ1  ˆ2 X i  min

Setting these equations to zero, we obtain

Correlation (to normalize covariance)

Nguyen Thu Hang- BMNV, FTU CS2 30

1 10 11 100 121 110

C 3.600000 2.090177 1.722342 0.1233

R-squared 0.518092 Mean dependent var 9.600000

Nguyen Thu Hang- BMNV, FTU CS2 33

Stt Xi Yi XiYi X^2

• Relation between rice price and demand price:

Yˆi  11.5 1.375 X i

Nguyen Thu Hang- BMNV, FTU CS2 35

Nguyen Thu Hang- BMNV, FTU CS2 36

• Keeping the value of income X fixed, say, at $80,

Nguyen Thu Hang- BMNV, FTU CS2 39

Nguyen Thu Hang- BMNV, FTU CS2 40

6. The assumptions underlying the OLS

Assumption 4: Homoscedasticity or equal

• The variation around the regression line

• The disturbances ui and uj are uncorrelated, i.e., no

Nguyen Thu Hang- BMNV, FTU CS2 44

• The disturbance u and explanatory variable X are

Nguyen Thu Hang- BMNV, FTU CS2 46

Nguyen Thu Hang- BMNV, FTU CS2 47

• Some important questions that arise in the

(1) What variables should be included in the model?

• If the second model is the “correct” or

Assumption 10: There is no perfect

• That is, there are no perfect linear

Nguyen Thu Hang- BMNV, FTU CS2 51