MULTICOLLINEARITY

Y 2 3X2 X3
X3 2X2 1

X2

X3

10

19

51

11

21

56

12

23

61

13

25

66

14

27

71

15

29

76

Suppose that Y = 2 + 3X2 + X3 and that X3 = 2X2 1. There is no disturbance term in the
equation for Y, but that is not important. Suppose that we have the six observations shown.
1

MULTICOLLINEARITY

80

Y
70
60
50
40
30

X3

20

X2

10
0
1

The three variables are plotted as line graphs above. Looking at the data, it is impossible to
tell whether the changes in Y are caused by changes in X2, by changes in X3, or jointly by
changes in both X2 and X3.
2

MULTICOLLINEARITY

Y 2 3X2 X3
X3 2X2 1
Change from previous observation

X2

X3

X2

X3

10

19

51

11

21

56

12

23

61

13

25

66

14

27

71

15

29

76

Numerically, Y increases by 5 in each observation. X2 changes by 1.

MULTICOLLINEARITY

80

Y
70
60
50

Y = 1 + 5X2 ?

40
30

X3
20

X2

10
0
1

Hence the true relationship could have been Y = 1 + 5X2.

MULTICOLLINEARITY

Y 2 3X2 X3
X3 2X2 1
Change from previous observation

X2

X3

X2

X3

10

19

51

11

21

56

12

23

61

13

25

66

14

27

71

15

29

76

However, it can also be seen that X3 increases by 2 in each observation.

MULTICOLLINEARITY

80

Y
70
60
50

Y = 3.5 + 2.5X3 ?

40
30

X3
20

X2

10
0
1

Hence the true relationship could have been Y = 3.5 +2.5X3.

MULTICOLLINEARITY

80

Y
70
60
50

Y = 3.5 2.5p + 5pX2 + 2.5(1 p)X3

40
30

X3
20

X2

10
0
1

These two possibilities are special cases of Y = 3.5 2.5p + 5pX2 + 2.5(1 p)X3, which would
fit the relationship for any value of p.
7

MULTICOLLINEARITY

80

Y
70
60
50

Y = 3.5 2.5p + 5pX2 + 2.5(1 p)X3

40
30

X3
20

X2

10
0
1

There is no way that regression analysis, or any other technique, could determine the true
relationship from this infinite set of possibilities, given the sample data.
8

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

What would happen if you tried to run a regression when there is an exact linear
relationship among the explanatory variables?
9

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

We will investigate, using the model with two explanatory variables shown above. [Note: A
disturbance term has now been included in the true model, but it makes no difference to the
analysis.]
10

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

X
Y

Y
X

X
2i 2 i 3i 3
X 3 i X 3 Yi Y X 2 i X 2 X 3 i X 3
b2
2
2
2
X 2i X 2 X 3i X 3 X 2i X 2 X 3i X 3

The expression for the multiple regression coefficient b2 is shown above. We will substitute
for X3 using its relationship with X2.
11

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

X
Y

Y
X

X
2i 2 i 3i 3
X 3 i X 3 Yi Y X 2 i X 2 X 3 i X 3
b2
2
2
2
X 2i X 2 X 3i X 3 X 2i X 2 X 3i X 3
2
2

X 3i X 3 [ X 2i ] [ X 2 ]

X 2 i X 2 2 X 2 i X 2
2

2 X 2i X 2

First, we will replace the terms highlighted.

12

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

2
2

X
Y

X
2i 2 i
2i 2
X 3 i X 3 Yi Y X 2 i X 2 X 3 i X 3
b2
2
2 2
2
X 2i X 2 X 2i X 2 X 2i X 2 X 3i X 3
2
2

X 3i X 3 [ X 2i ] [ X 2 ]

X 2 i X 2 2 X 2 i X 2
2

2 X 2i X 2

We have made the replacement.

13

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

2
2

X
Y

X
2i 2 i
2i 2
X 3 i X 3 Yi Y X 2 i X 2 X 3 i X 3
b2
2
2 2
2
X 2i X 2 X 2i X 2 X 2i X 2 X 3i X 3

2i

X 2 X 3 i X 3 X 2 i X 2 [ X 2 i ] [ X 2 ]
X 2 i X 2 X 2 i X 2
X 2i X 2

Next, the terms highlighted now.

14

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

2
2

X
Y

X
2i 2 i
2i 2
2
X 3 i X 3 Yi Y X 2 i X 2
b2
2 2
2
2 2
X 2i X 2 X 2i X 2 X 2i X 2

0
X2i 0 X 2 X 3i X 3 X 2i X 2 [ X 2i ] [ X 2 ]

X 2 i X 2 X 2 i X 2
X 2i X 2

We have made the replacement.

15

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

2
2

X
Y

X
2i 2 i
2i 2
2
X 3 i X 3 Yi Y X 2 i X 2
b2
2 2
2
2 2
X 2i X 2 X 2i X 2 X 2i X 2

0
X03i X 3 Yi Y [ X 2i ] [ X 2 ]Yi Y

X 2 i X 2 Yi Y
X 2 i X 2 Yi Y
Finally this term.

16

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

2
2

X
Y

X
2i 2 i
2i 2
2
X 2 i X 2 Yi Y X 2 i X 2
b2
2 2
2
2 2
X 2i X 2 X 2i X 2 X 2i X 2

0
X03i X 3 Yi Y [ X 2i ] [ X 2 ]Yi Y

X 2 i X 2 Yi Y
X 2 i X 2 Yi Y
Again, we have made the replacement.

17

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

2
2

X
Y

X
2i 2 i
2i 2
2
X 2 i X 2 Yi Y X 2 i X 2
b2
2 2
2
2 2
X 2i X 2 X 2i X 2 X 2i X 2

It turns out that the numerator and the denominator are both equal to zero. The regression
coefficient is not defined.
18

MULTICOLLINEARITY

Y 1 2 X 2 3 X 3 u

X 3 X 2

2
2

X
Y

X
2i 2 i
2i 2
2
X 2 i X 2 Yi Y X 2 i X 2
b2
2 2
2
2 2
X 2i X 2 X 2i X 2 X 2i X 2

It is unusual for there to be an exact relationship among the explanatory variables in a

regression. When this occurs, it s typically because there is a logical error in the
specification.
19

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
Source |
SS
df
MS
-------------+-----------------------------Model | 22762.4472
3 7587.48241
Residual | 89247.7839
536 166.507059
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
45.57
0.0000
0.2032
0.1988
12.904

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
------------------------------------------------------------------------------

EARNINGS 1 2 S 3 EXP 4 EXPSQ u

However, it often happens that there is an approximate relationship. For example, when
relating earnings to schooling and work experience, it if often reasonable to suppose that
the effect of work experience is subject to diminishing returns.
20

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
Source |
SS
df
MS
-------------+-----------------------------Model | 22762.4472
3 7587.48241
Residual | 89247.7839
536 166.507059
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
45.57
0.0000
0.2032
0.1988
12.904

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
------------------------------------------------------------------------------

EARNINGS 1 2 S 3 EXP 4 EXPSQ u

A standard way of allowing for this is to include EXPSQ, the square of EXP, in the
specification. According to the hypothesis of diminishing returns, 4 should be negative.
21

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
Source |
SS
df
MS
-------------+-----------------------------Model | 22762.4472
3 7587.48241
Residual | 89247.7839
536 166.507059
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
45.57
0.0000
0.2032
0.1988
12.904

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
------------------------------------------------------------------------------

EARNINGS 1 2 S 3 EXP 4 EXPSQ u

We fit this specification using Data Set 21. The schooling component of the regression
results is not much affected by the inclusion of the EXPSQ term. The coefficient of S
indicates that an extra year of schooling increases hourly earnings by $2.75.
22

MULTICOLLINEARITY
. reg EARNINGS S EXP
Source |
SS
df
MS
-------------+-----------------------------Model | 22513.6473
2 11256.8237
Residual | 89496.5838
537 166.660305
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 2,
537)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
67.54
0.0000
0.2010
0.1980
12.91

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.678125
.2336497
11.46
0.000
2.219146
3.137105
EXP |
.5624326
.1285136
4.38
0.000
.3099816
.8148837
_cons | -26.48501
4.27251
-6.20
0.000
-34.87789
-18.09213
------------------------------------------------------------------------------

EARNINGS 1 2 S 3 EXP 4 EXPSQ u

In the specification without EXPSQ it was 2.68, not much different.

23

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
Source |
SS
df
MS
-------------+-----------------------------Model | 22762.4472
3 7587.48241
Residual | 89247.7839
536 166.507059
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
45.57
0.0000
0.2032
0.1988
12.904

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
------------------------------------------------------------------------------

EARNINGS 1 2 S 3 EXP 4 EXPSQ u

The standard error, 0.23 in the specification without EXPSQ, is also little changed and the
coefficient remains highly significant.
24

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
Source |
SS
df
MS
-------------+-----------------------------Model | 22762.4472
3 7587.48241
Residual | 89247.7839
536 166.507059
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
45.57
0.0000
0.2032
0.1988
12.904

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
------------------------------------------------------------------------------

EARNINGS 1 2 S 3 EXP 4 EXPSQ u

By contrast, the inclusion of the new term has had a dramatic effect on the coefficient of
EXP. Now it is negative, which makes little sense, and insignificant.
25

MULTICOLLINEARITY
. reg EARNINGS S EXP
Source |
SS
df
MS
-------------+-----------------------------Model | 22513.6473
2 11256.8237
Residual | 89496.5838
537 166.660305
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 2,
537)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
67.54
0.0000
0.2010
0.1980
12.91

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.678125
.2336497
11.46
0.000
2.219146
3.137105
EXP |
.5624326
.1285136
4.38
0.000
.3099816
.8148837
_cons | -26.48501
4.27251
-6.20
0.000
-34.87789
-18.09213
------------------------------------------------------------------------------

EARNINGS 1 2 S 3 EXP 4 EXPSQ u

Previously it had been positive and highly significant.

26

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
Source |
SS
df
MS
-------------+-----------------------------Model | 22762.4472
3 7587.48241
Residual | 89247.7839
536 166.507059
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
45.57
0.0000
0.2032
0.1988
12.904

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
------------------------------------------------------------------------------

EARNINGS 1 2 S 3 EXP 4 EXPSQ u

The coefficient of EXPSQ is also strange. It is positive, suggesting increasing returns to
experience. However, it is not significant.
27

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
Source |
SS
df
MS
-------------+-----------------------------Model | 22762.4472
3 7587.48241
Residual | 89247.7839
536 166.507059
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
45.57
0.0000
0.2032
0.1988
12.904

-----------------------------------------------------------------------------. cor EXP EXPSQ

EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
(obs=540)
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
|
EXP
EXPSQ
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
------+-----------------EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
EXP |
1.0000
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
EXPSQ |
0.9812
1.0000
------------------------------------------------------------------------------

EARNINGS 1 2 S 3 EXP 4 EXPSQ u

The reason for these problems is that EXPSQ is highly correlated with EXP. This makes it
difficult to discriminate between the individual effects of EXP and EXPSQ, and the
regression estimates tend to be erratic.
28

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
-----------------------------------------------------------------------------. reg EARNINGS S EXP
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.678125
.2336497
11.46
0.000
2.219146
3.137105
EXP |
.5624326
.1285136
4.38
0.000
.3099816
.8148837
_cons | -26.48501
4.27251
-6.20
0.000
-34.87789
-18.09213
------------------------------------------------------------------------------

The high correlation causes the standard error of EXP to be larger than it would have been
if EXP and EXPSQ had been less highly correlated, warning us that the point estimate is
unreliable.
29

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
-----------------------------------------------------------------------------. reg EARNINGS S EXP
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.678125
.2336497
11.46
0.000
2.219146
3.137105
EXP |
.5624326
.1285136
4.38
0.000
.3099816
.8148837
_cons | -26.48501
4.27251
-6.20
0.000
-34.87789
-18.09213
------------------------------------------------------------------------------

When high correlations among the explanatory variables lead to erratic point estimates of
the coefficients, large standard errors and unsatisfactorily low t statistics, the regression is
said to said to be suffering from multicollinearity.
30

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
-----------------------------------------------------------------------------. reg EARNINGS S EXP
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.678125
.2336497
11.46
0.000
2.219146
3.137105
EXP |
.5624326
.1285136
4.38
0.000
.3099816
.8148837
_cons | -26.48501
4.27251
-6.20
0.000
-34.87789
-18.09213
------------------------------------------------------------------------------

Note that the coefficients remain unbiased and the standard errors remain valid.

31

MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
-----------------------------------------------------------------------------. reg EARNINGS S EXP
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.678125
.2336497
11.46
0.000
2.219146
3.137105
EXP |
.5624326
.1285136
4.38
0.000
.3099816
.8148837
_cons | -26.48501
4.27251
-6.20
0.000
-34.87789
-18.09213
------------------------------------------------------------------------------

Multicollinearity may also be caused by an approximate linear relationship among the

explanatory variables. When there are only 2, an approximate linear relationship means
there will be a high correlation, but this is not always the case when there are more than 2.
32

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

What can you do about multicollinearity if you encounter it? We will discuss some possible
measures, looking at the model with two explanatory variables.
1

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

Before doing this, two important points should be emphasized. First, multicollinearity does
not cause the regression coefficients to be biased.
2

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

The problem is that they have unsatisfactorily large variances.

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

Second, the standard errors and t tests remain valid. The standard errors are larger than
they would have been in the absence of multicollinearity, warning us that the regression
estimates are erratic.
4

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

Since the problem of multicollinearity is caused by the variances of the coefficients being
unsatisfactorily large, we will seek ways of reducing them.
5

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(1)

Reduce

u2 by including further relevant variables in the model.

We will focus on the slope coefficient and look at the various components of its variance.
We might be able to reduce it by bringing more variables into the model and reducing u2,
the variance of the disturbance term.
6

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ
Source |
SS
df
MS
-------------+-----------------------------Model | 22762.4472
3 7587.48241
Residual | 89247.7839
536 166.507059
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
45.57
0.0000
0.2032
0.1988
12.904

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
------------------------------------------------------------------------------

The estimator of the variance of the disturbance term is the residual sum of squares divided
by n k, where n is the number of observations (540) and k is the number of parameters (4).
Here it is 166.5.
7

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source |
SS
df
MS
-------------+-----------------------------Model | 28957.3532
5 5791.47063
Residual | 83052.8779
534 155.529734
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 5,
534)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
37.24
0.0000
0.2585
0.2516
12.471

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
------------------------------------------------------------------------------

We now add two new variables that are often found to be determinants of earnings: MALE,
sex of respondent, and ASVABC, the composite score on the cognitive tests in the Armed
Services Vocational Aptitude Battery.
8

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source |
SS
df
MS
-------------+-----------------------------Model | 28957.3532
5 5791.47063
Residual | 83052.8779
534 155.529734
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 5,
534)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
37.24
0.0000
0.2585
0.2516
12.471

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
------------------------------------------------------------------------------

MALE is a qualitative variable and the treatment of such variables will be explained in
Chapter 5.
9

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source |
SS
df
MS
-------------+-----------------------------Model | 28957.3532
5 5791.47063
Residual | 83052.8779
534 155.529734
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 5,
534)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
37.24
0.0000
0.2585
0.2516
12.471

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
------------------------------------------------------------------------------

Both MALE and ASVABC have coefficients significant at the 0.1% level.

10

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ
Source |
SS
df
MS
-------------+-----------------------------Model | 22762.4472
3 7587.48241
Residual | 89247.7839
536 166.507059
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
45.57
0.0000
0.2032
0.1988
12.904

Number of obs
F( 5,
534)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
37.24
0.0000
0.2585
0.2516
12.471

. reg EARNINGS S EXP EXPSQ MALE ASVABC

Source |
SS
df
MS
-------------+-----------------------------Model | 28957.3532
5 5791.47063
Residual | 83052.8779
534 155.529734
-------------+-----------------------------Total | 112010.231
539 207.811189

However they account for only a small proportion of the variance in earnings and the
reduction in the estimate of the variance of the disturbance term is likewise small.
11

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
-----------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
------------------------------------------------------------------------------

As a consequence the impact on the standard errors of EXP and EXPSQ is negligible.

12

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
-----------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
------------------------------------------------------------------------------

Note how unstable the coefficients are. This is often a sign of multicollinearity.

13

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
-----------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
. cor S ASVABC
EXP | -.0816828
.6441767
-0.13
0.899
1.183748
(obs=540)-1.347114
EXPSQ |
.0130223
.021334
0.61
0.542
.0549311
|-.0288866S
ASVABC
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
--------+-----------------ASVABC |
.2447687
.0714294
3.43
0.001 S | .1044516
.3850858
1.0000
_cons | -26.18541
5.452032
-4.80
0.000
ASVABC |-36.89547
0.5810 -15.47535
1.0000
------------------------------------------------------------------------------

Note also that the standard error of the coefficient of S has actually increased. This is
attributable to the correlation of 0.58 between S and ASVABC.
14

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.754372
.2417286
11.39
0.000
2.279521
3.229224
EXP | -.2353907
.665197
-0.35
0.724
-1.542103
1.071322
EXPSQ |
.0267843
.0219115
1.22
0.222
-.0162586
.0698272
_cons | -22.21964
5.514827
-4.03
0.000
-33.05297
-11.38632
-----------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
. cor S ASVABC
EXP | -.0816828
.6441767
-0.13
0.899
1.183748
(obs=540)-1.347114
EXPSQ |
.0130223
.021334
0.61
0.542
.0549311
|-.0288866S
ASVABC
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
--------+-----------------ASVABC |
.2447687
.0714294
3.43
0.001 S | .1044516
.3850858
1.0000
_cons | -26.18541
5.452032
-4.80
0.000
ASVABC |-36.89547
0.5810 -15.47535
1.0000
------------------------------------------------------------------------------

This is a common problem with this approach to attempting to reduce the problem of
multicollinearity. If the new variables are linearly related to one or more of the variables
already in the equation, their inclusion may make the problem of multicollinearity worse.
15

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(2)

Increase the number of observations.

Surveys: increase the budget, use clustering

The next factor to look at is n, the number of observations. If you are working with crosssection data (individuals, households, enterprises, etc) and you are undertaking a survey,
you could increase the size of the sample by negotiating a bigger budget.
16

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(2)

Increase the number of observations.

Surveys: increase the budget, use clustering

Alternatively, you could make a fixed budget go further by using a technique known as
clustering. You divide the country geographically by zip code or postal area.
17

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(2)

Increase the number of observations.

Surveys: increase the budget, use clustering

You select a number of these randomly, perhaps using stratified random sampling to make
sure that metropolitan, other urban, and rural areas are properly represented.
18

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(2)

Increase the number of observations.

Surveys: increase the budget, use clustering

You then confine the survey to the areas selected. This reduces the travel time and cost of
the fieldworkers, allowing them to interview a greater number of respondents.
19

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(2)

Increase the number of observations.

Surveys: increase the budget, use clustering

Time series: use quarterly instead of annual data

If you are working with time series data, you may be able to increase the sample by working
with shorter time intervals for the data, for example quarterly or even monthly data instead
of annual data.
20

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source |
SS
df
MS
-------------+-----------------------------Model | 161795.573
5 32359.1147
Residual | 476277.268 2708 175.877869
-------------+-----------------------------Total | 638072.841 2713 235.190874

Number of obs
F( 5, 2708)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

2714
183.99
0.0000
0.2536
0.2522
13.262

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.312461
.135428
17.08
0.000
2.046909
2.578014
EXP | -.3270651
.308231
-1.06
0.289
-.9314569
.2773268
EXPSQ |
.023743
.0101558
2.34
0.019
.0038291
.0436569
MALE |
5.947206
.5221755
11.39
0.000
4.923303
6.971108
ASVABC |
.2086846
.0336869
6.19
0.000
.1426301
.2747392
_cons | -27.40462
2.579435
-10.62
0.000
-32.46248
-22.34676
------------------------------------------------------------------------------

Here is the result of running the regression with all 2,714 observations in the EAEF data set.

21

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Number of obs =
540
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
-----------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC
Number of obs =
2714
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.312461
.135428
17.08
0.000
2.046909
2.578014
EXP | -.3270651
.308231
-1.06
0.289
-.9314569
.2773268
EXPSQ |
.023743
.0101558
2.34
0.019
.0038291
.0436569
MALE |
5.947206
.5221755
11.39
0.000
4.923303
6.971108
ASVABC |
.2086846
.0336869
6.19
0.000
.1426301
.2747392
_cons | -27.40462
2.579435
-10.62
0.000
-32.46248
-22.34676
------------------------------------------------------------------------------

Comparing this result with that using Data Set 21, we see that the standard errors are much
smaller, as expected.
22

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Number of obs =
540
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
-----------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC
Number of obs =
2714
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.312461
.135428
17.08
0.000
2.046909
2.578014
EXP | -.3270651
.308231
-1.06
0.289
-.9314569
.2773268
EXPSQ |
.023743
.0101558
2.34
0.019
.0038291
.0436569
MALE |
5.947206
.5221755
11.39
0.000
4.923303
6.971108
ASVABC |
.2086846
.0336869
6.19
0.000
.1426301
.2747392
_cons | -27.40462
2.579435
-10.62
0.000
-32.46248
-22.34676
------------------------------------------------------------------------------

As a consequence, the t statistics of the variables are higher. However the correlation
between EXP and EXPSQ is as high as in the smaller sample and the increase in the sample
size has not been large enough to have much impact on the problem of multicollinearity.
23

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Number of obs =
540
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
-----------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC
Number of obs =
2714
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.312461
.135428
17.08
0.000
2.046909
2.578014
EXP | -.3270651
.308231
-1.06
0.289
-.9314569
.2773268
EXPSQ |
.023743
.0101558
2.34
0.019
.0038291
.0436569
MALE |
5.947206
.5221755
11.39
0.000
4.923303
6.971108
ASVABC |
.2086846
.0336869
6.19
0.000
.1426301
.2747392
_cons | -27.40462
2.579435
-10.62
0.000
-32.46248
-22.34676
------------------------------------------------------------------------------

The coefficients of EXP and EXPSQ both still have unexpected signs since we expect the
coefficient of EXP to be positive and that of EXPSQ to be negative, reflecting diminishing
returns.
24

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Number of obs =
540
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
-----------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC
Number of obs =
2714
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.312461
.135428
17.08
0.000
2.046909
2.578014
EXP | -.3270651
.308231
-1.06
0.289
-.9314569
.2773268
EXPSQ |
.023743
.0101558
2.34
0.019
.0038291
.0436569
MALE |
5.947206
.5221755
11.39
0.000
4.923303
6.971108
ASVABC |
.2086846
.0336869
6.19
0.000
.1426301
.2747392
_cons | -27.40462
2.579435
-10.62
0.000
-32.46248
-22.34676
-----------------------------------------------------------------------------The EXPSQ coefficient has a rather large t statistic, which is a matter of concern. We could

assume that this has occurred as a matter of chance. Alternatively, it might be an indication
that the model is misspecified.
25

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Number of obs =
540
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
-----------------------------------------------------------------------------. reg EARNINGS S EXP EXPSQ MALE ASVABC
Number of obs =
2714
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.312461
.135428
17.08
0.000
2.046909
2.578014
EXP | -.3270651
.308231
-1.06
0.289
-.9314569
.2773268
EXPSQ |
.023743
.0101558
2.34
0.019
.0038291
.0436569
MALE |
5.947206
.5221755
11.39
0.000
4.923303
6.971108
ASVABC |
.2086846
.0336869
6.19
0.000
.1426301
.2747392
_cons | -27.40462
2.579435
-10.62
0.000
-32.46248
-22.34676
------------------------------------------------------------------------------

As we will see in the next and subsequent chapters, there are good reasons for supposing
that the dependent variable in an earnings function should be the logarithm of earnings,
rather than earnings in linear form.
26

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(3)

Increase MSD(X2).

A third possible way of reducing the problem of multicollinearity might be to increase the
variation in the explanatory variables. This is possible only at the design stage of a survey.
27

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(3)

Increase MSD(X2).

For example, if you were planning a household survey with the aim of investigating how
expenditure patterns vary with income, you should make sure that the sample included
relatively rich and relatively poor households as well as middle-income households.
28

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(4)

Reduce

rX 2 , X 3 .

Another possibility might be to reduce the correlation between the explanatory variables.
This is possible only at the design stage of a survey and even then it is not easy.
29

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(5)

Combine the correlated variables.

If the correlated variables are similar conceptually, it may be reasonable to combine them
into some overall index.
30

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source |
SS
df
MS
-------------+-----------------------------Model | 28957.3532
5 5791.47063
Residual | 83052.8779
534 155.529734
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 5,
534)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
37.24
0.0000
0.2585
0.2516
12.471

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
------------------------------------------------------------------------------

That is precisely what has been done with the three cognitive ASVAB variables. ASVABC
has been calculated as a weighted average of ASVAB02 (arithmetic reasoning), ASVAB03
(word knowledge), and ASVAB04 (paragraph comprehension).
31

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source |
SS
df
MS
-------------+-----------------------------Model | 28957.3532
5 5791.47063
Residual | 83052.8779
534 155.529734
-------------+-----------------------------Total | 112010.231
539 207.811189

Number of obs
F( 5,
534)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
37.24
0.0000
0.2585
0.2516
12.471

-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
------------------------------------------------------------------------------

The three components are highly correlated and by combining them as a weighted average,
rather than using them individually, one avoids a potential problem of multicollinearity.
32

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(6)

Drop some of the correlated variables.

Dropping some of the correlated variables, if they have insignificant coefficients, may
alleviate multicollinearity.
33

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(6)

Drop some of the correlated variables.

However, this approach to multicollinearity is dangerous. It is possible that some of the

variables with insignificant coefficients really do belong in the model and that the only
reason their coefficients are insignificant is because there is a problem of multicollinearity.
34

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(6)

Drop some of the correlated variables.

If that is the case, their omission may cause omitted variable bias, to be discussed in
Chapter 6.
35

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(7)

Empirical restriction

Y 1 2 X 3 P u

A further way of dealing with the problem of multicollinearity is to use extraneous

information, if available, concerning the coefficient of one of the variables.
36

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(7)

Empirical restriction

Y 1 2 X 3 P u

For example, suppose that Y in the equation above is the demand for a category of
consumer expenditure, X is aggregate disposable personal income, and P is a price index
for the category.
37

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(7)

Empirical restriction

Y 1 2 X 3 P u

To fit a model of this type you would use time series data. If X and P are highly correlated,
which is often the case with time series variables, the problem of multicollinearity might be
eliminated in the following way.
38

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(7)

Empirical restriction

Y 1 2 X 3 P u

Y ' 1' 2' X ' u

Y ' b1' b2' X '

Obtain data on income and expenditure on the category from a household survey and
regress Y' on X'. (The ' marks are to indicate that the data are household data, not
aggregate data.)
39

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(7)

Empirical restriction

Y 1 2 X 3 P u

Y ' 1' 2' X ' u

Y ' b1' b2' X '

This is a simple regression because there will be relatively little variation in the price paid
by the households.
40

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(7)

Empirical restriction

Y 1 2 X 3 P u

Y ' 1' 2' X ' u

Y ' b1' b2' X '

Y 1 b2' X 3 P u

Z Y b2' X 1 2 P u

Now substitute b'2 for 2 in the time series model. Subtract b'2X from both sides, and regress
Z = Y b'2 X on price. This is a simple regression, so multicollinearity has been eliminated.
41

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(7)

Empirical restriction

Y 1 2 X 3 P u

Y ' 1' 2' X ' u

Y ' b1' b2' X '

Y 1 b2' X 3 P u

Z Y b2' X 1 2 P u

There are some problems with this technique. First, the 2 coefficients may be conceptually
different in time series and cross-section contexts.
42

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(7)

Empirical restriction

Y 1 2 X 3 P u

Y ' 1' 2' X ' u

Y ' b1' b2' X '

Y 1 b2' X 3 P u

Z Y b2' X 1 2 P u

Second, since we subtract the estimated income component b'2 X, not the true income
component 2X, from Y when constructing Z, we have introduced an element of
measurement error in the dependent variable.
43

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(8)

Theoretical restriction

Last, but by no means least, is the use of a theoretical restriction, which is defined as a
hypothetical relationship among the parameters of a regression model.
44

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(8)

Theoretical restriction

S 1 2 ASVABC 3 SM 4 SF u

It will be explained using an educational attainment model as an example. Suppose that we

hypothesize that highest grade completed, S, depends on ASVABC, and highest grade
completed by the respondent's mother and father, SM and SF, respectively.
45

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg S ASVABC SM SF
Source |
SS
df
MS
-------------+-----------------------------Model | 1181.36981
3 393.789935
Residual | 2023.61353
536 3.77539837
-------------+-----------------------------Total | 3204.98333
539 5.94616574

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
104.30
0.0000
0.3686
0.3651
1.943

-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------

A one-point increase in ASVABC increases S by 0.13 years.

46

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg S ASVABC SM SF
Source |
SS
df
MS
-------------+-----------------------------Model | 1181.36981
3 393.789935
Residual | 2023.61353
536 3.77539837
-------------+-----------------------------Total | 3204.98333
539 5.94616574

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
104.30
0.0000
0.3686
0.3651
1.943

-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------

S increases by 0.05 years for every extra year of schooling of the mother and 0.11 years for
every extra year of schooling of the father.
47

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg S ASVABC SM SF
Source |
SS
df
MS
-------------+-----------------------------Model | 1181.36981
3 393.789935
Residual | 2023.61353
536 3.77539837
-------------+-----------------------------Total | 3204.98333
539 5.94616574

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
104.30
0.0000
0.3686
0.3651
1.943

-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------

Mother's education is generally held to be at least, if not more, important than father's
education for educational attainment, so this outcome is unexpected.
48

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg S ASVABC SM SF
Source |
SS
df
MS
-------------+-----------------------------Model | 1181.36981
3 393.789935
Residual | 2023.61353
536 3.77539837
-------------+-----------------------------Total | 3204.98333
539 5.94616574

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
104.30
0.0000
0.3686
0.3651
1.943

-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------

It is also surprising that the coefficient of SM is not significant, even at the 5% level, using a
one-sided test.
49

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. reg S ASVABC SM SF
Source |
SS
df
MS
-------------+-----------------------------Model | 1181.36981
3 393.789935
Residual | 2023.61353
536 3.77539837
-------------+-----------------------------Total | 3204.98333
539 5.94616574

Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
104.30
0.0000
0.3686
0.3651
1.943

-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
-----------------------------------------------------------------------------. cor SM SF
(obs=540)
|
SM
SF
--------+-----------------SM |
1.0000
SF |
0.6241
1.0000

However assortive mating leads to correlation between SM and SF and the regression
appears to be suffering from multicollinearity.
50

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(8)

Theoretical restriction

S 1 2 ASVABC 3 SM 4 SF u

3 4

Suppose that we hypothesize that mother's and father's education are equally important.
We can then impose the restriction 3 = 4.
51

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(8)

Theoretical restriction

S 1 2 ASVABC 3 SM 4 SF u

3 4
S 1 2 ASVABC 3 ( SM SF ) u
1 2 ASVABC 3 SP u

This allows us to rewrite the equation as shown.

52

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

u2

1
1
2
u
b2

2
2
2
1

r
n
MSD
(
X
)
1

X
X2 ,X3
2
X2 ,X3
2i 2

(8)

Theoretical restriction

S 1 2 ASVABC 3 SM 4 SF u

3 4
S 1 2 ASVABC 3 ( SM SF ) u
1 2 ASVABC 3 SP u

Defining SP to be the sum of SM and SF, the equation may be rewritten as shown. The
problem caused by the correlation between SM and SF has been eliminated.
53

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. g SP=SM+SF
. reg S ASVABC SP
Source |
SS
df
MS
-------------+-----------------------------Model | 1177.98338
2 588.991689
Residual | 2026.99996
537 3.77467403
-------------+-----------------------------Total | 3204.98333
539 5.94616574

Number of obs
F( 2,
537)
Prob > F
R-squared
Adj R-squared
Root MSE

=
=
=
=
=
=

540
156.04
0.0000
0.3675
0.3652
1.9429

-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1253106
.0098434
12.73
0.000
.1059743
.1446469
SP |
.0828368
.0164247
5.04
0.000
.0505722
.1151014
_cons |
5.29617
.4817972
10.99
0.000
4.349731
6.242608
------------------------------------------------------------------------------

The estimate of 3 is now 0.083.

54

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. g SP=SM+SF
. reg S ASVABC SP
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1253106
.0098434
12.73
0.000
.1059743
.1446469
SP |
.0828368
.0164247
5.04
0.000
.0505722
.1151014
_cons |
5.29617
.4817972
10.99
0.000
4.349731
6.242608
-----------------------------------------------------------------------------. reg S ASVABC SM SF
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------

Not surprisingly, this is a compromise between the coefficients of SM and SF in the

previous specification.
55

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. g SP=SM+SF
. reg S ASVABC SP
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1253106
.0098434
12.73
0.000
.1059743
.1446469
SP |
.0828368
.0164247
5.04
0.000
.0505722
.1151014
_cons |
5.29617
.4817972
10.99
0.000
4.349731
6.242608
-----------------------------------------------------------------------------. reg S ASVABC SM SF
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------

The standard error of SP is much smaller than those of SM and SF. The use of the
restriction has led to a large gain in efficiency and the problem of multicollinearity has been
eliminated.
56

POSSIBLE MEASURES FOR ALLEVIATING MULTICOLLINEARITY

. g SP=SM+SF
. reg S ASVABC SP
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1253106
.0098434
12.73
0.000
.1059743
.1446469
SP |
.0828368
.0164247
5.04
0.000
.0505722
.1151014
_cons |
5.29617
.4817972
10.99
0.000
4.349731
6.242608
-----------------------------------------------------------------------------. reg S ASVABC SM SF
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------

The t statistic is very high. Thus it would appear that imposing the restriction has improved
the regression results. However, the restriction may not be valid. We should test it. Testing
theoretical restrictions is one of the topics in Chapter 6.
57