Multicollinearity

Measures for Detection

There are following measures for detection of multicollinearity;
1.

Rule of thumb: High R2 and insignificant t-ratios

2.

Correlation between X-variables

3.

Auxiliary regressions

4.

Kliens rule of thumb: multicollinearity is troublesome only if R 2 from

auxiliary-regression > R2 from regular-regression

5.

Tolerance and VIF

6.

Eigenvalues and CI

Rule of thumb:
High R 2 and insignificant t-ratios

When value of R2 is high and the F-statistic is significant, but value of tstatistic for large number of individual coefficients i are statistically
insignificant. The existence of this problem reflects multicollinearity among
explanatory variables.
For Example:
Model

R Squr

Adj R Squr

Sig. F

.987

.876

.869

40.274

.000

Sig

2.37

.01

1.03

.38

Unstd Coeffint

Std Coeffint

Model

Std Er

Beta

Const

.804

.330

PJ

.021

.550

.0237

Correlation b/w Explanatory Variables

Correlations
DJ

PearsonCorrelation

DJ

PJ

IJ

INJ

AEE

Sig.(1-tailed)

DJ

PJ

IJ

INJ

AEE

PJ

INJ

AEE

1.00

0.68

0.51

0.57

0.21

0.68

1.00

0.56

0.66

0.13

0.51

0.56

1.00

0.54

0.11

0.57

0.66

0.54

1.00

0.12

0.21

0.13

0.11

0.12

1.00

0.00

0.00

0.00

0.00

0.00

0.00

0.014

0.00

0.035

.000

IJ

.000

0.00 .

.000

0.00

.000

0.014

0.00 .
0.035

0.024
0.024 .

Auxiliary Regression
By putting an independent variables and test the significance of the
coefficients on added variables. e.g. test of repressor in IV context is
called auxiliary regression
Regression function for Job Satisfaction
JS

F(DJ, PJ, IJ, INJ, AEE)

We have run five times auxiliary regressions on five independent

variables, as follows:
DJ = F(PJ, IJ, INJ, AEE)
PJ =
IJ =
INJ =

F(DJ, IJ, INJ, AEE)

AEE=

F(DJ, PJ, IJ, INJ)

F(DJ, PJ, INJ, AEE)

F(DJ, PJ, IJ, AEE)

Auxiliary regression
The Regression results of five independent variables
and R- Square( R2 ) of each regression are as following.
Variables

R- Square
Auxiliary regression

Regular Regression

DJ

0.516

0.255

PJ

0.596

0.255

IJ

0.383

0.255

INJ

0.494

0.255

AEE

0.040

0.255

Auxiliary regression
The F-statistics is calculated by using value R2 in auxiliary
regression of each variables with following formula

Fi = {R2/(k-2)}/{(1-R2)/(n-k+1)}
n= number of observations
k= number of independent variables
K-2= k-(1+1) as another independent variables get out from
independent list of the original JS regression and consequently
reduce degree of freedom for independent variables measurement.
n-k+1= explanation as above, and consequently it will reduce
degree of freedom for error term measurement.

Auxiliary regression
F-statistics for DJ

FDJ
=

{R2/(k-2)}

{0.516/(4-2)}

/{(1-R2)/(n-k+1)}

/{(1-0.516)/(264-4+1)

= {0.516/2}/{(0.484)/(261)
= {0.258}/(0.001854)
= 139.1281
F-calculated = 139.1281 > F-tabulated = 4.61, with DF = 2 & 261 at p <
0.01, suggesting explanatory variable DJ is strongly correlated
with other explanatory variables.

Auxiliary regression:(Cont..)
F-statisticsforPJ

FPJ =

{R2/(k-2)}

/{(1-R2)/(n-k+1)}

= {0.596/(2)} {(0.404)/(261)}

= {0.298} {(0.001548)}
= 192.5198
F-calculated = 192.5198 > F-tabulated = 4.61, with DF = 2 & 261 at p <
0.01, suggesting explanatory variable PJ is strongly correlated
with other explanatory variables.

Auxiliary regression:(Cont..)
F-statisticsforIJ

FIJ =

{R2/(k-2)}

/{(1-R2)/(n-k+1)}

= {0.383/(2)} {(0.617)/(261)}

= {0.1915} {(0.002364)
= 81.00729
F-calculated = 81.00729 > F-tabulated = 4.61, with DF = 2 & 261 at p <
0.01, suggesting explanatory variable IJ is strongly correlated with
other explanatory variables.

Auxiliary regression:(Cont..)
F-statisticsforINJ

FINJ =

{R2/(k-2)}

/{(1-R2)/(n-k+1)}

= {0.494/(2)} {(0.506)/(261)}

= {0.247} {(0.001939)
= 127.4051
F-calculated = 127.4051 > F-tabulated = 4.61, with DF = 2 & 261 at p <
0.01, suggesting explanatory variable INJ is strongly correlated
with other explanatory variables.

Auxiliary regression:(Cont..)
F-statisticsforAEE

FAEE

{R2/(k-2)}

/{(1-R2)/(n-k+1)}

= {0.040/(2)}/{(0.960)/(261)}
= {0.020}/{(0.003678)
= 5.4375
F-calculated = 5.4375 > F-tabulated = 4.61, with DF = 2 & 261 at p <
0.01, suggesting explanatory variable INJ is moderately
correlated with other explanatory variables.

Kliens Rule of Thumb

According to Kliens rule of thumb, if Auxiliary

R2s are greater than the R2 obtained from regular
regression than multicollinearity will be a
troublesome.

Kliens Rule of Thumb

Variables

R- Square
Auxiliary regression

Regular Regression

DJ

0.516

0.255

PJ

0.596

0.255

IJ

0.383

0.255

INJ

0.494

0.255

AEE

0.040

0.255

Except AEE R2 of auxiliary regression all R2 of auxiliary regression are greater

than regular regression R2 , which show severe multicollinearity.

Tolerance and VIF

TOLERANCE,oritsabbreviation,TOLhasspecialusein
Econometrics,andismeasuredas:
TOL=1R2J
WhereR2jisR2obtainedfromauxiliaryRegression.
Incaseofperfectcollinearityamongsttwoexplanatory
variablesR2Jwillmeasureequalto1,andTOL=0;andin
caseofzero-collinearity,R2Jwillmeasureequalto0,and
TOL=1;

Tolerance and VIF (Cont)

In Case of Perfect Collinearity,

R2J=1and
TOL=1-1=0
In case of no Collinearity
R2J=0and
TOL=10=1
In Case of Imperfect-Collinearity
(0 R2J 1) and
(1 TOL 0)

TOLwillincreaseas
farasR2Jdecreases
(andviceversa)

Tolerance and VIF (Cont)

VIF
TOL has an inverse relationship with variance-inflating-factor, abbreviated as VIF,
like:
VIF = 1 / TOL

or

TOL = 1 / VIF

TheSPSSsregressionoutputcanprovidestatisticsonTOLandVIF,if
regressionisrunwithanadditionaloptionCOLLINERITY
DIAGNOSTICSinstatistics.

ResultsInterpretationofCollinearity
Diagnostic

(1)TOLrangesbetween0and1,thatis:0<TOL<1;So:
(a)ThecloserisTOLtozero,thegreateristhedegreeof
collinearityofthatexplanatoryvariablewithother
explanatoryvariables;hence,wecanidentifywhichoneof
theexplanatoryvariablesiscontributingthehighest
collinearity.

ResultsInterpretationofCollinearity
Diagnostic(Cont)

(1)TOLrangesbetween0and1,thatis:0<TOL<1;So:
(Cont)
(b)ThecloserisTOLto1,thegreateristheevidenceof
non-collinearityofthatexplanatoryvariablewithother
explanatoryvariables.

ResultsInterpretationofCollinearity
Diagnostic(Cont)
(2)TOLandVIFareinversetoeachother,thatis:
VIF =

1/TOL =

1/(1R2J)

(a)IfR2J=0(zero-collinearity),thenTOL=1,andVIF=1
(soVIFhasthelowestlevel=1).
IfR2J=1(perfectcollinearity),thenTOL=0,andVIF=
(VIFgoestoinfinity).
SoVIFrangesbetween1and.

ResultsInterpretationofCollinearity
Diagnostic(Cont)

R2J

TOL = 1 - R2J

VIF = 1 / TOL

0.00

1-0.00= 1/1=
1
1

0.25
1-.25= 1/.75=
(2)TOLandVIFareinversetoeachother,thatis(Cont)
.75
1.33
(b) 0.50
1-.50= 1/.50=
.50

0.75

1-.75= 1/.25=
.25
4

0.90

1-.90= 1/.10=
.10
10

0.95

1-.95= 1/.05=
.05
20

ResultsInterpretationofCollinearity
Diagnostic(Cont)

Relationship of R2, TOL and VIF

R2andTOLhaveinverserelationshipi.e.R2increasesfrom
0to1andTOLdecreasesfrom1to0.
R2andVIFhavedirectrelationshipi.e.R2increasesfrom0
to1andVIFincreasesfrom1to.

EigenvalueandConditionIndex
(CI)

CIisderivedonthebasisofEigenvalueasfollows:
CI==

Eigenvaluecalculationinvolvematrixalgebraandbeyond
thescopeofthisclass,soweconcentrateonlyonCI

EigenvalueandConditionIndex(CI)
(cont..)

CI Calculation:
UsingSPSScommandAnalyze...RegressionLinearDependent
independentstatisticsCollinearityDiagnosticok

EigenvalueandConditionIndex(CI)
(cont..)
Dimension

Eigen Value

CI=

1
1

5.871
5.871

==1

0.057

2
3

CI Calculation:

0.057
.026

==10.164

3
4

.026
.018

==15.035

5
4

.016
.018

.012

.016

==17.946
==18.931

EigenvalueandConditionIndex(CI)
(cont..)

Rule of Thumb for CI:

(a) There would be moderate to strong multicollinearity if CI falls
withinarangeof10to30.
(b)MulticollinearitywouldbesevereifCIexceeds30.