INTRODUCTION

Despite findings in various pieces of literature that the decades of the 1960s and in

some cases the 1970s saw impressive changes in the lives of people in many parts of the

developing world, it soon became clear that the growth and development trend was not

replicating itself into the 1980s. Among some of the persistent and salient ills plaguing third

world economies were most notably poor country specific macro-economic performance.

Thus after two decades of infrastructure and finance lending in the 1950s and 1960s,

it was decided that a continued focus on economic growth alone as the sole indicator of

progress could not be relied upon to reduce poverty, one of the most disturbing indicators of

worsening economic performance. As a result, the decade of the 1980s marked the onset of

Structural Adjustment Programs (SAPs), administered by the International Monetary Fund

(IMF) and in some cases in conjunction with the World Bank for a number of eligible

developing countries. SAPs were a combination of stabilization and adjustment policies

aimed mostly at revamping the macro-economic performance of adjusting countries.

In this light, a comparative analysis of four studies has been carried out in this paper.

The studies each examine the macro-economic outcomes associated with the implementation

of IMF adjustment programs by developing countries around the world. The analysis has

been done with respect to key design features that include the unit of analysis, dependent and

independent variables, type of research design used, elaboration paradigm, temporal order,

MAXMINCON principle, sampling design and external validity, as well as data

development. These papers include:

1. The Macroeconomic Effects of Fund-Supported Adjustment Program - Khan,

Mohsin (1990).

2
 2. Foreign Debt Expansion, the International Monetary Fund, and Regional Variation in

Third World Poverty1 - Bradshaw, Y and Wahl, A (1991).

3. Evaluating the Effect of IMF Lending to Low-Income Countries - Dicks-Mireaux et

al (2000).

4. Policy-Based Lending, Structural Adjustment and Economic Growth in Pakistan -

McGillivray, Mark (2003).

It will be the aim of this analysis to point out the respective strengths and weaknesses of

each study and also suggest some improvements that could be made by future inquiries in

this area of research.

UNIT OF ANALYSIS

The unit of analysis for all studies is a country and its associated macro-economy.

These units of analysis comprise developing countries located in any one of the poorer

regions of the world notably Africa, certain parts of Asia and Latin America. In each study

this is undoubtedly the logical choice of a unit of analysis since IMF structural adjustment

programs are conventionally administered specifically to countries that qualify, through their

governments, with an aim of enhancing their macro-economic performance. At the same

time, a countrys macro-economy is a good indicator of the economic growth and

development as well as the overall performance of a countrys economy. It is thus a valid

starting point when investigating the impacts of IMF interventions.

It must however be noted that measuring the attributes of the unit of analysis is

plagued with a number of difficulties. Indeed differences exist in most studies in this area

owing to the fact that IMF programs have been differentially implemented in various

1
Note that this study has two dependent variables. I have considered only the first, which is the one relevant for
this study i.e. economic development and growth. This is done in recognition that in the modeling section of the
study, each dependent variable is separately regressed on the set of independent variables laid out by the study.

3
countries. This means that in some cases, countries have implemented differing sets of

adjustment programs - so as to suit country specific deficiencies, needs and capabilities, and

over different time periods depending on when countries became eligible (or otherwise) for

IMF support.

For these and other reasons still related to differing background/country specific

variations in e.g. social, political and economic statures, the unit of analysis for the four

studies may not represent a uniform macro-economic outlook at the onset of any study

combining a number of these countries into one data set. Also, problems associated with

institutional weaknesses have a direct impact on the accuracy and consistency of

measurements for the units of analysis.

DEPENDENT VARIABLE

All the studies have their main dependent variable as economic growth. Other

correlates to economic growth are suggested which are the current account, balance of

payments and inflation (Khan, 1990) and inflation and the external debt/service ratio (Dicks-

Mireaux et al, 2000).

The study by Khan conceptualizes economic growth as the rate of growth of real

Gross Domestic Product (GDP). Additionally, the current account is conceptualized as the

ratio of the current account surplus to GDP, the balance of payments as the ratio of balance of

payments to GDP and inflation as the rate of inflation. Bradshaw and Wahl conceptualize

economic growth (and development) as the per capita Gross National Product (GNP) while

Dicks-Mireaux et al conceptualize economic growth as the growth rate of real GDP, inflation

as consumer price inflation and the external debt/service ratio as the ratio of external

debt/service to exports. The final study on Pakistan conceptualizes economic growth as the

growth in real GDP.

4
 In so far as the studies by Khan, Dicks-Mireaux et al and McGillivray all

conceptualize economic growth as an increase in the rate of growth of real GDP, they are

together more highly ranked than the study by Bradshaw and Wahl. This preference is in

recognition that literature in the field of Economic Development posits the opinion that

among growth indicators in the developing world, GDP is often easier to estimate2.

Additionally movements in GDP can be more pragmatically interpreted as increases in a

countrys final outputs. However, movements in per capita GNP are considered more

controversial since a majority of the population may remain poor while a rich minority

continues to get richer, thus distorting the GNP per capita measure and giving it a false

upward bias. It would also be more appropriate to use GDP rather than GNP in so far as using

it may help to avoid any errors that may potentially be made in its compilation. One concern

would for example be the question of whether census data used to calculate per capita values

is accurate.

Again among the studies by Khan, Dicks-Mireaux et al and McGillivray, the first two

do a better job than the study by McGillivray (and thereby that by Bradshaw and Wahl)

regarding the incorporation of additional dependent variables which are indeed deemed

important in the literature in so far as determining the full impact of IMF restructuring

programs is concerned.

INDEPENDENT VARIABLE

As expected, the main independent variable in all the studies is the set of IMF

conditionalities that comprise a structural adjustment program3. However, the fourth study by

McGillivray on Pakistan includes in this variable World Bank policies in addition to those of
2
referenced from Meier and Rauch 2000 pp 5
3
These consist of various specific measures or sets of policy intentions adopted by adjusting countries in order
to restore sustainable balance between aggregate demand and supply, and to simultaneously expand the
production of tradeables. Adjustment tools used by the IMF will generally involve the tightening of fiscal
measures, interest rate management and trade liberalization among others.

5
the IMF. According to Khan, some of the policy variables4 proscribed by the IMF can be

conceptualized in terms of the most important policy measures found in IMF programs which

include:

i. The percentage change in domestic credit

ii. The real effective exchange rate

iii. The ratio of the fiscal balance to GDP.

In the study by Bradshaw and Wahl, the main independent variable - IMF conditionality -

is captured by the use of an index that measures the following:

i. The number of IMF Extended Fund Facility Arrangements for each country (1975-

1985).

ii. The number of multilateral debt renegotiations for each country (1975-1985).

iii. The number of multi lateral debt restructurings during 1975-19855.

iv. The number of debt restructurings between a country and a private commercial bank

during 1978-1985.

A potential source of measurement error arises if these measures become confounded

with the dependent variable as a result of being estimated during the same time period. To

prevent this from happening, the authors ensure that the time period used to generate the IMF

conditionality index does not overlap with the time period from which a measure of the

dependent variable is derived.

In the study by Dicks-Mireaux et al, the main independent variable is depicted as a

dummy variable for the existence (d=1) or otherwise (d=0) of an IMF supported programme

(jIMF). This dummy term representing IMF support is formulated so as to capture four

channels through which IMF support could affect a countrys macro-economy:

4
These are included only in his third model/estimation test referred to as the generalized evaluation method.
5
This includes restructurings carried out between a developing country and the Paris Club, OECD or other
creditor groups not explicitly associated with private commercial banks.

6
 i. Changes in the state of confidence in the economy.

ii. Changes in the desired value of targets.

iii. Policies different from what they would have been in the absence of a program.

iv. Changes in the effectiveness of any given stance of policies.

Measurement errors associated with the IMF conditionality term jIMF are found in the

observation that being an additive constant term, it is only capable of capturing the first two

aspects listed above but not the other two channels which can only be adequately and

comprehensively captured through the use of a more explicit and complex estimation of the

policy reaction function(s).

In the final case study on Pakistan, the main independent variable is conceptualized as

including the liberalization of foreign trade, a reform of financial markets as well as the

deregulation of prices and investment in the agricultural and industrial sectors. In this study,

any measurement errors arise from the fact that due to unique country specific characteristics

such as the institutional capabilities for developing accurate and comprehensive data bases,

accurate measures of all the proxies representing for example trade liberalization and

deregulation may not always be available.

In looking at how each of the studies conceptualize the main independent variable, the

first study by Khan excels in this regard because it makes a thorough attempt at selecting and

operationalizing, from the existing literature, the most relevant measures of the independent

variable6. After Khan, Bradshaw and Wahl make a commendable attempt in this category

through their recognition that the IMF imposes a myriad of conditions on adjusting countries.

To make it easier to represent these aspects in a more or less unified way, they compute an

index by transforming four IMF components/policies into z-scores and then summing them

6
These include the percentage change in domestic credit, the real effective exchange rate and the ratio of fiscal
balance to GDP.

7
up. However, we know that indices are sometimes vulnerable to subjective influences and

errors and are thus are not an entirely dependable measure.

In third place is the study by Dicks-Mireaux et al who though they do well in

representing IMF conditionality as a dummy variable, they also recognize that such a simple

invariant additive term may not capture all the four channels through which IMF support is

expected to affect macro-economic target variables. Finally, the fourth study is faulted in that

despite providing an appropriate summary of the individual IMF conditionalities, it stops

short of indicating how best these aspects can be clearly and practically operationalized.

RESEARCH DESIGN TYPE

The four studies represent designs that are difficult to specifically categorize as either

experimental or quasi-experimental. This is so owing to the fact that the researchers do not

have any control at all over what countries get IMF programs (treatment category) and which

ones do not (control group), and additionally the fact that the designs do not involve random

assignment to treatment and control groups. Rather the treatment and control groups

wherever they may be present are clearly an instance of self-selection in terms of countries

that qualify and seek IMF support towards macro-economic restructuring and those which do

not qualify or seek this kind of intervention. The case study on Pakistan could however be

classified as non experimental.

Even then, all the four studies contain various aspects of experimentation such as in the

attempts of some authors to ensure the measurement of a counterfactual. Looking across the

four articles, we see that two of them (Khan and Dicks-Mireaux et al) are able to recognize

the importance of and indeed provide an estimate for the counterfactual. Khan designs his

study such that the estimation of program effects, that includes the estimation of a

counterfactual, follows two main directions:

8
 A review of existing empirical studies on the effects of Fund programs. Here the

author reviews literature that reveals some of the approaches used to assist

researchers in the estimation of program effects as well as the counterfactual through

(i) a before-after approach7, (ii) a with-without approach8, (iii) an actual-versus-

targets approach9 and (iv) a comparison of simulations approach10.

An empirical estimate by the author of IMF program effects. Similar to the above

section where the author reviews existing work, in this section he investigates

program effects through the use of (i) a before-after estimator, (ii) a with-without

estimator and (iii) a Generalized Evaluation Estimator (GEE)11, elsewhere referred to

as a modified control group methodology.

Since this design incorporates treatment (IMF program) and control groups (non

program) whose before and after measures are estimated, this design comes closest to a

comparative time series. The design ensures a comprehensive estimation of the original

relationship as seen in the movement to the second and third equations with each subsequent

estimation being an improvement over the last. This design is able to avoid problems related

to spuriousness and history. The weakness of this design may however be found in threats

that may arise from contamination and selection such as selection Q, which implies certain

important variables have been omitted from the analysis. Unlike in a pure comparative time

series design, a selection P threat may present itself as a problem in so far as groups/countries

start off differently and due to uncontrolled issues such as sampling inadequacies, these

differences may not be fully accounted for in this kind of analysis.

7
Compares macro-economic performance during a program with the performance witnessed prior to the
program
8
Compares the macro-economic outcomes in countries with programs to performance in a control group of
countries that did not have IMF programs
9
Involves a comparison of actual macro-economic outcomes to the targets specified by the IMF for these
variables at the start of the adjustment period.
10
Relies on a comparison of simulations of economic models involving Fund program-type policies to
simulated performance with some other set of policies
11
The aim is to identify the specific differences between program and non-program countries in the pre-program
period and control for these differences in the post-program macro-economic performance.

9
 The analysis performed in the study by Bradshaw and Wahl does not include a

measurement of a counterfactual in that it has neither a before treatment measure nor a

control group. The study may however be said to conceptually approximate a factorial

design. This type of design is efficient, provides an opportunity for enhancing the

treatment/program and enables a researcher to examine which features or combination of

features in a program have an effect. The design also presents the only effective way for

examining interaction effects such as those developed for region and type of debt in the

Bradshaw and Wahl study. The one obvious disadvantage for this design though is that it may

often prove cumbersome even when there are only a few factors/variables to analyze

(Trochim, 2001).

Dicks-Mireaux et al are notable for designing their study so as to apply the

Generalized Evaluation Estimator (GEE) methodology to their data, similar to the third

estimation technique used by Khan. This study uses a control group but has no before-

treatment measures. In this respect it comes close to a comparative posttest, but like in the

previous two studies the researchers have no control over the allocation of subjects to the

treatment or control group. The advantage of this type of design is that it will be able to

control for threats related to history and spuriousness, but selection threats such as those

involving omitted variable bias as well as contamination in the process of the application of

the treatment (IMF program) or controls in the model remain a significant disadvantage.

Noting that the counterfactual is directly observable only for non-program periods, a

key element brought out in this design is the construction of a counterfactual for policies

during IMF programs. This process is accomplished by the estimation of a policy reaction

function that links changes in policy instruments to the observed lagged value for each target

from its desired value12.

12
See equation 2, page 498

10
 Finally, the study on Pakistan by McGillivray presents a design similar to the one-

shot case study. Here the sample size is equal to one and therefore there is no variation to

explain and thus draw a statistical association from. Additionally, Mohr (1995 pp 64) notes

that this design doesnt include a before measure or even a measure of the counterfactual.

Rather, this study performs one estimation after the treatment has occurred. This lack of a

counterfactual becomes the primary threat especially to internal validity13. History too

remains a significant threat to this type of design.

Despite the lack of a counterfactual measure, the strength of this design, and thus its

ability to establish the original relationship, is attributed to the fact that the author modifies

the research design to incorporate a transition term. This is commendable because even

though analysis pertains to only one country, the design becomes a starting point in initiating

a re-direction of conceptions in this area from past schools of thought that assume the

introduction of IMF programs results in sudden discrete changes in macroeconomic

performance - to a more realistic outlook which posits that since IMF programs are

implemented in a gradual manner over a number of years, their effects are unlikely to be

evidenced in a single year but rather are witnessed through a smooth transition process14

between the pre and post adjustment regimes.

ELABORATION PARADIGM

This paradigm places importance in the art of drawing in or evaluating the effects

generated by the incorporation of alternative explanations into a design and subsequent

13
Indeed the study results suggest Pakistans post-program growth performance has not been stimulated by the
adjustment programs per se.
14
See equations (1) and (2), page 115 where the levels of the dependent variable-real GDP smoothly changes
from equation 1 to equation 1 (b)

11
model. All the four studies each have a different way in which the notion elaboration is made

manifest. The study by Khan handles elaboration by including in subsequent models

additional explanatory variables that are considered to be covariates. This is done as follows:

i. The before-after estimator is a representation of the change in the target variable

with a dummy for program status.

ii. With some elaboration, the with-without estimator comprises the model in (i) plus

a constant term.

iii. The generalized evaluation estimator then comprises the equation in (ii) as well as

nine other explanatory variables such as change in domestic credit, changes in the

terms of trade among others.

The regression results presented in tables two and three indicate that in moving from

model one to model two, two items become statistically significant i.e. the balance of

payments and the current account) as opposed to the significance of only the current account

variable in the first model. Looking at the program effects results for the third model15,

adjustment for pre-program differences matters as seen by the statistical significance of the

own lagged values of each of the outcome variables. A further analysis that includes a two-

year instead of one year regression of model three indicates that positive program effects on

balance of payments, the current account balance and inflation become stronger, while the

negative effects on the main dependent variable (growth rate) become weaker.

Slightly differently from Khan, the study by Bradshaw and Wahl handles elaboration by

way of bringing in and performing additional estimations that incorporate regional aspects

for Africa and Latin America. In this regard, we see that the base model estimates a general

Ordinary Least Squares (OLS) model without regional effects16. Subsequently, two other

models are estimated one for the African and another for the Latin American region. Each

15
Results found in table 3 pp 214, last column
16
See table 3, pp 260

12
of the elaborated models includes two additional variables namely, a region dummy and an

interaction term for region and type of debt17. By comparing the F-test values for this

unrestricted model with the F-test values for the original restricted model represented in

table three, the following results emerge for Africa considering only equation one (table six)

as a result of this elaboration:

Like the results in the original model without regional effects, IMF conditionality,

change in total debt and state intervention continue to show a negative

relationship with economic growth an instance of replication.

The dummy intercept for Africa yields additional information by showing that this

group has a lower economic growth rate (-0.36) than that for the overall sample.

This information was previously suppressed.

The F-test for the unrestricted model accounts for more variation than a

comparable equation for the restricted model with no interaction term. The R-

squared is slightly higher too in model six than in model three.

For Latin America, the addition of a region dummy as well as an interaction term for

region and type of debt leads to the observation that none of the four equations in table seven

explain more of the models variation than the original restricted model does another

example of replication. This leads the authors to conclude that the effects of different types of

indebtedness in Latin American countries are not significantly different from those of

countries in the general sample. In fact we note that the coefficients for the effects of IMF

conditionality on growth are practically identical between model three and model seven.

Again, because this information could not be deduced from the original analysis, suppression

remains a factor as well.

17
See table 6, pp263 and table 7, pp 265

13
 Dicks-Mireaux et al make an attempt at elaborating their basic model, but with little

success. These efforts involve several modifications of the basic model by adding country

and time dummies in order to help account for cross-country differences in economic

structures and time specific exogenous developments. The authors are however forced to

exclude the time dummies from the estimation results of the basic model because they had

little effect or even worsened the overall fit of the equations.

In the Pakistan case study, the author estimates only one overall model with a fixed set of

explanatory variables namely the time trend (t), the transitions term (st) as well as a vector of

four explanatory variables (xt). Despite this fixed nature of estimation, one may view the

model as achieving elaboration through the transition term. Without an adjustment

programme, the model that would apply is represented by equation (1a). However, with the

implementation of an IMF/World Bank program for the country, a transition term is

incorporated into the original model so that the new equation to be estimated is now equation

(1b). With this kind of elaboration and inclusion of a transition term, autonomous real GDP

changes from 1 to 1+ 2 while the rate of growth changes from 1 to 1+ 2.

TEMPORAL ORDER

Temporal precedence is a central aspect in any design in that a researcher has to show

that the causal variable (in this case IMF programs) comes before the effect (changes in the

rate of economic growth). To address this aspect, the study by Khan makes the most

impressive attempt by utilizing designs with the before-after and with-without estimation

techniques. Additionally this study, like the one by Bradshaw and Wahl, attempts to improve

its chances of proving temporal order by using a panel design with 1,104 observations for a

fixed number of countries18. Khan presents an example on the importance of a large number

18
Khan (1990) uses has a set of 69 developing countries while Bradshaw and Wahl perform their estimations on
a group of 62 developing countries

14
of observations in the establishment of causal order by noting that in estimating the GEE

function, the use of fewer observations to perform one year pre-post comparison runs the risk

of failing to capture the full program effects and may also distort the temporal order by

misrepresenting the direction of these effects. To address this issue, the author re-runs the

estimations this time comparing the behavior of the target variables in the pre-program year

with the average behavior of these variables during both the program year and the succeeding

year. The results indicate that as the time period is extended, the positive program effects

such as on the balance of payments become stronger while the negative effects on the growth

rate become weaker. A more solid temporal order is thus established.

In the studies by Dicks-Mireaux et al and McGillivray, both address the issue of temporal

order by recognizing that in this area of study, despite numerous sets of results from a variety

of studies, it is not easy to clearly determine when exactly IMF conditionalities result in

altered macro-economic outcomes. For example, the authors observe that in some instances,

policy change may occur in anticipation of IMF support (such as an attempt to improve

certain macro-economic indicators e.g. interest rates), while at other times the IMF continues

to influence policies even after the particular IMF program is completed.

In the case of Pakistan, the temporal order of adjustment and macro-economic

performance has been thrown into jeopardy with the realization that the transition in

Pakistans real GDP growth had a mid-point during 1982; yet the adjustment program was

officially introduced in 1980, providing evidence therefore that the origins of the growth

transition possibly pre-date and are not caused by the IMF adjustment program. These last

three studies each concede that the direction of causality cannot always be conclusively

determined, despite varied evidence that indeed there are macroeconomic benefits that are

attributable to the structural adjustment programs of the IMF.

15
MAXMINCON

In using panel designs, the first three studies have an advantage in that the individual

units of analysis are used as their own controls. The study by Khan makes the most concerted

and impressive effort at satisfying the MAXMINCON principle in that it aims at maximizing

all possible variance in the main independent variable - the IMF programs - by estimating

this variable in three subsequent equations in an attempt to gauge its best outcome/value.

Additionally, the author incorporates into the final GEE model up to nine other explanatory

variables in an attempt to control extraneous variance but also in order to generate additional

information on the independent effect of these variables on the main as well as each of the

other associated dependent variables. The use of lagged values for both the target and policy

variables is a clear realization of the error minimizing benefits of working with changes in

variables rather than their absolute values (similar to a fixed effect model).

The study by Bradshaw & Wahl seems to inadequately control for all possible

extraneous variance by emphasizing a comparative analysis by region. In the process

however it leaves out key adjustment variables noted in other literature in this area of

research such as exchange rates, fiscal measures and external trade among others. The study

however broadens the scope of its explanatory power by being the first of its kind to test the

impact on economic growth not only of IMF programs but also the different types of foreign

debt held by adjusting countries; two aspects that the authors realize are related rather than

mutually exclusive. In reviewing the works by Dicks-Mireaux et al, despite a commendable

inclusion of a variety of explanatory variables that help to control for extraneous various, the

ability to maximize the experimental variance of the main independent variable is diminished

even by the authors own admission that confounded with estimates of the effects of IMF

supported programs are likely to be the effects of parallel World Bank programs.

16
 Finally the study by McGillivray exhibits an innovative estimation technique by using

the transition function but does not do nearly enough in controlling for variation in

covariates. The vector of covariates used in the model is restrictive and could have been

broadened to especially include more relevant explanatory variables such as the balance of

trade and significant fiscal aspects. All the four studies are able to minimize their error

variance through the use of reasonably large (though in some cases not large enough)

samples.

SAMPLING DESIGN AND EXTERNAL VALIDITY

It is observed that incidentally, though the data is national level data, the samples used

in the first three studies are actually adversely selected such that the threat of selection bias

remains significant. For example, in the study by Khan, the treatment sample comprises

countries that specifically had upper credit tranche lending from the IMF in 1973-88

(referred to as program countries), while the control group comprised of countries without

programs during this time (non-program countries). The author has no control over the

sample19. Rather researchers in this area tend to use most of the data they can get without

performing any specific sampling procedures.

In this and similar studies, the ability to handle external validity remains tenuous. One

of the factors contributing to this aspect is that the samples used are not large enough to suit

the analysis desired by some of the researchers (such as a time series analysis). The study by

Khan makes two important observations regarding external validity:

Despite consistency in the results for the current account and balance of payment

effects, the same cannot be generalized about the effects of IMF programs on

inflation and growth on countries choosing to implement adjustment programs. If

19
Apart from stating that 7 countries are excluded from the total sample of all programmes (pp229).

17
 anything some studies have shown no effects from IMF programs, and others

have found that these programs are associated with a rise in inflation and a fall in

growth rate. The evidence is thus inconclusive, making it impossible to generalize

on any one particular set of results with certainty.

The samples used contain a mix regarding the degree of implementation of IMF

policies within each country. This lack of distinction not only serves as a potential

for bias in the results, but again also makes it difficult to generalize the results to

any set of adjusting countries at a specific level of policy implementation.

Bradshaw and Wahl too do not select a sample from available data but rather use all

data available for a sample of 62 countries with data for the period 1975-1987. This study

exhibits perhaps more sensitivity to external validity than the previous one by its recognition

that studies of this nature need to be carried out on a sub-regional level. Due to the fact that

different regions portray different development characteristics, greater sensitivity to these

differences should be applied in determining IMF program effects. Only then can there be

more accurate generalizations of empirical outcomes based on regional analyses.

Like the preceding studies, the Dicks-Mireaux et al one relies on a self-selected

sample of 61 countries that were eligible for ESAF in the period 1986-1991. The authors take

note of the issue of sampling from increased homogeneity by explaining that, even though

only low-income developing countries were in this ESAF-eligible category, the countries

were still diverse especially for the control group/non-program countries which are located in

Africa, Southeast Asia, the Caribbean as well as Pacific island countries. This study is better

placed than the ones by Khan and by Bradshaw and Wahl in addressing both sampling design

and external validity. This superiority is observed when the authors perform the Heckman

18
(1979) procedure to correct for sample selection bias20 in recognition that unobservable

factors may have influenced a countrys decision to seek IMF support.

Regarding external validity, the Dicks-Mireaux et al study posits the crucial question

of whether indeed the assumptions of the much acclaimed GEE methodology hold valid for

ESAF countries across the board. Noting that a number of diagnostic tests indicate that the

overall fit of the GEE model is poor, and that the counterfactual policy reaction function does

not have significant explanatory power for the non-program sample, the authors are

compelled to conclude that without a rigorous test of the underlying assumptions, it is not

possible to use the GEE methodology to obtain reliable, conclusive and generalizable

inferences about the independent effects of IMF-supported programs.

The last study by McGillivray is a one-item sample and as such ranks lowest in its ability

to facilitate the generalization of the impact of IMF programs to other developing countries.

The main drawback is that the sample size is one and the sampling design in no way includes

an estimate of the counterfactual.

DATA DEVELOPMENT

All the data used in the studies comes from secondary sources mostly at the IMF and

the World Bank. For all the studies therefore, the following advantages and drawbacks

generally apply in using this kind of data. Among some of the advantages include:

This data is efficient to use in that it has already been collected and perhaps

cleaned by someone else

Using this data allows the researcher to extend their scope (in both time and

place) of analysis in a significant way for example in performing various regional

analyses at the same time.

20
See appendix A, pp 523-525

19
 It is less costly than would be the case in generating ones own data say from a

primary survey.

In international comparative studies such as the ones performed by the first three

studies, the secondary data can be re-coded and re-classified to suit the

researchers needs.

This data is especially useful in enabling the replication of research findings from

one study to another. As we know, the true scientific test and proof of a theory is

its ability to be replicated through repeated testing.

The following drawbacks are noted with regard to using secondary data:

Use of this type of data across the board does not take into consideration the array

of problems encountered during the collection of the data. It may then be the case

that country specific data will sometimes be very divergent across the board even

regarding the same variable, owing to aspects such as instrumentation and

measurement errors as well as varied and broader institutional constraints. These

problems may bias or distort the entire data set.

Researchers have to sometimes make certain assumptions about the variables and

their associated data points, assumptions that may lead to biased results if they are

inappropriate or incorrect.

In the process of using these large secondary data sets, some country-specific

aspects of programs are inevitably lost.

More specifically, Khan makes use of pooled time series cross-sectional data available at

the time to analyze program and preprogram effects. Seeing that this is a much earlier study,

the data used may not have been as comprehensive as it would be today for the same

variables analyzed. At the same time, adjusting countries are no doubt more sensitive about

the need to collect more data and also ensure that this data is as accurate as possible. This

20
may not have been the case with the data set used by Khan. The study by Bradshaw and Wahl

notes that the panel data they use in their study is only used because of the unreliability (and

thus the potential for being dirty) of the more preferred time series data. The limitations

leading to this deficiency are attributed to the newness of the debt crisis at the time, as well

as the availability of cross-national data at five year intervals rather than on an annual basis.

On a different level, the study by Dicks-Mireaux et al had to drop a number of data points

in the estimation phase owing to inadequate data as a result of civil strife e.g. in Afghanistan

and Angola. At the same time, the rest of the available data was still of poor quality due for

instance to weak measures of the macroeconomic variables. It was thus necessary to perform

arbitrary corrections for breaks in the time series data, with the obvious disadvantage that

with such manipulation of the data, the resultant inferences cannot be expected to be wholly

credible. Another weakness in the data set regards the short time spanned by the sample,

which imposes a further limit on the level and variety of analyses that can be performed with

the data.

In carrying out the study on Pakistan, the author generally comments on making efforts at

recognizing the constraints imposed by data quality and availability in making his choice of

the vector of independent variables xt to be included in the model. However, the data used is

still secondary and would suffer from the drawbacks mentioned earlier but more on the

whole using secondary data for case study analyses is a time-consuming and expensive way

of generating evidence of program impacts.

CONCLUDING REMARKS

In their own right, the four studies whose research designs have been analyzed in this

paper portray concerted attempts at addressing, to the best of their ability, the macro-

economic outcomes of IMF conditionalities on adjusting countries. Even then, it is clearly

21
recognized that with increasing globalization, the macro-economic trends in the developing

world are increasingly becoming fluid, further necessitating more dynamic research that can

aid in macro-economic policy making for these regions.

It is thus imperative that future analytical endeavours into the effects of multilateral

lending to developing countries broaden their scope to include design features such as those

suggested in this next section.

SUGGESTIONS FOR FUTURE IMPROVEMENTS

Inevitably, analysis of IMF programs and impact in poorer countries continues to be

and will remain a significant challenge to researchers for decades to come. Following are

suggestions that future studies could draw upon:

i. The issue of sampling has been downplayed in the four studies and needs to be

given more thought. Most samples used are self-selected based simply on the

adoption of an IMF program. Despite the recognition that self selection is difficult

if not impossible to eradicate, there is need to have designs that re-sample among

adjusting countries for example on the basis of institutional development, number

of IMF programs adopted as well as the intensity or level of policy discipline

maintained throughout the adjustment period. This will help to enhance external

validity by increasingly basing this type of impact analysis on measures that

discriminate among countries as much as possible.

ii. There is also a need to more practically enhance the ability of studies to control

for extraneous variance through the incorporation of other equally significant but

omitted variables and controls that are clearly related to a number of the

explanatory variables included in the studies. Such controls could include a

countrys record of political performance/stage in democratization, the state of the

22
 world political-economy, civil strife but even more interestingly an ability to

separately control for the gradual growth effects of general macro-economic

stability generated by IMF policies21.

iii. It is difficult to conduct experimental designs (as laid out in Mohr) in this area of

research because countries cannot be consciously requested to adopt/provided

automatic structural adjustment lending or otherwise refrain/automatically denied

from implementing IMF restructuring programs for experimental/research

purposes. In light of this self-selection constraint, future research designs call for

increased innovation and could involve designs integrating case studies as well as

multi-country data. Information from case studies could be a useful supplement to

the broader multi-country analysis on certain specific issues lacking emphasis in

the multi-country analysis.

iv. One aspect that does not come across clearly from the four studies analyzed is

whether an empirical proof has been carried out to ascertain that indeed the

relationship between IMF programs and macro-economic outcomes such as

economic growth is strictly linear. Future designs that explicitly test this may lend

more credence to the true nature of this relationship wherever it is found to exist.

EXTRA CREDIT

The article selected for this section is that by Bradshaw and Wahl (1991). The article

was best noted for its application of Mills third criterion of elaboration in a broadly similar

manner (though not identical) to the empirical outcomes found in the article by McWilliams

21
Currently the only proxy for this seems to be changes in the rate of capital formation.

23
and Siegel22 (2000) titled Corporate Social Responsibility and Financial Performance:

Correlation or Misspecification? In this article, the authors realize that when Research and

Development (RDINT) is left out of the model, the estimate for Corporate Social

Performance (CSP) is upwardly biased. However, inclusion of RDINT in the second model

showed that not only is RDINT a significant explanatory variable but that CSP has only a

neutral and not upwardly significant effect on the target variable Financial Performance

(PERF). When additional explanatory variables (industry dummies) are added, RDINT

maintains its significance.

In comparison, the study by Bradshaw and Wahl makes a realization that the effects

of IMF conditionalities as well as type of debt are correlated and interact with a left out

variable which in this case is geographical region. The study by Bradshaw and Wahl goes a

step further than the one by McWilliams and Siegel in not only including the left out variable

regional dummy (which can be compared to the addition of RDINT), but also includes an

interaction term comprising this left out variable and the second independent variable. The

initial model specification found in table 3 pp 260 in the Bradshaw and Wahl study can be

compared to the first equation in the McWilliams and Siegel article. For the African region,

the elaborated model with a region dummy and slope interaction term for type of debt is

presented in table 6, pp 263. The same analysis is done for the Latin American region, whose

elaborated results are in table 7, pp 265.

The results for Africa indicate that the slope dummy for private debt is negative and

significant as depicted by the F-test for the interaction term. Looking at the Latin America

results, we see that the effect of private debt changes to be insignificant, meaning private debt

is not as harmful in this region like it is in the African region. In all elaborations, the

significance of the main independent variable, IMF conditionalities, is maintained.

22
See attached article pp 608

24
 We then see that the Bradshaw and Wahl study provides more elaborate analysis of

the left out variable more so through the interaction term. Without elaboration to include

geographical differences, it would not be known with certainty whether IMF effects on

economic growth are similar across the regions and also what type of debt is possibly most

harmful in each of the two developing regions.

LOCATOR MATRIX

Study 1 Study 2 Study 3 Study 4

(Khan, M - (Bradshaw& (Dicks-Mireaux (McGillivray, M

25
 1990) Wahl - 1991) et al - 2002) - 2003)
U/A 196 257 495 113

DV 195,211 251,257 495,497,501,503 114,115

IV 196,212 258-259 497-498, 499-500 113-114

RD 197,198-212 256-257 496-498 113-116

EP 211-218 252,260 505 115-116

263-256
T 196,216-218 261-262,256 504,521,522 114,119-120

MMC 211-212 251 505 115-116

SD 210-211,222 259,269 500-501,505 113

522-525
DD 210 257 500-501 259

REFERNCES

Bradshaw and Wahl., Foreign Debt Expansion, the International Monetary Fund, and

Regional Variation in Third World Poverty. International Studies Quarterly (1991)

Vol 35, 251-272

Dicks-Mireaux et al., Evaluating the Effect of IMF Lending to Low-Income Countries.

Journal of Development Economics Vol 61 (2002) 495-526

Khan, Mohsin. The Macroeconomic Effects of Fund-Supported Adjustment Program. IMF

Staff Papers Vol 37 No. 2 June 1990

McGillivray, Mark. Policy-Based Lending, Structural Adjustment and Economic Growth in

Pakistan. Journal of Policy Modeling Vol 25 (2003) 113-121

26
 ADDITIONAL REFERENCES

Hakim, C. Research Design, 1897. London: Allen & Unwin Publishers Ltd.

McWilliams, A and Siegel, D. Corporate Social Responsibility and Financial Performance:

Correlation or Misspecification? Strategic Management Journal 21: (2000) 603-609

Meier, G and Rauch, J. Leading Issues in Economic Development, 2000. New York:

Oxford University Press.

Mohr, L. Impact Analysis for Program Management, 1995. California: Sage Publications.

Trochim, W. The Research Methods Knowledge Base, 2001. Ohio: Atomic Dog Publishing.

27