Abstract

Since the 1960s, IMT/PIM has formed the central tenant of institutional reform in the irrigation sector worldwide. Despite the widespread application of IMT as a remedy for poor systems performance, the actual effectiveness of transfer programs in terms of improving irrigation, agricultural and management efficiency is still a subject of much debate. Through the systematic review of 65 case studies of IMT/PIM in Africa and Latin America, this study identifies important gaps in the literature on management transfer, in particular the need for higher quality and longer-term impact assessments of transfer programs to better establish a causal link between IMT intervention and impact. It also provides insights into the conditions that shape the effectiveness of institutions for irrigation management. In particular we find that in the African context, three factors were consistently correlated with successful IMT/PIM, these are credit and market access, property rights and NGO-donor assisted implementation. In Latin America, governance issues such as WUA organizational structure and water rights appear to increase the likelihood of successful management transfer. Thus, our research suggests that what matters in Africa is not always relevant in Latin America and vice versa, Even within these regions performance is mixed and impact across various technical, social, economic and agricultural indicators is highly variable. Moreover, our analysis suggests that some factors may be so critical to successful transfer, for instance, the ability to absorb the costs of management that they impact the significance of others, such as the type and size of scheme. Put simply, no clear, consistent pattern emerges that offers a recipe for the successful replication of IMT/PIM interventions elsewhere. Rather our work suggests that irrigation management reform must move beyond single-policy solutions if it is to address farmers development needs.

Introduction
Since the 1960s, the debate on how to best improve the management of the public irrigation sector has stagnated. The policy which has ruled irrigation reform discourse for the past 50 years claims the answer lies in devolving some or all of the management authority for irrigation systems to organised user groups or Water User Associations (WUA). This phenomenon is termed Irrigation Management Transfer (IMT) or Participatory Irrigation Management (PIM) and has been adopted in more than 60 countries around the world (FAO, 2007). The literature on public sector irrigation reform is replete with case studies of PIM/IMT. Although individual case studies are an important source of data and analysis on performance trends at a scheme or country-level it is also useful to situate them in a larger context and differentiate between localised trends and regional/global developments (Giordano et al. 2010?). This is especially pertinent when single policy solutions are taken up based on their success in particular schemes and transplanted to a wide range of locations and problems, as has been the case with IMT/PIM interventions. In spite of this, very few papers systematically combine and analyse the findings of IMT/PIM case studies. The value of such systematic review is established in the works of Petrosino et al. (2001); Farrington (1998, 2003); Welsh and Farrington (2006) and Van der Knapp et al. (2008) who argue that when done well and with full integrity, they provide the most reliable and comprehensive statement about what policy interventions work, under which conditions and why. This review attempts to partially fill that gap. It draws on the most comprehensive database of IMT/PIM cases in Africa and Latin America to date and updates the very limited number of existing reviews on the policy (Vermillion, 1997; FAO, 2007) with new evidence and methods of analysis. While this paper builds on previous research it also differs from it in several ways. In particular, with 65 case studies from 20 different countries it taps into a much larger database and geographical scope than Vermillion or FAO and is thus amenable to regression analysis.1 Moreover, unlike FAO we analyse only localised case studies of IMT/PIM and hence are able to capture in-country variation in terms of IMT/PIM impacts and outcomes. The results of this review provide insights into the quality of evidence and research methods used in IMT/PIM impact evaluations. Through the systematic analysis of case study design we establish that available impact assessments largely fail to causally link improvements in scheme performance and transfer programs. For this reason, the actual effectiveness of IMT in terms of improving irrigation, agricultural and management efficiency remains questionable.

FAOs review (2007) covered only 8 cases from Africa and 7 from Latin America and Vermillions (1997), analyzed only 4 from Africa and 4 from Latin America.
1

We also develop and apply a methodology for classifying each of these IMT/PIM case studies as successful or failed. Performance is measured from several perspectives, drawing on ten financial, technical, social and agricultural outcome and impact indicators. Each intervention is assigned a unique score and is ranked in terms of success or failure. Finally, drawing on these success scores we investigate patterns of success/failure through a combination of simple correlation, binary logistic regressions and qualitative comparative analysis. From this we provide empirical support to some prevalent hypotheses regarding the pre-conditions for successful management transfer. Without doubt, PIM policy has generated some success stories, particularly in the dynamic commercial farming sectors of South Africa and Northern Mexico. However, overall our results show that the outcomes of IMT/PIM in the African and Latin American regions are mixed and the positive impacts of the devolution program, ranging from livelihood security to improved technical efficiency, show great variability across and within locations. This variation, we argue, reflects the difficulty of transplanting institutions from one context to another when local situations vary significantly in terms of technology, water availability, cropping patterns, market development, social capital and government policies (Meinzen-Dick, 2007: 15200). Nowhere is this institutional mismatch and disconnect more pronounced than in the small-holder irrigation context of Africa. In this context of moderate dynamism and poverty, interventions in the irrigation sector must move beyond single-policy solutions and encompass, land reform, market access, and other support services which will help farmers increase their livelihoods and consequently their ability to absorb the costs and responsibilities of irrigation management.

Research Questions
Our review considers evidence from 65 case studies of IMT/PIM in Latin America and Africa to answer the following questions: 1. How well have the impacts and outcomes of IMT/PIM in Africa and Latin America been documented and what have we learnt from those studies? 2. How can we evaluate the impact and outcomes of IMT/PIM and differentiate the successful cases from not so successful ones? 3. What are the conditions under which successful or at least promising WUAs in public irrigation systems are found? Are those conditions replicable? The substantive sections of this paper each address one of these research questions.

Methodology
Given the objectives of this paper, a five-stage review process was undertaken. First we attempted to locate all existing case studies of IMT/PIM by searching academic

databases and Google and Google Scholar search engines. 2 Sources were also identified from the bibliographies of previous meta-reviews (Vermillion, 1997: Shah 2002; FAO 2007). These searches returned over 2,500 records. However only studies which met the following criteria were included in our review: a) Case studies were limited in geographic scope to Africa and Latin America. b) Records were localized case studies of one or more Water User Association and not country level studies. c) Case studies were of publicly owned irrigation systems that were later handed over to the farmers for management either as a decision of the government, donors or due to the interest of the farmers. This meant we did not include case studies of farmer managed irrigation systems (FMIS), which, unlike the turned over or transferred systems of our interest, were built and managed by the farmers themselves. d) Case studies were published after 1994. The year 1994 marked a watershed year in the study of IMT/PIM initiatives. It was in this year that a major conference on PIM/IMT was held at Wuhan, China. Here, more than 100 case studies were presented and later documented in 2 volumes of conference proceedings. Based on the assumption that IMT/PIM initiatives in the pre-1994 era were adequately studied and documented, we wanted to review only the post-1994 case studies so as to reduce duplication in results and make our review more up to date and focused. e) Case studies were documented in the English language. We include the references for papers excluded on this basis so that they may be incorporated into future reviews. After reviewing titles and abstracts, all but 137 case studies were excluded on the basis of relevance. A further 57 studies were eliminated because they did not contain a localised intervention. Another 8 sources were eliminated as they did not provide enough information on the impact and performance of IMT and 7 sources were eliminated because they were published in languages other than English (French, Spanish and Japanese). In total, 65 distinct case studies of IMT/PIM were included in our review. We made a concerted effort to include as many case studies from as many countries as possible. To this end we undertook country and schemes specific searches in an attempt to maximize sample size and country representation; however despite our best efforts potentially relevant sources were either inaccessible or failed to meet our inclusion
The following databases were searched: Water Resources Abstract, CAB Direct, Econ Lit, Sociological Abstracts, Web of Science, Scopus, World Bank Publications, FAOs AGRIS database, Environmental Sciences and Pollution Management, ProQuest Central, EVA Environmental Abstracts, Ingenta Connect, Ovid Databases, JSTOR, Sage Journals, Science Direct and IWMIs catalogue.
2

Comment [SN(1]: In his comments earlier this year, Mark thought this was not a good reasonperhaps some more justification that there would be overlap?

criteria. As a result, there are regional biases in the data collected, with some countries such as South Africa, Mexico and Columbia over-represented and others, such as Burkina Faso, Chad and Ecuador not accounted for at all. Second, we coded studies on a range of methodological, descriptive and outcome/impact related attributes. The purpose of the coding was to make comparable a number of seemingly diverse and inconsistent results across IMT/PIM case studies. Coding was based on the Institutional Analysis and Development (IAD) framework which draws on various disciplines such as political science, economics, anthropology, game theory and law, to study common property resources (CPR). In short, this framework analyses the physical attributes of the resource, the community attributes of the people managing them and the attributes of the institutions that have been formed to manage the resource (Oakerson 1986). Coding was done to gain insights into the three core research questions which frame this paper. It enabled us to: a) Assess the quality and strength of evidence used in case studies. This was done by identifying and coding the study design and data collection methods used in case studies. We collated information on case study authors methods of analysis; whether or not they undertook independent verification of outcomes and impacts and the number of years that elapsed between the IMT/PIM intervention and the impact assessment. There were 21 methodological parameters assessed in this study and these were then used to evaluate the quality of research into IMT/PIM in general. Our list of indicators conforms to and builds on Vermilions (1997) analysis of the evidence on IMT/PIM. b) Measure IMT/PIM case studies in terms of their outcomes and impacts. We chose 10 indicators of success which closely conform to the definition of successful IMT/PIM initiatives endorsed by both the International Network on Participatory Irrigation Management (INPIM, see http://www.inpim.org/) and FAO (2007). We measure the following outcomes: irrigation service fee collection rates; financial viability of WUA; functional condition of infrastructure (good maintenance); enhanced equity; reliability and adequate water supply; reduction in government expenditure after transfer, popular participation and reduction in disputes (or increase in rule conformance). In addition we measure the following impacts: crop related impacts (increase in production, yield, cropping intensity) and livelihoods related impacts (increase in income, employment, wage rates, reduction in forced out-migration etc.). Table X presents the list of outcome and impact indicators we used to evaluate success. c) Identify factors that that influence the success of IMT interventions. To do this we coded cases on 25 descriptive attributes including their technical specifications (type of scheme, size, complexity, whether or not it had been rehabilitated); socio-economic and agricultural characteristics (population served, major crops, per capita GDP, access to credit, markets, importance of agriculture, social 5

cohesion, land rights, water rights); and their IMT/PIM experience (level of transfer, training, political will, legislation, election to WUAs). Third, to allow for further commentary on the methodological quality of each selected IMT/PIM case study we developed and applied a modified version the Maryland Scientific Methods Scale (SMS) which is recommended for systematic reviews by Welsh and Farrington (2006) and Van der Knapp et al. (2008). This is a five-point scale that rates the strength of scientific evidence used in impact evaluations, with 1 being the weakest and 5 the strongest scientific evidence in terms of inferring cause and effect. 3 Given that our sample does not contain a single case study with a traditional SMS score of 4 or above (i.e. none used an experimental design with statistical control of extraneous influences on the transfer process) we adapted the scale to better reflect the actual analytical techniques and evidence used in our source documents. This is because the nature of PIM/IMT interventions rules out impact assessments by experimental design (such evaluation research measures change before and after an intervention in units that are randomly assigned to experimental and control conditions).4 It does not, however rule out scope for rigor by combining a before-after with a with-without comparison. We assigned a rating of one to four based on design robustness, as described in detail below. Level 1. Studies which measure the correlation between an intervention and an indicator of performance at a certain point in time, after the introduction of the intervention. This design fails to rule out any threat to internal validity and also fails to establish causal order. Level 2. Studies which measure the performance before and after the intervention, without a (comparable) control condition or statistical analysis. Level 3. Studies which measure the performance before and after the intervention, without a (comparable) control condition but with statistical analysis. Level 4. Studies which measure the performance before and after the intervention, and also have a comparable control group (quasi-experimental design) with statistical analysis. By grouping case studies according to SMS levels we can distinguish between high and low quality impact assessments. Given that the primary objective of a systematic review is to combine the findings of a large number of studies with the aim of drawing credible, generalizable conclusions about what policy interventions work, where and why,
The SMS was designed by Sherman et al, as a tool to evaluate the methodologies for their review of over 500 crime prevention interventions for the U.S congress. The scale has since been considered a key resource for appraising quantitative studies. The five levels are described by Welsh and Farrington (2006).
3 4

This type of impact assessment is common in the field of public health.

attribution challenges arise from the inclusion of studies with non-experimental evaluation designs. Through the SMS we can assign labels of success or failure with caveats and soften many problems that arise from the presence of low quality studies (SMS of 2 and below) in a review of this kind. In the end, systematic reviews are only as robust as the existing evidence on a given intervention enables them to be. Fourth, we constructed a composite success score (CSS) from the coding of outcome and impact indicators and use this to differentiate successful cases of IMT from not so successful ones. This scoring system (CSS) is adapted from Mukherji et al. (2009).5 The CSS draws on the dichotomous coding of outcome and impact indicators (1= if performance improved; 0= no change in performance or deteriorated) outlined in Table X and simply involves adding up all positive scores and dividing this sum by the maximum possible score. Here, the challenge was to deal with missing values in each of the case studies (as the data availability among case studies varied between 4-10 outcome/impact indicators). We decided that whenever data was not available, we would leave it out from our calculation. For example, if outcome and impact indicators were available for 8 out of 10 (maximum) possible indicators, then we calculated the CSS assuming a maximum possible score of 8 (and not 10). If out of these 8 indicators, four were positive (therefore getting a cumulative score of 4) and 4 were negative or neutral (getting a cumulative score of 0), our CSS would be 4 out of 8 or 0.5 which can be converted to 5 on a 10 point scale (see Appendix 7).
Table X: List of outcome and impact indicators used for construction of success scores
Sr. No. Name of the indicator Scoring system Number of cases where this indicator was present (Max=65) 56

A. 1.

Outcome indicators Irrigation service fee collection rate

1= if it has gone up 0= no change or declined n/a = not available 1= if it has improved 0= no change or deteriorated n/a = not available 1= if it has improved 0= no change or deterioration n/a = not available 1= if it has gone up 0= no change or declined n/a = not available 1= if it has gone up 0= no change or declined

2.

Financial viability of WUA

59

3.

Functional condition of infrastructure

64

4.

Equitable distribution of water

46

5.

Reliability and adequacy in water distribution

49

For a discussion on the strengths and limitations of this scoring method please refer to Mukherji et al (2009); pp. 13-14.
5

Sr. No.

Name of the indicator

Scoring system

Number of cases where this indicator was present (Max=65)

n/a = not available 6. Reduction in government expenditure 1= Yes 0= Otherwise n/a = not available 1= if it has gone up 0= no change or declined n/a = not available 1= Yes 0= No or got worse n/a = not available 1= If any one of these registered an increased after transfer 0= Otherwise n/a= not available 1 = if any of these have gone up after transfer 0=Otherwise n/a = Not available 30

7.

Popular awareness and participation in WUA activities

60

8.

Reduction in frequency of disputes

33

B. 1.

Impact indicators Crop related impacts (production, yields, cropping pattern, cropped area)

50

2.

Livelihoods and household parameters (income, wage, employment, poverty reduction, reduction in forced migration)

39

All cases that got a value of 5 and above in our CSS scale were categorized as effective and those with scores of 5 and less were categorized as ineffective cases of IMT. While it may be argued that using a score of 5 as the cut off point is rather arbitrary, we justify our cut off point on the grounds that varying the cut off point between 5 and 6 (when 5.5 is the mean value on a scale of 0 to 10) does not change the overall number of successful and failed cases. In short, every case study was scored and ranked on a uniform scale of 0 to 10. In cases which scored 0, the transfer process produced no positive changes in scheme performance. In cases that scored 10, the transfer process produced only positive changes in scheme performance There are obvious limitations with some of the indicators used it our system of classification. This is particularly true for the impact indicators. Very often, increase in crop yields, incomes or area under acreage, even, if it happens after the intervention, may not be directly attributable to greater farmer involvement in O&M per se. Rather, many factors exogenous to IMT interventions such as, changes in fertilizer application rates, higher crop prices, rainfall patterns and more favourable market conditions can affect the performance of irrigated agriculture. Furthermore, it should be pointed out that investments in farmer management are often part of a wider set of economic and agricultural interventions, including promoting marketing and micro-credit, highyielding seed technology and better on farm management. However, because case studies do not explicitly examine casual links between IMT interventions and systems 8

performance our CSS is forced to draw on these problematic but very well documented indicators (over 70% of case studies measure success or failure of IMT interventions based on crop and livelihood related impacts). This may mean that our evaluation of IMT may appear to be more positive than might actually be the case. An additional source of bias with our database stems from the fact that these studies are snapshots of cases at one point in time ranging from 1995 to 2010. In most cases, we do not have any information on current status of these schemes. It is possible that schemes classified as success/potential success stories in our analysis might as well have failed over the years or our failed cases might have turned around and become successful later on. We do not have the time or the resources needed for verifying the current status of our case study schemes. In short, this type of review gives expression to static perspectives: performance is measured at one point in time. However, we recognize that WUAs may be at different stages of maturity and that IMT interventions are impacted by spatial and temporal dimensions. Finally, we drew on simple correlations and regression analysis to identify and investigate which socio-economic and technical mechanisms appear to be active in successful management transfer. Given the limited number of case studies from Latin America (18 in total) our regressions only include data from African case studies. The dependent and independent variables that we consider for our Logit model are detailed in Table X. The dependent variable in our case is dichotomous and takes the value 1 if an IMT intervention was successful or 0 if failed. On the other hand, we considered several independent variables that might influence the probability of successful management transfer. All are drawn from the list of 25 descriptive indicators coded in step 2. Two variables were dropped from the regression as they were problematic these are, importance of agriculture in people's livelihood and amount of O&M authority transferred. Table X: Description of dependant and independent variables used in the logit model.
Variables Dependent Variable SUCCESS Domain Binary dummy variable, where 1 = successful/potentially successful case of IMT/PIM and 0= failed/potentially failed cases Dummy variable: scheme_type where Lift & pump =1, others =0 Dummy variable: scheme_size where Small=0, Large & Medium=1 Dummy variable: crop_value where low or mixed=0, high=1

Independent Variables: Type of scheme Size of scheme Value of crops grown

Type of crops grown Access to credit/markets/support services Land rights Implementing Agency

Dummy variable: crop_type where Paddy=0, non paddy and mixed=1 Dummy variable: acess_credit where low or mixed=0, high=1 Dummy variable: landrights where low or mixed=0, high=1 Dummy variable: Implementer where Donor & NGO assisted or farmers =1, Government=0

By correlating success and failure with individual scheme characteristics, we were not only able to identify patterns of success but also, test some of the hypotheses put forward in the literature on the determinants of successful management transfer. Our regressions do not attribute causality rather we can only comment only on the association of variables and rates of success.

1. Review of the Evidence: How well have the impacts and outcomes of IMT/PIM
in Africa and Latin America been documented and what have we learnt from those studies? Over a decade ago, Vermillion characterised the literature on the impacts and outcomes of IMT as a disparate collection of definitions and methodologies from which it is difficult to deduce general conclusions or policy implications (1997: 5). At the heart of his criticisms were problems of causal attribution which arise as a result of post-hoc fallacies, fallacies of induction and short-term impact assessments. Our review, fourteen years on, is frustrated by these very shortcomings and as such the generalizations that can be derived from these case studies are very limited. We are however, able to systematically assess the evidence about the impacts of management transfer with reference to performance measures and the quality of research methods used in our case studies. By grouping case studies according to the Maryland Scientific Methods Scale (SMS) we show the overwhelming majority of evidence on IMT/PIM in Africa and Latin America, is of low scientific quality. As Figure X indicates most IMT impact assessments rely solely on post-impact observations and data (SMS 1) to infer cause and effect. Assessments which claim interventions produced changes in performance without providing a reference point for those comparisons are inherently problematic. Studies that use only the before and after or with or without methodological design (SMS 2 and 3) also often fail to establish a causal relationship between IMT intervention and impacts. Many factors exogenous to IMT such as better farm management, changes in fertiliser application rates, rainfall patterns, changes in technology or cultivation practices and more favourable market conditions can affect the performance of 10

irrigated agriculture. Moreover, in the majority of available case studies IMT followed or was part of a wider set of neoliberal economic and agricultural reforms all of which impact systems performance.

40 35 Number of Case Studies 30 25 20 15 10 5 0

37

11

13 4

Adapted SMS Score

Figure X: IMT case studies according to SMS

We found that only 4 of our case studies (out of 65) used a quasi-experimental design to assess impact by combining before after and with-without analysis (SMS level 4). Qualitative review of these high qualitycase studies suggest that it is only possible to trace certain changes to IMT/PIM policies such as, improvements in management efficiency, staff accountability and the significant shift in the burden of cost from the government to farmers. From these high quality studies it is clear that transfer has not had substantial impacts on the performance of operations and maintenance, or on the agricultural and economic productivity of irrigated land or water neither improving negative performance nor causing detriment where performance is positive (Vermillion and Garcs-Restrepo, 1998: 33). For example, in the rigorous evaluations of Samac and RUT irrigation systems (Columbia), performance cross almost all documented impact/outcome indicators, remained constant or could not be linked to management transfer. In Samac and RUT farmers shifted to higher value crops just before transfer and thus, positive posttransfer crop impacts may not be attributable to farmer management. These high quality impact assessments do not shed light on the factors that increase the probability of IMT failure, as much as they reinforce arguments about the difficulty of attributing changes in scheme performance to IMT alone. Indeed, they draw attention to the link between the quality of design and empirical findings, by reinforcing Rossis (1987)

11

stainless steel law of evaluation which states that the better designed impact evaluations are, the less effective the interventions or programs seem to be. It is also worth noting that many of these attribution problems are manifest in case studies which evaluated rehabilitated schemes. In 31 studies, transfer was accompanied by physical improvement of the system. In these cases, it is difficult to determine the extent or degree to which initial positive results, in terms of water delivery and crop related impacts, are attributable to either farmer management or to systems rehabilitation. Of the 31 studies which were rehabilitated before or during transfer only 4 were evaluated via with and without comparisons. As a result, case studies authors made little attempt to disaggregate the effects of rehabilitation and transfer by comparing systems that were transferred with and without rehabilitation. These problematic causal links between IMT and changes in scheme performance make it difficult to draw robust conclusions about the impacts of transfer. We also assessed case studies according to their use of time series and statistical analysis. Of the 24 studies which compared performance before and after transfer, only 14 examined data on 5 or more consecutive time periods. Without sufficient time series analysis it is impossible to firmly establish the timing of impacts, to control for alternative causes of change and to identify the extent transfer has influenced the outcome (Vermillion, 1997). The low rates of quantitative analysis also exacerbate problems of casual attribution. Only 30 cases used statistical tests compared to 52 studies which quoted descriptive statistics. Statistical and econometric analyses may help to overcome some of the methodological shortcomings identified in this review. For example, combining with and without and before and after analyses with difference in difference analysis will help establish a causal link between intervention and impact. Using other quasiexperimental methods such as an instrumental variable approach will similarly help in identifying casual attribution and therefore resolve the most pertinent methodological weakness in IMT/PIM studies. Further attribution problems arise from the types and rigour of data collection methods used by case study authors.

12

60 51 Number of Case Studies 50 40 40 31 30 20 10 0 Literature Review/Synthesis of data from other studies Secondary data from agency/ government departments Direct measurment of operations Farmer Interviews Stakeholder perspectives Systematic Sampling Anectodal evidence/ field observations Direct inspection of structures Independent measurement of outcomes & impacts? 19 13 17 19 32 46

Indicators

Figure X: Data collection methods used in IMT case studies

In particular, Figure X suggests several common tendencies. First, 94% of case studies rely on secondary sources, including data collected by irrigation agencies, government departments and/or other scholars. Second, less than half (41%) of the studies which conducted farmer interviews selected participants through systematic, random sampling methods. As a result, the conclusions drawn and perspectives expressed on the basis of farmer interviews may not represent the general perception or actual performance of the transferred unit. Finally, out of 65 cases we found that only 13 evaluations undertook independent measurements of water distribution operations and only 17 involved the direct inspection of scheme infrastructure. Taken together, these flaws suggest that in many cases, authors may have either overestimated or underestimated performance levels in their studies. For example, before and after analyses which depend on data from government sources may be biased in the reporting of pre-transfer performance levels leading to an underestimation of posttransfer impacts.6 Similarly, interviews or surveys which only report the perspectives of upstream farmers or WUA leaders may risk overestimating positive changes in terms of water distribution and equity. In short, our analysis indicates that most studies failed to adequately verify their conclusions about the outcomes and impacts of IMT.

For a detailed discussion of this see: Vermillion (1997).

13

Our case studies also varied in terms of the number of years that had elapsed between the transfer process and the impact assessment/documentation carried out by authors (we refer to this as the period of evaluation). For the purpose of our review we categorized case studies as either, short term (3 or less years) medium term (4-10 years) or long term (over 10 years) studies. Although our adapted SMS ranking does not explicitly address the period of evaluation as a scoring criteria, our data reveals a clear relationship between short-term impact assessments and low overall methodological quality. All short-term studies scored a SMS of 1, suggesting that 3 years is not enough time to fully understand the outcomes and impacts of IMT. Problems of casual attribution, discussed above, are also more pronounced in short-term impact assessments, especially when the transfer process is accompanied by the rehabilitation of physical infrastructure, which often produces immediate improvements to the technical efficiency of the system. The methodological quality of medium and long-term studies varies significantly. 31 medium and long-term studies were categorised as SMS 1 or 2 due to their low overall methodological quality. This is hardly surprising given that only 22 (out of 46) long and medium-term studies used time series analysis and actual drew upon long term data. Thus, period of evaluation is not a clear cut measure of methodological quality. Long and medium term studies do however, have the potential to contribute new and useful insights into our understanding of IMT/PIM impacts and if integrated with more systematic research methods, to permit conclusions about the conditions under which transfer programs could be expected to succeed or not. Taken together, our review and analysis of IMT case studies reveals that data on the impact and outcomes of management transfer is weak in terms of rigor and method. Having said that, case studies do include a range of perspectives on the impacts of management transfer and some results are independently verified. Ultimately, these studies are the only available sources of information on the impact of IMT/PIM and we rely solely on the information contained therein to construct our CSS and to evaluate and classify each of the projects as successful or failed interventions.

2. Definition of success: How can we evaluate the impact and outcomes of

IMT/PIM and differentiate the successful cases from not so successful ones? One of the main objectives of this review is to evaluate the overall impact and effectiveness of IMT/PIM interventions in Africa and Latin America. To do this, we first needed to define what constitutes a successful IMT/PIM intervention. In the literature (for example, FAO, 2007; Tang, 1992; INPIM website) IMT success is defined as multidimensional. We found that except for Tang (1992), who defined success in terms of only two indicators namely, rule conformance and good maintenance, most others, 14

including our case study authors, defined success based on a number of outcome and impact related indicators. On the whole, the performance criteria used by case study authors to evaluate success are quite diverse. Studies in Latin America emphasised financial viability and improved water supply while the most common performance indicators in African case studies were functional condition of infrastructure and popular awareness and participation. On average, CSS was calculated on the basis of 6 criteria and none of the case studies contained data on all 10 impact and outcome indicators. This stems from a lack of consensus on what constitutes successful transfer in the literature. As Mukherji et al. (2009) argue, definitions of success are often informed by authors disciplinary background. Social scientists for instance, define success in terms of popular, community participation, while water resource scientists would define success in terms of better water control of delivery schedules (Mukherji et al: 18). A single indicator, such as increased reliability of water supply or crop productivity may highlight particular aspects of an IMT intervention but taken alone does not allow us to comment on the overall impact of management transfer or to examine trade-offs between key performance measures. For example, changes in short-term productivity versus long-term reliability and adequacy of supply (Vermillion, 1997). For this reason our system of classification (CSS) measures success from several perspectives, drawing on ten financial, technical, social and agricultural outcome and impact indicators. Theoretically, successfully transferred systems should show a marked improvement in systems performance because users receive adequate and reliable supply of water at reasonable and affordable costs over a sufficiently long period of time enabling them to increase their crop production, productivity and incomes. This definition of success integrates technical, financial and agricultural outcomes over the long-term.

Figure X: Distribution of successful and failed IMT interventions in Africa and Latin America (one map) What are the chances that our method and success scores may misclassify the actual outcome of an IMT intervention? To ensure the robustness of our system of classification we compare our success scores with our evidence base. For this purpose we documented authors classifications of success or failure; when authors did not give an explicit opinion on the success or failure of the initiative we coded it is inconclusive. Table 2 compares our success ratings based on CSS, with those of the authors. We find that none of the cases where authors have declared a case to be a failure, turned out to be successful according to our success scores, showing, there is not a single example of a failed case, mistakenly coded as a successful one by us. Similarly, we find that only one case study which the author declared to be successful turned out to be a failed case in 15

our scoring system. There were 23 cases where the authors had not given any firm verdict on the outcome of IMT and which we classified as inconclusive. Of these, 66% turned out to be failed according to our CSS, which is not surprising given our way of calculating CSS involves multiple indicators, while case study authors often declare an intervention to be successful based on one or few indicators.
Table 2: Comparison of authors evaluation of success with our Composite Success Score
Evaluation according to CSS Evaluation according to case study authors Categories Successful cases Successful 15 Failed 1 Total 16

Failed cases

26

26

Inconclusive cases

16

23

Total

22 (34%)

43 (66%)

65 (100%)

It is important to note however, that the high level of consistency between our classifications of success and failure does not confirm the robustness of the evidence itself. Given that we did not have the resources to verify the accuracy of the data and conclusions drawn by case study authors via field research, our system of classification instead distinguishes between high and low quality studies. In doing so we acknowledge that many changes in scheme performance may have been misclassified (by case study authors) due to methodological shortcomings.

3. Review of IMT/PIM interventions: What are the conditions under which

successful or at least promising WUAs in public irrigation systems are found? Are those conditions replicable? A primary aim of this review is to gain insights into how effective IMT interventions work (or why they fail). To this end, we examined the relationship between a range of descriptive attributes and rates of successful management transfer. By correlating success and failure with individual scheme characteristics, we were not only able to 16

identify patterns of success but also, test some of the hypotheses put forward in the literature on the determinants of successful management transfer. However, due to the poor methodological quality of the majority of studies in the field, it is not possible to make definitive conclusions about what matters and why in successful IMT interventions. As a result some of the key arguments of the paper remain hypotheses rather than conclusions and must be verified with extensive and robust field research and analysis. However, a central argument of this paper that imposing blue-print, one-size fits all institutional models will not fix the complex and diverse management problems of irrigation systems remains a robust conclusion.

Degree of commercialisation: Success as a function of access to support services and

crop value A core objective of IMT was to make irrigation schemes financially self-sufficient however as argued by Veldwisch (2006), this objective has an implicit bias towards commercialisation. In order for farmers to absorb O&M expenses the costs of selfmanagement must be a small proportion of their income (Vermillion, 1997; Shah et al 2002). Put simply, irrigated agriculture must produce enough economic benefits to cover the economic and financial expense of farmer management. Often this requires commercialisation. To investigate the relationship (if any) between commercialisation and successful management transfer we systematically documented farmers access to credit/markets and crop value. Our results suggest that when output marketing systems are strong and well developed and farmers have access to credit the chances of successful IMT increase significantly in African schemes. According to our system of classification, of those African schemes with no or low access to input and output markets 92 % are failures. In contrast, high access to credit and support services did not appear to have a strong relationship with successful management transfer in Latin America. Rather, the highest rates of success in the region were found in schemes where the degree of commercialisation is varied (i.e. schemes served both commercial farms and emerging or subsistence farmers). Indeed, our sample suggests that intra-scheme inequity in terms of access to credit does not impede successful transfer in Latin America. The divergent result in Africa and Latin America on this front may be the result of biases in the literature. Of the African case studies which house both commercial and historically disadvantaged farmers (HDFs), 8 centred upon the perspectives of HDFs and as such there was greater emphasis on the legacies of historical inequity and their continued influence in scheme performance. In Latin American case studies there is less stress placed on the bifurcated structure of irrigated agriculture. Indeed, even though, there are clear class divisions in the agricultural sectors of Mexico and Colombia and these are often perpetuated in the structure and composition of WUAs, these perspectives are rarely taken up by case study authors in their assessments of IMT impacts. 17

The evidence from both Latin America and Africa also suggests that in most cases, cultivating high value crops does not greatly increase the probability of success. Rather in both regions, it appears that diversifying cropping systems (which include both high and low value crops) are most positively correlated with successful management transfer. Having said that, rates of failure in schemes which cultivate low value crops were consistently high: at 91 % in Africa and 75% in Latin America. Taken together, our evidence suggests that the relationship between commercialisation and successful management transfer is mixed. It is important to consider these results in context; in many small-holder African schemes the enabling environment for commercialisation is often not in place, this is in part due to poor support services, inaccessible and unaffordable credit, unreliable markets and poor marketing skills. In the White Nile Pump Schemes (Sudan), for example, repressive state regulation of cotton and wheat production (the dominant crops on the schemes) and poorly developed private markets are a critical factor underpinning the sub-optimal performance of these schemes (Samad et al. 1995). Similarly, in the Sepeteri and Itoikin schemes of Nigeria, farmers associated the high risks of irrigated agriculture with low credit availability and market access (Olubode-Awosola et al., 2005). Moreover, prior to transfer small-holder irrigated farming in Africa was a highly capitalintensive activity. Indeed, many small-holder schemes were only made financially viable as the result of large government subsidies. As a result, the expectation that farmers are capable of absorbing the costs of self-management (when these costs were a source of budget deficits for many governments) is paradoxical unless it is accompanied by a significant improvement in farmers livelihoods.

Capacity: Success as a function of pre-existing Farmers Organisations

In both Africa and Latin America, schemes that had farmers organizations prior to transfer appear to succeed more often that those without. However, in the literature the relationship between pre-existing organizations and successful management transfer is not clear cut; sometimes WUAs were built on existing organizations (Mexico, white commercial sector in South Africa, Dominican Republic), and sometimes they neglected them all together (Mali). Moreover, even when pre-existing organizations become the basis of WUAs, they sometimes reinforce pre-transfer power hierarchies and social inequality. In Mexico for example, the IMT program was developed on an already existing, strong organizational base: the ejidos and the organizations of private growers. However, more than ten years after the transfer process, ejido farmers continue to complain that they lack the financial and political muscle to check the influence of large landowners over the decision-making process (Wilder & Romero Lankao, 2006). In spite of the post-transfer politicization of WUAs, the results of our study suggests that pre-existing organizational strength, which we regard as a proxy for farmers 18

management capacity, impacts the likelihood of successful transfer. Moreover, we argue that the high rate of failure among studies which did not have any pre-existing organisations, 89%, reflects enduring capacity deficits especially in African small-holder schemes. Indeed, in many cases, farmers were entirely excluded from management decisions and often were dependant upon the central agency for farm management and support services. For example, in the Sepeteri scheme (Nigeria) and Hindustan Irrigation System (South Africa), the agency applied a strict system of management under which the farmers and very little freedom of choice or control in crop management, marketing, land tenure and other economically significant aspects of irrigated agriculture (Olubode-Awosola et al, 2005; Tren and Schur, 2000). Ultimately, farmers were responsible for little more than weeding and applying fertilizers to their plots. More broadly, the capacity deficit or dependency syndrome which is characteristic of some African case studies highlights important contradictions in IMT policy. The way in which the farmer participation concept has been applied in many locations often implies that WUAs are designed to replace the government agency in scheme management, at least at the tertiary level. However, in cases where farmers were previously excluded from management decisions and where local leadership and organisation have been crippled for decades, it is unrealistic to expect a culture of collective management to take hold (Mukherji et al. 2009; Shah, 2002). IMT interventions where there has been a long history of donor involvement are similarly paradoxical. In the Musengezi schemes in Zimbabwe for example, farmers viewed the turnover experiment as a game of finding the right savior to bail them out on operation and maintenance costs. Horticultural companies, international donors and politicians took turns in picking up the tab.7 As Bolding et al. (2003) suggest, farmers internalized the idea that both the state and donor are responsible for bankrolling cultivation and maintenance.

Stakes: success as a function of property rights and the importance of irrigated

agriculture IMT/PIM policy was built upon the assumption that as farmers have the largest stake in irrigation schemes, they have more incentive than a government agency to manage these systems efficiently and sustainably. While this line of reasoning is suggestive, it presupposes that farmers are a homogenous group with harmonious interests. The evidence from our review indicates that the stakes farmers have in scheme management, which we measured by the importance of irrigated agriculture to livelihoods and property rights, varies considerably across and within locations.
Farmers were not passive recipients of aid but rather exploited political competition in the area to settle their debts and keep the scheme operational. See: Bolding et al. (2003): 197.
7

19

Our results indicate that in Africa the greater the importance of irrigated agriculture in peoples livelihoods the greater the likelihood of success. According to our system of classification 63% of schemes which were cultivated and managed by full-time farmers succeded. This suggests that when farmers derive a substantial proportion of their livelihoods from irrigated agriculture it builds their stake in self management and increases the likelihood that they will commit time and resources to it. The 100 % rate of failure among schemes where irrigated agriculture is not central to livelihoods, also raises questions about the compatibility of IMT and part-time farming. In many smallholder schemes, farmers depend on a variety of sources to earn a livelihood, such as pensions, livestock and jobs in urban centres. In Strydkraal, Krokodilheuvel, Wonderboom and Veeplaats (South Africa), for example, it is common for men to work in mines, as farm labour for commercial farmers and in cities while women cultivate the plots. For a majority of households in these schemes, off farm income (pensions, labour, wages etc.) exceeds farm income.8 Farmers who derive their incomes from diverse sources have a reduced stake in managing the system. The high degree of failure among this group suggests this has important flow on effects in terms of the time and resources that are committed to cultivation and scheme management. Latin American case studies only offer fragmentary data on the importance of agriculture in the livelihoods and farmers (only 33% of studies had data on this indicator) and hence we could not use this information for analysis. Property rights (both land and water) also provide important security and incentives for farmers to invest in irrigation management. Insecure tenure, lack of crop choice, rigid rental markets and the inability to offer land as collateral for obtaining credit, limits the time, materials and cash farmers are willing to commit to long term development investments on their land (let alone in the scheme as a whole). Moreover, insecure land rights restrict the incentive/ability for uninterested farmers to sell out, and interested and capable ones to expand their holdings (Shah et al., 2002). For example, in the small-holder pump schemes of Saga, Kourani-Baria I and II (Niger) the precise ownership rights of irrigated lands remains unclear as scheme members cannot sell their land (even if they wish to) but may be evicted from their land (against their will) by the Water User Association (Abernethy et al, 2000). The issue of uncertain land tenure arrangements also assumes a gendered dimension in Veeplaats irrigation scheme (South Africa). Kamara et al (2002) note that well over 50% of farmers currently cultivating plots (mostly women) do not have the Permission to Occupy (PTO) registered in their name. Rather, titles are mostly in the names of men, creating disincentives for the actual farmers (women) on the scheme. In Latin America land tenure is not a major institutional issue, indeed all of the schemes in our sample had either high or mixed degrees of land rights and neither had a significant relationship with success rates in the region. As a result many of the
8

For detailed analysis see: A.B. Kamara et al. (2002): 820.

20

challenges posed by insecure land rights in Africa (documented above) do not appear to be relevant in the Latin American context. However, our results indicate that water rights do appear to impact the extent to which farmers are willing to invest in collective action for irrigation management (see Figure 17). Of the case studies which had clearly defined water rights, almost 60% were from Mexico, and all successful cases (in Latin America) with comprehensive rights were Mexican (thus our results on this front are skewed by a regional bias and may also reflect other factors unique to the Mexican case). Indeed, our results on a number of indicators are biased by the strong representation of Mexican schemes. As a result, it is difficult to tease out which factors matter most and why. We can only assume that several factors may have influenced these positive results. First, Mexicos IMT program was initially concentrated in the large, commercial areas of the North and Northwest where the concept of turnover had strong local support. Five (out of a possible six) cases in our study were drawn from these areas and thus our results from the country have a strong regional bias. Second, the IMT program had high level political commitment (from the Office of the President), and as a result, turnover was preceded by legislation (the national water law) which established water rights and the legal status of WUAs. Finally, the speed and success of IMT in Mexico can partly be explained by the high degree of social organization within the irrigation sector prior to transfer. As mentioned earlier, users were mainly organized through the ejido, production cooperatives, and farmer unions. WUAs were built upon these traditional organizational lines and thus harnessed existing social capital or patterns of cooperation. Most of these factors were highly correlated with successful transfer in Latin American schemes (see Table 3). Only 45% of African case studies had data on this indicator and thus comparative analysis on this front is not possible. Taken together our results suggest that when farmers are heavily invested in scheme management, either through property rights or because they derive their main source of livelihoods from the scheme, the likelihood of successful transfer is high.

Process: Success as a function of implementing agency and level of transfer

As argued by Shah et al. (2002), the overwhelming majority of the literature on IMT/PIM assumes that successful management transfer to user organizations is in principle viable provided the process is right (Vermillion 1996; Fredericksen and Vissia 1998; FAO, 2007). Drawing on our rich database we were able to test this hypothesis by analysing the relationship between the method and extent of devolution and rates of successful management transfer. Our results indicate that chances of success are significantly higher when NGOs or donors assist the implementation of IMT/PIM policies. This finding was consistent

21

across Africa and Latin America (and Asia).9 For instance, in the case of a pilot WUA within the Gezira Scheme (Sudan), farmers benefited from extensive donor and government funds and intensive capacity building by national consultants and international organisations such as, FAO and the World Bank. It is important to note that the areas chosen as pilot sites for IMT are by no means representative of irrigation farming communities in Africa as a whole. For example, farmers identified for the pilot project in Gezira had well defined land rights; were located in proximity to trainers and consultants (Sudan Gezira Board headquarters), and had above average levels of education. In addition the pilot site was identified based on high average production levels (prior to transfer, the average yield of the pilot site was 16% above the scheme average), adequate water supply, and absence of socio-economic constraints (Hussein, 2006). As part of the project, farmers were trained for 3 hours a week throughout the season on maintenance of minor canals, water management and to some extent in matters such as cropping systems, animal husbandry, marketing and accounts keeping. Training also stressed equity in distribution along minor canals. At the field level, over 900 training sessions were held through 12 Farmers Field Schools (FFS). FAO also established a Revolving Fund in the first season ($15,000) which was used to procure fertilizers and seeds. Abdelhadi et al. (2004), further attributes the success of the Gezira pilot project to farmers accumulated experience in technology transfer. Put simply, farmers in pilot areas had prior training through the auspices of another project (Integrated Pest Management Research and Training Centre (IPMRTC) and FAO had been training farmers in the project area through field schools since 1993). In contexts such as these, it is very likely that NGO assisted implementation produced favourable outcomes, especially vis-a-vis comparable schemes who did not benefit from similar long-term investments in capacity building. However, many factors that shaped the success of this particular case have their roots in pre-transfer processes and trends. Thus, similar results cannot be expected in other blocks of the same scheme (the majority of which do not share these traits) let alone in entirely different systems which are costly to run, suffer from technical problems, dilapidated infrastructure and poverty, have lower productivity levels and are unable to collect adequate irrigation fees from farmers to cover O&M. Moreover, because the pilot experience was so process intensive it difficult and costly to replicate on a mass scale. More broadly, NGO involvement in IMT/PIM implementation has been highly localized and rarely produces reform beyond a very local scale (Shashidharan 2000). Finally, in our sample the majority of the NGO assisted case studies were short-term impact assessments and therefore may not capture the actual management capacities of farmers or the sustainability of NGO-donor investments in capacity building etc.
9

See Mukherji et al, (2009).

22

In Latin America, there was also a 100 per cent success rate amongst IMT programs initiated by farmers. However, the four studies in question (Coello Irrigation District, Saldana Irrigation District, A.U.P.A Culiacancito WUA and Productores Agrcolas Asociados, A.C WUA) are exceptional cases in which existing farmer cooperatives were well organized and were motivated by a strong belief that they could administer schemes more efficiently than the government. Indeed in Coello and Saldana (Colombia) high de facto farmer participation in irrigation management existed prior to transfer and the schemes were already highly profitable. Both cases are from a relatively prosperous farming area, where agriculture and agro-business are the mainstream of the local economy. In short, these case studies share the double coincidence of need and capacity. The extent of devolution varies significantly across and within locations as do the roles of farmers and irrigation agencies in O&M. Our sample contains a mix of IMT models, in which either total or partial management authority is devolved at one of the following hydraulic levels: headworks, subsystem, main/branch or distributary. Our results show that the hydraulic level transferred does not have any bearing on success rates but partial O&M transfer is more likely to succeed that total O&M transfer. This is consistent with earlier findings that suggest continued agency involvement in O&M is sometimes necessary, especially in contexts where there is a significant capacity shortfall. On the whole however, it is difficult to make clear distinctions between farmer and agency management in many post-transfer systems. Rather, in cases of total O&M transfer, the agency is still involved in systems management at higher hydraulic levels (our sample, includes only one case in which management of headworks were transferred to farmers). According to Aw and Diemer (2005), many of the inefficiencies of farmer management at the distributary level can be traced to main system managers, who fail to deliver reliable water supplies in predictable schedules at the outlets. In such cases, power continues to be vested outside WUAs, technically in the main system and institutionally with the Irrigation Agency (Narain, 2008). So while one to one correlation indicates that the right process is an important factor determining success, it may not be sufficient to explain performance outcomes, especially in the African small-holder context. Our results suggest that the relationship between process and successful transfer is mediated by other factors. In the smallholder schemes of Africa, for example, irrigated farming faces an entire complex of constraints which prevents effective farmer management (many discussed above, such as market access, property rights and low incomes and productivity).10 Indeed, in many case studies, income per household remains too low to meet all subsistence requirements let alone generate the surplus needed for development (Shah et al. 2002). In situations such as these the method and extent of devolution do not address the true

Shah et al. refer to these constraints as downward ratchets or multiple constraints which disable poor people like movements down past a cog which are difficult or impossible to reverse, making poor people permanently poorer. (Chambers 1983: 115).
10

23

causes of sub-optimal performance. This requires support systems that extend beyond irrigation and which significantly enhance farmers livelihoods.

Technical Specifications: Success as a function of the size of system and type of

scheme A common assumption underpinning the literature on common pool resources (CPR) is that small sized systems with well defined boundaries are more amenable to cooperative management.11 This idea draws strength from the endurance of many traditional self-governed irrigation systems (FMIS) which are well bounded in terms of both physical infrastructure and scheme membership. However, our results indicate that irrigation systems which were constructed and formerly managed by the state do not conform to the hypothesis put forward in CPR literature. On the contrary, in our sample the likelihood of success does not vary much according to size, if anything smaller systems are slightly more likely to fail. According to our system of classification, small and medium-large systems in Africa had failure rates of 85% and 58% respectively. In particular, cooperative farmer management has failed to make a mark in smallholder farming contexts of Africa. This inconsistency stems from fundamental differences in the socio-technical conditions and organisational characteristics of WUAs vis--vis farmer organisation under FMIS. As Mukherji et al (2009) argue the farmer participation concept and culture of irrigation common to FMIS is not easily reproduced in government management systems. This is because there are significant differences of design and socio-technical complexity between government systems and FMIS. Indeed, in Africa many irrigation sites were designed based on politico-social rather than technical rationales (i.e. African irrigation schemes provided the means to realize food self sufficiency and/or a policy of segregated development, with much less attention paid to their commercial viability). In addition, small-holder farming in Africa under agency-management was highly mechanized. The ARDC in South Africa for example, used heavy equipment for ploughing, land preparation, spraying and harvesting (Shah et al. 2002). The transfer to farmer management has significantly increased the cost of hiring farm equipment which has reduced the profitability and increased the risks of small-holder farming. Thus, the poor design of the majority of smallholder schemes exposes a central contradiction in IMT policy: farmers are now being called upon to manage expensive and complex outfits that were never meant to be run by farmers.

11

Refer to: Agrawal (2001); Wade (1988); Ostrom (1990) and Baland and Platteau (1996).

24

The mismatch between IMT models and farmers managerial and financial capacities appears to have serious implications for rates and success and failure, especially in the African context. For instance, the high rate of failure among African pump schemes, 82%, in our sample is largely due to the high cost of energy (following the removal of government subsides) and pump maintenance or repair. In an analysis of four smallholder pump schemes in Niger, Shah et al. (2002) note that turned over pump schemes almost always leave farmers in the red as net income is 20-25 per cent of gross income and irrigation fees are as high as 15-20 per cent of gross income. Similarly, in Senegal Valleys small-scale schemes, rundown pumps coupled with high energy and maintenance costs have resulted in the suboptimal performance of pump schemes (Wester et al, 1995).12 In particular, higher input prices and the greater complexity of dry season irrigation after management transfer in the Doue Region, lead farmers to alter their cultivation patterns and focus on wet season crops. In short, low farm productivity and income prevents many African smallholders from taking advantage of the managerial advantages and better water control offered by high cost pump schemes. In Latin American pump schemes, high energy and maintenance costs have also led to sub-optimal results across key performance indicators. In the case of Roldanillo-UnionToro (RUT), for instance, the total cost of irrigation is relatively high due to two-stage pumping. The financial viability of the scheme, prior to transfer, was assured by large government subsidies which covered 60-80% of total costs. When the government eliminated subsidies to the agriculture sector and halted its expenditures for O&M in the scheme, farmers struggled to absorb the high pumping costs involved in irrigated farming. As Vermillion and Garcs-Restrepo note, over 6 years after transfer, total farmer payments declined in real terms by 22 per cent, from US$83 to US$65 per hectare by 1995. This was due to both a decline in fee-collection rate, from above 90 per cent before transfer to less than 70 per cent by 1995 and a reduction in the amount of water delivered per hectare (1998: 13). The decline of RUTs financial solvency appears to be attributable to its high cost and the sudden loss of substantial subsidies from the government. Samac irrigation district (Columbia) followed a similar trajectory post-transfer. It was also a high-cost system previously sustained by generous government subsidies, which witnessed substantial cuts in O&M expenditure and declines in irrigation fee collection rates following IMT. In contrast, low cost Rio Recio was able to sustain financial solvency for the first 5 years after transfer and dramatically increase fee collection rates. The O&M budget remained at virtually the same pre-transfer level for several years after transfer. Qualitative review of failed cases suggests that if a scheme is financially viable prior to transfer, it is likely to financially viable following transfer. If however, pre-transfer

Farmers had to travel 350km to Saint-Louis or 650km to Dakar to obtain spare parts needed for pump repair.
12

25

viability depends of significant government inputs, then farmers must be able to absorb the costs of irrigated farming by cross-subsidising irrigation costs.13

Table X: Frequency distribution of descriptive indicators

AFRICA DESCRIPTIVE INDICATOR Type of scheme Size of scheme Scheme complexity Age of the scheme Crop systems (high/low value) Crop Systems (Paddy/non Paddy) Population served by the scheme Physical water scarcity GDP/capita (PPP US$) Access to support services Importance of agriculture in people's livelihood Social cohesion Well defined water rights Land rights Average size of land holding Year of transfer/formation of WUA/initiation of PIM Highest hydraulic level transferred Amount of O&M authority transferred Implementer Rehabilitation before or during transfer Training and capacity building Farmer's organization created before the transfer Strong political will Supporting legislation before the transfer WUA Chairman are elected % of data available 98 97 85 70 88 97 65 80 100 83 87 55 45 83 68 82 70 98 100 68 58 73 52 47 75

LATIN AMERICA % of data available 100 100 94 67 89 94 44 100 100 72 33 44 94 89 33 89 72 100 100 72 61 100 83 83 94

This was done in Coello, Colombia by diversifying revenue sources after transfer, including equipment rental, interest from small loans, provision of fertilizers and agro-chemicals etc.). See: Garcs-Restrepo and Johnson III (1997).
13

26

27

Table X: Contextual factors correlated with successful IMT/PIM

TYPE OF SCHEME AFRICA Percentage successful Percentage Failed Total number of cases Diversion Pump and Lift Storage Mixed

SIZE OF SCHEME Small Medium and Large

SCHEME COMPLEXITY Simple Complex

CROP SYSTEMS Paddy NonPaddy High

CROP VALUE Low Mixed

PHYSICAL WATER SCARCITY High Low Mixed Percentage successful Percentage Failed Total number of cases

ACCESS TO SUPPORT SERVICES High Low Mixed

IMPORTANCE OF AGRICULTURE IN PEOPLE'S LIVELIHOOD High Low Mixed

WELL DEFINED WATER RIGHTS High Low High

LAND RIGHTS Low Mixed

28

AFRICA Percentage successful Percentage Failed Total number of cases

HIGHEST HYDRAULIC LEVEL TRANSFERRED Main/ Headworks Branch Subsystems Distributary

AMOUNT OF O&M AUTHORITY TRANSFERRED Partial Total

IMPLEMENTER

REHAB

TRAINING

Donor/NGO Assisted

Gov

Farmers

Yes

No

Yes

No

FARMER'S ORGANIZATION CREATED BEFORE THE TRANSFER No Yes

Percentage successful Percentage Failed Total number of cases

STRONG POLITICAL WILL Yes No

SUPPORTING LEGISLATION BEFORE THE TRANSFER Yes No

WUA CHAIRMAN ARE ELECTED

Yes

Low

29

Table 5: Logit estimates (40 observations from Africa); Dependant variable: SUCCESS
Logit estimates Log likelihood = -9.2293616 Number of obs LR chi2(7) Prob > chi2 Pseudo R2 = = = = 40 36.09 0.0000 0.6616

-----------------------------------------------------------------------------success_sc~s | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------scheme_type | .816976 1.934049 0.42 0.673 -2.97369 4.607642 scheme_size | -.4322562 1.705611 -0.25 0.800 -3.775192 2.910679 crop_value | .2825835 1.659349 0.17 0.865 -2.96968 3.534847 crop_type | 1.388716 1.904762 0.73 0.466 -2.344549 5.121981 acess_credit | 3.29745 1.813107 1.82* 0.069 -.256174 6.851074 landrights | 3.437275 1.529967 2.25* 0.025 .4385957 6.435954 Implementer | 4.084172 2.312326 1.77* 0.077 -.4479038 8.616248 _cons | -7.047084 3.61534 -1.95* 0.051 -14.13302 .038852 ------------------------------------------------------------------------------

From this model, we comment only on the variables that are significant. In this case, high access to credit/markets, well defined land rights and NGO-assisted implementation are significant at 10% levels. The model correctly predicts in 92% of cases. Following this, the marginal effects model was also estimated. This model shows how much the probability of success changes when various independent variables change from 0 to 1
Table 6: Marginal effects model
Marginal effects after logit y = Pr(success_scores) (predict) = .21811936 -----------------------------------------------------------------------------variable | dy/dx Std. Err. z P>|z| [ 95% C.I. ] X ---------+-------------------------------------------------------------------schem~pe*| .1423305 .34043 0.42 0.676 -.524898 .809559 .45 schem~ze*| -.0701373 .26589 -0.26 0.792 -.591271 .450996 .3 crop_v~e*| .0512344 .3079 0.17 0.868 -.552248 .654717 .1 crop_t~e*| .1767934 .20553 0.86 0.390 -.226038 .579625 .85 acess_~t*| .6516267 .24434 2.67* 0.008 .172736 1.13052 .275 landri~s*| .6210161 .22622 2.75* 0.006 .177633 1.0644 .4 implemen*| .647597 .22287 2.91* 0.004 .210779 1.08442 .5 ------------------------------------------------------------------------------

(*) dy/dx is for discrete change of dummy variable from 0 to 1 The marginal effects model shows that for a discrete change in type of scheme from 0 to 1, the probability of success increases by 0.14 units. For discrete changes in access to credit, landrights and implementing agency from 0 to 1 (i.e. from low to high; or from government to donor-NGO assisted) the probability of success increases by more than 0.62 units and this is significant at 1%. We also predicted the logs odds ratio, our results are given below.

30

Table 7: Logs odds ratio

Logit estimates Log likelihood = -9.2293616 Number of obs LR chi2(7) Prob > chi2 Pseudo R2 = = = = 40 36.09 0.0000 0.6616

-----------------------------------------------------------------------------success_sc~s | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------scheme_type | 2.263644 4.377999 0.42 0.673 .0511143 100.2475 scheme_size | .6490431 1.107015 -0.25 0.800 .0229327 18.36927 crop_value | 1.326553 2.201213 0.17 0.865 .0513197 34.28978 crop_type | 4.009698 7.63752 0.73 0.466 .0958905 167.6671 acess_credit | 27.04359 49.03291 1.82 0.069 .7740073 944.8949 landrights | 31.10209 47.58516 2.25 0.025 1.550528 623.8777 implementer | 59.39275 137.3354 1.77 0.077 .6389662 5520.634 ------------------------------------------------------------------------------

This model shows that for one unit increase (or in this case, a change from 0 to 1) in type of scheme, the odds of success increase by a factor of 2.26. For a change in implementer (from 0 to 1) that is from government to NGO assisted, the odds of success increase by a factor of 59.39. Lastly, we ran a simple test to see if the model is specified correctly. Through this test we can determine if any other significant variable has been left out of our analysis.
Table 8: Link test
Logit estimates Log likelihood = -9.2238174 Number of obs LR chi2(2) Prob > chi2 Pseudo R2 = = = = 40 36.10 0.0000 0.6618

-----------------------------------------------------------------------------success_sc~s | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------_hat | .9941093 .352405 2.82 0.005 .3034082 1.68481 _hatsq | -.0130492 .1252164 -0.10 0.917 -.2584688 .2323703 _cons | .0449714 .7504288 0.06 0.952 -1.425842 1.515785 ------------------------------------------------------------------------------

The results show that the model is correctly specified in the sense that no variable which can significantly influence the probabilities has been left out. This is because the _hat is significant but _hatsq is not significant. This is an indication of correct model specification. From this analysis three factors consistently appear to influence the probability of successful management transfer in Africa, these are high access to credit/markets, well defined land rights and NGO assisted implementation. The other variables do not appear to be significant. ***********************

31

Taken together, our analysis suggests that the results of management transfer, across and within Latin America and Africa, are mixed. In the African context, schemes which have high access to credit/markets, large stakes in irrigated agriculture, and which benefited from a NGO-Donor assisted implementation process are most amenable to successful management transfer. In Latin America, water rights, farmers organisational capacity and political commitment at the highest levels of government were all positively correlated with success. Thus, our analysis makes plain that what matters in Africa is not always relevant in Latin America and vice versa. It also suggests that some factors may be so critical to successful transfer (i.e. the ability to absorb the costs of management) that they impact the significance of others (i.e. type and size of scheme). Moreover, our results indicate that there is great variation in terms of the maturity and functionality of schemes earmarked for IMT/PIM interventions. Different phases of political, administrative and institutional development shape the maturity and functionality of WUAs, which in turn determines the modes and outcomes of PIM that are realistic at the present moment. It follows then that there is not a sole IMT/PIM model that can be replicated across locations and that context shapes the scope for change and as a result institutions that are effective in some places (Northern Mexico) cannot be transplanted or socially engineered in another environment (smallholder irrigation sectors of Africa) with the same outcome. In the case of the latter it may be neither necessary nor possible to create and sustain formal local organisations focused on irrigation management, such as WUAs. Rather as Srivastava and Brewer argue in societies characterised by diversified livelihoods and marginally profitable irrigated agriculture, it may not be worthwhile for farmers to invest heavily in management associations (1994:15). In such contexts, adapting existing multipurpose local organisations, like village based administrative units, Farmers Organisations or traditional water operators to manage irrigation may be the more sustainable and effective option.

Conclusions
The lessons that emerge from our review are multi-fold. First, although the literature on IMT/PIM is extensive, the evidence linking improvements in scheme performance to management transfer is far from compelling. Our study highlights important limitations of existing impact assessments, particularly their heavy reliance on post-intervention data to infer cause and effect. It also highlights the value of high quality studies (with both a strong comparison group and pre-and post-intervention data) and long-term impact assessments for making credible, generalizable conclusions about what works, where and why. We also contribute new insights into which socio-economic and technical mechanisms appear to be active in successful management transfer. In both qualitative and quantitative review of African case studies three factors were consistently correlated with successful IMT/PIM, these are credit and market access, property rights and NGO32

donor assisted implementation. In Latin America, governance issues such as organizational structure and water rights increased the likelihood of successful management transfer. Our research suggests that at least in the African context, the IMT process needs to be accompanied by greater interventions, relating to land reform, market access, and other support services which will help farmers increase their livelihoods and consequently their ability to absorb the costs and responsibilities of management. Our analysis also indicates that management transfer, in itself, rarely produces significant changes in the quality of financial management or technical efficiency. In the dynamic agrarian sectors of Africa and Latin America the transition to famer management was smooth, effortless and in many cases merely formalised the status quo.14 In the overwhelming majority of systems however, transfer takes place in a context of moderate dynamism and poverty. 15 Here, the causes of sub-optimal scheme performance rarely lie in the degree or intensity of farmer participation but rather, reflect design flaws, problems of market access and property rights, land fragmentation, the low profitability and productivity of irrigated agriculture, unstable water supply and unequal water distribution. Unless, the actual causes of poor performance are understood and addressed reform of the public irrigation sector will remain only a piecemeal affair, as it has been so far

Examples include Coello and Saldana (Colombia), A.U.P.A Culiacancito WUA and Productores Agrcolas Asociados, A.C WUA (Mexico) and the white, commercial farms of South Africa and Zambia) 15 Examples include most African small holder schemes , Maria La Baja (Columbia) Yaque del Norte Project (Dominican Republic)
14

33