Terms of Reference

For the external evaluation of the following Action contre la Faims programme funded by UNICEF

Emergency intervention in Nutrition and Mental Health Earthquake in Port-au-Prince (Hati)

References of the contracts: D3SB PCA signed between ACF and UNICEF on January 28th, for Rponse urgence Sisme Port-au-Prince

TORs produced on the 28/06/2010

ACF Hati

June 2010

I. CONTRACTUAL DETAILS OF THE EVALUATION

I.1 Evaluation Dates

Expected as soon as possible possible timeframe: Expected Start Date: * End Date: Submission of Draft Report: Submission of Final Report: 30/07/2010 20/09/2010 03/09/2010 17/09/2010

NB. ACF-UK requires reception of the ToR two months before the start date for each evaluation.

I.2

Language of the Evaluation *

Language of the report: French (Reports submitted in any working language of ACF-IN - English, French, Spanish - will be accepted if the consultant also covers the cost of translation to the required language).

II. WORK PLAN AND TIMETABLE *

Activities Working Days Briefing 1 Travel to the mission 1 Briefing Mission, review of documents, and preparation of field work 2 Field Work plus capitalization work 12 Meeting with others actors in the Field & Collection of secondary 2 information in Port-au-Prince Data Analysis and preparation of the draft report 5 Debriefing in country on the basis of the draft report 1 Travel to HQ 1 Debriefing HQ 1 Draft report submission 3 HQ feedback (5 working days) Finalization of the evaluation report, capitalization and writing of article on 5 the basis of Field, HQ, and ACF-UK comments HQ approval of final report (2 working days) = Invoice payment TOTAL (working days, 6 days / week in the field) 34

III. BUDGET (GBPS)

Budget for the Evaluation: 16,816 Deadline for invoicing e.g. contract end: 20/09/2010 Per Diem: Per diem is included in the consultant daily rate and therefore will not be paid separately. Breakdown of Budget: Daily rates for consultancy are between 150 to 400 per day dependent on experience and evaluation of the ToR. Management fee is a minimum of 700 GBP but may be more depending on the scope of the evaluation and donor funding available. There is a discretionary 75.00 charge for advertising on www.charityjobs.co.uk

ToR Evaluation Baby Friendly tents programme in Port au Prince

ACF Hati

June 2010

Evaluation Budget
Daily rate (GBP) 380 units 34 total 12,920 1,640 721

Consultant name xx International Travel & Visa Local Transport and communication

Management Budget
Management fee Upgraded advertising charityjob.co.uk add 75 700 75

Total Budget
TOTAL 16,056 GBP

Exclusions: Accommodation, food and local transport will be provided by the mission at field level. Insurance and any materials to carry out the evaluation e.g. laptop, must be provided by the consultant (no exceptions will apply). In order to comply with donors requests, a quotation (or draft contract) must always be provided to be annexed to the final reporting information. ACF-UK will flag HQ in case any aspect of the budget cannot be respected.

IV. CONTRACTUAL DETAILS OF THE PROGRAMME

Please see Annex 1 for an overview of the current context and ACF presence in the region / country. Overview * The earthquake that struck Haiti in January 2010 caused significant mortality and damages, destroying private houses as well as public infrastructures, destabilizing the whole country. In this context, the degradation of living conditions as a result of the earthquake can lead to a deterioration of the nutritional situation, especially for young children. In such conditions, adequate diet and care for infants and young children are very difficult to maintain for material, organisational and psychosocial reasons. It is essential to provide nutritional and psychosocial support to pregnant, lactating women and children under 2 to prevent further degradation of the nutrition status of already vulnerable populations, child development delay, and psychological suffering as well as the spread of disease and mortality. In the current emergency context, exclusive breastfeeding during the first 6 months and continuous breastfeeding in association with complementary feeding until 2 years of age, are extremely important and must be protected, promoted and supported. Care practices in general and psychosocial activities are very important to reinforce in such period, when other issues of worries can disturb the link between the mother and the child. To this end, ACF has installed Baby Friendly Tents in order to create quiet and safe spaces for lactating women, encourage breastfeeding, as well as adequate feeding and care practices for infants and young children. This support is accompanied by psychosocial and psychological support for the mothers and their children. Further, ACF has opened two distribution points for Ready to Use Infant Formula, in collaboration with UNICEF and Save The Children. RUIF is exclusively for infants, less than
ToR Evaluation Baby Friendly tents programme in Port au Prince 3

ACF Hati

June 2010

one year old, who cannot be breastfed, as to avoid the use of powdered milk or other inappropriate foods, which can cause diarrhoeas or other infections due to poor sanitation conditions. This strategy was adopted by the Haiti Nutrition Cluster. The MSPP (Ministre de la Sant Publique et de la Population) and its nutrition partners have drafted national guidelines for the setting up of the so-called baby tents or other IYCF counselling points (Points de Conseil en Nutrition pour Bb) and the supply of RUIF.

IV.1

General Objective *

To prevent mortality among the earthquake affected population.

IV.2

Specific Objectives and Activities *

To provide psychological and/or nutritional support to young children, as well as pregnant and lactating women and people in distress. Results: Breastfeeding and adequate feeding practices for children less than one year old are promoted and supported. Activities: Coordination with partners and advocacy for breastfeeding promotion, as well as adequate use of infant formula with respect to the International Code on the Marketing of Breast Milk Substitutes Installation of baby friendly tents providing support to breastfeeding mothers and infants feeding (using RUIF for infants who cannot be breastfed), and psychological support for children and caretakers. Sensitization about breastfeeding and psychosocial issues in the baby friendly tents, and at gathering points of earthquake affected people (sensitization of key persons, diffusion of radio messages)

V. AIM OF THE EVALUATION

ACF-IN promotes and uses Evaluation as a tool to enhance operational performance as well as a way to increase ACF-IN accountability towards beneficiaries, partners and donors. The evaluation conducted must always comply with donor requirements and / or ACF evaluation policies.

V.1

Objective of the Evaluation *

To evaluate the ACFs emergency response to the earthquake in terms of breastfeeding support/promotion, infants adequate feeding, psychosocial and psychological support towards vulnerable groups of lactating and pregnant women, infant and children less than 2 years. This approach is innovative for ACF and the first component of this evaluation will consist in capitalization on this approach. At the end of the evaluation, the evaluator should provide an article for a publication on this approach.

ToR Evaluation Baby Friendly tents programme in Port au Prince

ACF Hati

June 2010

V.2

General terms and factors to consider

The evaluation should consider the objectives, results and indicators outlined in the logical framework of the project. It will be undertaken as an independent examination of the background, assessment, activities and means deployed by ACF, in the implementation of the project, in order to facilitate a learning workshop and evaluation report outlining the main findings and recommendations for future interventions and decision making. The evaluation should examine the standard and quality of goods and services generated by the project, in the opinion of the beneficiaries, ACF management and technical departments, technical governmental departments and other key stakeholders.

VI. EVALUATION CRITERIA

ACF-IN subscribes to the Development Assistance Committee (DAC) criteria for evaluation: Impact, Sustainability, Coherence, Coverage, Relevance / Appropriateness, Effectiveness and Efficiency. ACF-IN also promotes systematic analysis of the monitoring system and cross cutting issues (gender, HIV/AIDS etc). * Please see the attached ACF-IN Evaluation Policy & Guidelines for further details in respect of these criteria. Impact Impact looks at the wider effects of the action. Impact can be short or long-term, intended or unintended, positive or negative, macro (sector) or micro (household). What direct and indirect evidence is available that the action taken contributed to the improvement of the breastfeeding and complementary feeding practices and psychological wellbeing of young children, their caretakers and pregnant women? What direct and indirect evidence is available that the action taken contributed to assure a safe feeding for infants who cannot be breastfed? What direct and indirect evidence is available that the action taken contributed to the strengthening of resilience and coping mechanism of person psychologically affected by the earthquake? How much effective is the psychosocial and psychological support? How can it be improved? Have the stated project goal, specific objectives, and indicators as shown in the project logical framework been achieved, at least partly? What are the positive and negative, including unexpected, impacts? What is the additional value of ACF Baby Friendly Tent approach on the impact of the project? Does it, as aimed, increase the access and acceptance of the programme? Is the outcome satisfying? How can the Baby Friendly Tent approach be improved? Are the systems or indicators used to evaluate the impact of the work adequate? Is there any way to improve the impact and its measurement? What is the short-term impact of the program for the children and their families? Do mothers/ caretakers improved breastfeeding practices and care to their children? Do they continue playing, breastfeeding, etc? Did the RUIF supply compromise the breastfeeding practices for infants under one? Are beneficiaries of psychological support coping with the situation and putting in place adaptive mechanism? Do they are able to make adjustment in their daily life?

ToR Evaluation Baby Friendly tents programme in Port au Prince

ACF Hati

June 2010

Coverage Is there an appropriate knowledge of the needs level? And of the number of potential targeted persons that the project reaches in fine? What could be done to improve it? How were the different program component selected? Was there an appropriate or sufficient geographical coverage? Was the targeted population properly covered? Was there a fair identification and targeting of beneficiaries? Are the criteria for beneficiary identification in the different component of the programmes adequate and pertinent? Were the population, camp committees and stakeholders made aware of the activities and were they able to access them? Coherence What steps were taken by ACF to ensure the integration of the different project components, especially: how were the psychosocial and nutrition approaches integrated together? How is the project coordinated with other national/international agencies and bodies? How has ACF adapted its strategy considering the co-existence with many others NGO in ACF intervention zone and field? What is the coherence of ACF project with national policy (PCNB guidelines) throughout the different phases? How has ACF coordinate with the Nutrition and Mental Health Clusters? Is there coherence between donors and partners with ACF project response? What is the perception of the national staffs regarding the coherence of the project (between mental health and nutrition but also with the other ACF projects)?

Appropriateness/Relevance Relevance is concerned with assessing whether the action correctly identified problems and real needs and whether the projects funded by UNICEF were in line with local needs and priorities as well as with donor policy. Besides this general definition, these key questions have to be answered: Operational Strategy & context Does the current project design fit with the specific context of Port au Prince; more specifically are the risks and assumptions properly defined? How the design of such project could be improved? Is the implementation strategy adapted to the local community organisation, local customs and culture? Were beneficiaries and or local stakeholders involved in the problem and solution analysis? Are the different activities proposed to filled the problem has been agreed by the communities? Evaluate the degree of satisfaction with the program in the targeted communities. What is the perception of ACF from non beneficiaries about the program? Programme Objectives versus needs Is the problematic analyses filled with the context? Logic of intervention (refer to the LFA): Are the proposed objectives / results /activities relevant to the needs assessment analysis.

ToR Evaluation Baby Friendly tents programme in Port au Prince

ACF Hati

June 2010

Technical Approach Are the activities implementation approach was appropriate to reach the general objective? What other approaches can be developed? Are the proposed activities relevant to prevent inadequate practices concerning breastfeeding and weaning practices following trauma, earthquake, to avoid mother to interrupt breastfeeding? Are the proposed activities relevant to improve psychological wellbeing of beneficiaries? Are the psychological support and psychosocial activities protocols relevant to respond to the psychosocial needs? Are they the most appropriate? Are they adapted to the local constraints? Are the different topics proposed in the groups (playing with the child, massages) realistic, relevant, regarding the culture and the living conditions? What are the challenges to ensure the proper implementation of the program? Are there recommendations and practical tools to better implement the program?

Effectiveness Effectiveness measures the extent to which the activities funded under the action achieve their purpose, or whether this can be expected to happen on the basis of the results. Implicit within the effectiveness criterion is timeliness. Effectiveness should indicate the real difference made in practice by the activities funded, how timely the intervention is; equally how far means were used to their maximum effect, how far the intended beneficiaries really benefited from the products or services it made available. Similarly, issues of sourcing and preparedness may be addressed. Are the delivered services to the beneficiaries in accordance with the planned ones (results and indicators)? Compare actual schedules and action plans. Were the different adaptations proposed throughout the programme decided on time? Were these adaptations effectively implemented? What have been the effects? How can ACF improve the performance of project activities? Is the level of the staff hired sufficient for the program and activities proposed? Are trainings level proposed to them sufficient and adapted?

Efficiency Efficiency measures how well the various activities transformed the available resources into the intended results (outputs), maximising quality, quantity and timeliness. This links with the question were things done in the best way possible? and thereby also addresses the concept value-for-money that is whether similar results could have been achieved more by other means at lower cost in the same time. General efficiency: Are the tools adapted and efficient? Do the teams understand them? Are the means relevant in regard to the activities proposed? Is the collaboration with the camp committees efficient? How to improve it? Cost efficiency: Was there any way to reduce the cost per beneficiary but keeping the same level of outcomes? E.g.: Is the number of staff working in the programme justified for the number of beneficiaries?

ToR Evaluation Baby Friendly tents programme in Port au Prince

ACF Hati

June 2010

Is the budget used appropriately / as intended/ in accordance with original narrative and budget? Could the cost of operations be deemed efficient and line with expected outputs? Is there any overlapping on human resources in the Nutrition / Mental health department? How to better optimize the use of resources?

Sustainability Sustainability is concerned with measuring whether an activity is likely to continue after donor funding has been withdrawn and also whether its longer-term impact on the wider development process can also be sustained at the level of the sector, region or country. In the case of strictly humanitarian actions, connectedness might replace the concept of sustainability. It is defined as the extent to which activities of a short-term emergency nature, which are not in principle supposed to be sustainable, are carried out in a context that takes longer-term and interconnected problems into account. An analysis of sustainability will focus on the aspects below. Their relative importance will depend on the nature of the project. Beside this general definition, these keys questions have to be taken into account: Was the assistance provided in a way that took into account the evolution of the emergency context? Did a rehabilitation or exit strategy exist in any form; was this utilized and adhered to? What could be an adequate adaptation of the project in the mid-term? How is the team aware and prepared for working on it? How successful were the activities to strengthen the capacity of communities and individuals? Monitoring Which tools are developed to monitor the progress and impact of the project? Are they implemented effectively? How can they be improved? Were appropriate and relevant indicators developed for this? How is the monitoring process formalized? Is it adapted? Were the distribution and information flow, data collection and analyse was satisfactory among the team? How could it be improved? Is the information delivered in time (monitoring data / financial follow up) in order to facilitate decision-making? If not what are the constraints and how could they be released? How have lessons learned from monitoring / evaluation been incorporated into the program, and shared with partners? Was there a mechanism which dealt with complaints / dissatisfaction in relation to the program? Are outcomes used to improve the quality of the programme? How can the information on psychological and psychosocial problematic being more identified, highlighted and shared? Gender Include an analysis of the integration and participation of men and women in the project. Is the program team balanced in term of men and women? Was gender balance possible in such Program dealing with women?
ToR Evaluation Baby Friendly tents programme in Port au Prince 8

ACF Hati

June 2010

VII. METHODOLOGY OF THE EVALUATION

VII.1 Briefing
Prior to the evaluation, a briefing will take place in the ACF office in Paris (headquarters managing the Hati mission). Another briefing will take place upon the arrival on the mission, with the Country Director for general matters and security rules, and with the Technical coordinators for an introduction to the programme.

VII.2 Field activities

Data collection by three means: Direct information: Interviews with beneficiaries - Visit to project sites and to the facilities provided to the beneficiaries Indirect information: Interviews with local representatives; interviews with project staff expatriate and national staff); meeting with local authorities, groups of beneficiaries, humanitarian agencies, donor representatives and other stakeholders. > For indirect data collection, standard and participatory evaluation methods are expected to be used (HH interviews and FGDs with beneficiaries, non-beneficiaries, key informants health workers, teachers and leaders) Secondary information analysis e.g. analysis of project monitoring data or of any other relevant statistical data.

VII.3 Report
The report should be presented in the format set out in Annex 3. The draft report must be submitted no later than 8 days before the end date agreed by the contract. The final report will be submitted no later than the end date of the consultancy contract. A maximum of 5 days is allowed between reception of the draft report and completion of the final report including all feedback. The report can only have a maximum of 2 versions (Draft report 1 Draft report 2 (This version includes all the feedback - This version should be the final report but is sent for approval to HQ) and Final report (HQ approval). The evaluation report shall have a maximum length of 50 pages including the Executive Summary at the beginning of the document, Findings, Conclusions and Recommendations. The report should be presented in draft form for comment, before the final report is completed. Relevant comments from the Mission and ACF HQ debriefings should be incorporated in the final report. The final report will be submitted in an electronic version to the given reference staff supporting the programme evaluation (as agreed at the Briefing), including all annexes, together with a hard copy. Annexes to the report will be accepted in the working language of the country and programme subject to the evaluation.

ToR Evaluation Baby Friendly tents programme in Port au Prince

ACF Hati

June 2010

VII.4 Debriefing / Learning Workshop

The evaluator should facilitate a learning workshop with the following objectives: To present the draft findings of the review to the Mission and other stakeholders To gather feedback on the findings and build consensus on recommendations To develop action-oriented workshop statements on lessons learned and proposed improvements for the future.

VII.5 Debriefing with ACF HQ

The evaluator should provide a debriefing through a presentation to the ACF HQ in Paris on the main findings, conclusions and recommendations of the evaluation, and relevant comments should be incorporated in the final report.

VIII. EXPECTED OUTCOME OF THE EVALUATION *

The result of this evaluation should be presented in a written report that can lead to a publication related to it, and through several oral presentations, One on the mission, including facilitation of a learning workshop One at the headquarters An article of 3 pages for Field Exchange on ACF experience in Haiti

IX. PROFILE OF THE EVALUATOR

Significant knowledge in care practices, mental health and infant and young children feeding [Significant] field experience in the evaluation of humanitarian / development projects Relevant degree / equivalent experience related to the evaluation to be undertaken Significant experience in coordination, design, implementation, monitoring and evaluation of programmes Ability to write clear and useful reports (may be required to produce examples of previous work) Fluent in French and English Understanding of donor requirements Ability to manage the available time and resources and to work to tight deadlines Independence from the parties involved Available for the whole time of the evaluation

Conditions apply 40% of fees will be paid after reception of the draft report, 60% will be paid after validation of the final report by ACF-UK Travel to mission and to base and field will be arranged by ACF Accommodation (hotel for briefing / in-country), food, will be at the charge of the evaluator Insurance costs will not be covered and the evaluator shall manage their own insurance and provide details of this cover to ACF-UK before departure Any materials to carry out the evaluation e.g. laptop, must be provided by the consultant

ToR Evaluation Baby Friendly tents programme in Port au Prince

10

ACF Hati

June 2010

X. RIGHTS
The ownership of the draft and final documentation belong to the agency and the funding donor exclusively. The document, or publication related to it, will not be shared with anybody except ACF before the delivery by ACF of the final document to the donor. ACF is to be the main addressee of the evaluation and its results might impact on both operational and technical strategies. This being said, ACF is likely to share the results of the evaluation with the following groups: Donor(s) Governmental partners Various co-ordination bodies Intellectual Property Rights All documentation related to the Assignment (whether or not in the course of your duties) shall remain the sole and exclusive property of the Charity

XI. LIST OF ATTACHED DOCUMENTS

1. ACF-IN Evaluation Policy & Guidelines 2. ACF-UK Welcome Pack for Consultants

denominates information for the advertisement (consultant recruitment)

ToR Evaluation Baby Friendly tents programme in Port au Prince

11

ACF Hati

June 2010

ANNEX 1
BACKGROUND *
ACF in Haiti
ACF has been working in Haiti since 1985 and has developed a long-term project strategy in the Water, Sanitation, Hygiene (WASH), Food Security and Nutrition sectors based on capacity building of local authorities. Furthermore, ACF carries out disaster risk reduction activities. ACF implements development projects in Port-au-Prince, Gonaives and Port-dePaix. In addition, the organisation maintains an emergency response capacity through the pre-positioning of contingency stocks in both locations and a human resource network of trained national staff readily available throughout the country.

Current context of intervention

An earthquake measuring 7 on the Richter scale shook Haiti on January 12th 2010 at 16:53. The earthquake epicentre has been located 6 miles deep and 10 miles away from the capital. Several subsequent quakes have followed, some with a magnitude of more than 5.5 The earthquake caused many damages and an extremely important number of victims (more than 220,000 deaths, 300 to 400,000 of injured and 1,200,000 of IDPs). It is estimated that almost the half of Port au Prince has been destroyed and the even the cities Gressier, Logne, Grand Gove and Jacmel for about the 50 to 90%. These damages affected the individual houses but the public infrastructures as well. The Direction of Civilian Protection was very affected and this limited his capacity of response in the immediate assistance to the victims and the organization of reconstruction. The National Direction of Water and Sanitation was relatively saved and this permitted a rapid response by the Haitian authorities. An extensive international operation is now under way in Haiti, with up to 500 agencies supporting the Haitian Government to provide humanitarian relief. The loss of many members of government and the national institutions can be a risk of political instability. For this reason, elections initially foreseen for March 2010, has been put off to end of the year. The weakness of central administration could cause regional claims and rivalries against the central government. About 570,000 (22% of the population) left Port-au-Prince by 31 January. The three departments that received the largest absolute numbers of displaced persons were Sud, Ouest (the areas outside of the Port-au-Prince metropolitan area) and Artibonite. The 2010 hurricane season formally started on 01 June. Predicted to be severe, this heralds a potentially new crisis. While contingency plans are under way, including dedicated 24-hour humanitarian rapid-response teams in case of rain or hurricane related incidents in spontaneous sites, the danger posed by hurricanes to the already vulnerable populations left homeless by the quake is considerable. The loss of close relatives and friends in terrifying conditions, people/family members trapped under the rubble, serious injury and loss of all material possessions are among the main causes for the psychological distress people suffer from. After five months, people are still extremely affected by the events a large number of earthquake survivors continue experiencing psychological distress. A lack of specialized psycho-socio services and support for vulnerable groups has been identified. In additions to this their present life conditions (deprivations, loss, unemployment, family links weak or broken, basic need not covered, insecure environment, etc.) have a strong influence on their capacity to put in place adaptive behaviours. Previous experiences of earthquakes have highlighted the psychosocial needs of the population, the necessity to strengthen the resilience and to support the adequate coping mechanisms. This type of support helps the communities to cope with to the situation in the best possible way (in terms of solidarity, self
ToR Evaluation Baby Friendly tents programme in Port au Prince 12

ACF Hati

June 2010

support) and to limit potential negative side-effects (persons unable to carry out their daily tasks and/or to go back to work, to take care for the most vulnerable within the population infants, young children, elderly, etc.). ACF conducted a nutritional survey in Port au Prince in March 2009 that shows that the prevalence of global and severe acute malnutrition remains relatively low and well below the 10% alert level defined by the WHO. The survey also demonstrates that exclusive breastfeeding was not practiced for nearly three quarters of children in this age group. This confirms the findings of the national survey EMMUS IV. A psycho-social survey conducted in November 2008 by ACF also showed that women stopped breastfeeding as soon as they experienced strong emotion, or fell ill and that they did not begin breastfeeding again once it had been stopped for several days .The children themselves may also have suffered intense fear or traumatism which may manifest itself in eating difficulties. This was observed following the cyclones that hit Gonaives in 2008.

ToR Evaluation Baby Friendly tents programme in Port au Prince

13

ACF Hati

June 2010

ANNEX 2
GUIDELINES TO THE EVALUATION CRITERIA Please amend the criteria accordingly if the evaluation is to look more deeply into some of the criteria than others.
GUIDELINES TO THE EVALUATION CRITERIA

Please amend the criteria accordingly if the evaluation is to look more deeply into some of the criteria than others. Impact Impact looks at the wider effects of the action. Impact can be short or long-term, intended or unintended, positive or negative, macro (sector) or micro (household). What direct and indirect evidence is available that the action taken contributed to the improvement of the food security and livelihood of the population? Have the stated project goal, specific objectives, and indicators as shown in the project logical framework been achieved, at least partly? What are the positive and negative, including unexpected, impacts? What is the additional value of ACF approach on the impact of the project? Does it, as aimed, increase the access and acceptance of the programme? Is the outcome satisfying? Are the systems or indicators used to evaluate the impact of the work adequate? Is there any way to improve the impact and its measurement? What is the long-term impact of the program for the individual (households level)? Coverage Is there an appropriate knowledge of the needs level? What could be done to improve it? How were the different program component selected? Was there an appropriate or sufficient geographical coverage? Was the targeted population properly covered? Was there a fair identification and targeting of beneficiaries? Are the criteria for beneficiary identification in the different component of the programmes adequate and pertinent? Both socio economics as well as age and gender should be considered. Were the population and stakeholders made aware of the activities and were they able to access them? Coherence What steps were taken by ACF to ensure the integration of the different project components? How is the project coordinated with other national/international agencies and bodies? How has ACF adapted its strategy considering the arriving of others NGO in ACF intervention zone and field? What is the coherence of ACF project with national policy throughout the different phases? Is there coherence between donors and partners with ACF project response?

ToR Evaluation Baby Friendly tents programme in Port au Prince

14

ACF Hati

June 2010

Appropriateness/Relevance Relevance is concerned with assessing whether the action correctly identified problems and real needs and whether the projects funded under the Global Plan/Operation were in line with local needs and priorities as well as with donor policy. Besides this general definition, these key questions have to be answered: Operational Strategy & context Does the current project design fit with the specific context of Vahun; more specifically are the risks and assumptions properly defined? Is the implementation strategy adapted to the local community organisation, local customs and culture? Were beneficiaries and or local stakeholders involved in the problem and solution analysis? ACF has work with the community for the ranking of the need. Did the population understand well importance of the ranking? Did ACF explain to the population their participation/involvement for the success of the program? According to that what was their motivation? Are the different activities proposed to filled the problem has been agreed by the communities? Evaluate the degree of satisfaction with the program in the targeted communities. Was the partnership with Cooperative relevant? Was it an appropriate way to ensure sustainability of the activities implemented by ACF? What is the perception of ACF from non beneficiaries about the program? Programme Objectives versus needs Is the problematic analyses filled with the context? Logic of intervention (refer to the LFA): Are the proposed objectives / results /activities relevant to the needs assessment analysis. Technical Approach Are the activities implementation approach was appropriate to reach the general objective? What other approaches can be developed? Are the proposed activities are relevant to improve the food security and livelihood of the population? What are the challenges to ensure the proper implementation of the program? Are there recommendations and practical tools to better implement the program?

Effectiveness Effectiveness measures the extent to which the activities funded under the action achieve their purpose, or whether this can be expected to happen on the basis of the results. Implicit within the effectiveness criterion is timeliness. Effectiveness should indicate the real difference made in practice by the activities funded, how timely the intervention is; equally how far means were used to their maximum effect, how far the intended beneficiaries really benefited from the products or services it made available. Similarly, issues of sourcing and preparedness may be addressed. Are the delivered services to the beneficiaries in accordance with the planned ones (results and indicators)? Compare actual schedules and action plans. Are the planned action plans in accordance with realized ones, and if not why? Are these deviations justified? How this could be improved for the future intervention? Were the different adaptations proposed throughout the programme decided on time? Were these adaptations effectively implemented? What has been the effects?
ToR Evaluation Baby Friendly tents programme in Port au Prince 15

ACF Hati

June 2010

How can ACF improve the performance of project activities (e.g. capacity building of national staff and counterparts, reduction in number of expatriates.)

Efficiency Efficiency measures how well the various activities transformed the available resources into the intended results (outputs), maximising quality, quantity and timeliness. This links with the question were things done in the best way possible? and thereby also addresses the concept value-for-money that is whether similar results could have been achieved more by other means at lower cost in the same time. General efficiency: Are the tools adapted and efficient? Do the teams understand them? Are the means relevant in regard to the activities proposed? Is the partnership with the Guma Mende farmers Cooperative society efficient? How to improve it? Cost efficiency: Was there any way to reduce the cost per beneficiary but keeping the same level of outcomes? E.g.: Is the number of staff working in the programme justified for the number of beneficiaries? Is there any grey area in the reported and effective use of the food security supply dedicated to the beneficiary? What systems are in place to ensure inputs provided are of the highest possible quality and are acceptable to beneficiaries? Is the budget used appropriately / as intended/ in accordance with original narrative and budget. Could the cost of operations be deemed efficient and line with expected outputs? Is there any overlapping on human resources in the food security department? How to better optimize the use of resources? What systems of financial and logistical monitoring / control were in place in relation to standard procedures?

Sustainability Sustainability is concerned with measuring whether an activity is likely to continue after donor funding has been withdrawn and also whether its longer-term impact on the wider development process can also be sustained at the level of the sector, region or country. In the case of strictly humanitarian actions, connectedness might replace the concept of sustainability. It is defined as the extent to which activities of a short-term emergency nature, which are not in principle supposed to be sustainable, are carried out in a context that takes longer-term and interconnected problems into account. An analysis of sustainability will focus on the aspects below. Their relative importance will depend on the nature of the project. Beside this general definition, these keys questions have to be taken into account: Was the assistance provided in a way that took into account the longer-term context? Did a rehabilitation or exit strategy exist in any form; was this utilized and adhered to?

ToR Evaluation Baby Friendly tents programme in Port au Prince

16

ACF Hati

June 2010

Does the team work out an efficient exit strategy in which local authorities and local actors are identified appropriately? Is this exit strategy process on going effectively? How is it integrated into the programme? How successful were the activities to strengthen the capacity of local agencies, organizations and individuals? How could the collaboration with the Guma Mende farmers cooperative society been reinforced?

Monitoring Which tools are developed to monitor the progress and impact of the project? Are they implemented effectively? How can they be improved? Were appropriate and relevant indicators developed for this? How is the monitoring process formalized? Is it adapted? Is the information delivered in time (monitoring data / financial follow up) in order to facilitate decision-making? If not what are the constraints and how could they be released? How have lessons learned from monitoring / evaluation been incorporated into the program, and shared with partners? Was there a mechanism which dealt with complaints / dissatisfaction in relation to the program? Are outcomes used to improve the quality of the programme? Gender Include an analysis of the integration and participation of men and women in the project. Is gender considered in ACFs need assessment, and in the implementation of the program? Did the program include special components for women and if so, were these systematically monitored? Is the program team balanced in term of men and women?

ToR Evaluation Baby Friendly tents programme in Port au Prince

17

ACF Hati

June 2010

ANNEX 3
REPORT FORMAT
The report shall follow the below format. Cover page Title of the evaluation report: Date of the evaluation; Name of the consultant; Indication that the report has been financed by ____________; the comments contained herein reflect the opinions of the Evaluator only Table of contents Executive Summary A tightly-drafted, to-the-point and free-standing Executive Summary is an essential component. It should be short, no more than two or three pages. It should focus on the key purpose or issues of the evaluation, outline the main points of the analysis, and clearly indicate the main conclusions, lessons learned and specific recommendations. Main body of the report The main body of the report shall elaborate the points listed in the Executive Summary. It will include references to the methodology used for the evaluation and the context of the action. In particular, for each key conclusion there should be a corresponding recommendation. Recommendations should be as realistic, operational and pragmatic as possible; that is, they should take careful account of the circumstances currently prevailing in the context of the action, and of the resources available to implement it both locally and in the Commission Annexes: Listed and correctly numbered. Format for the main body of the report is: 1. Background information Target Area Main objectives of the project Development of the project Describe briefly the development of the projects activities, including the contextual evolution if necessary. Methodology What evaluation method was chosen and why? What are the main constraints (staff, political, security, access, rainy season etc.)? Include here a reference to an appendix such as questionnaire samples of the methods used. Who, when, why and where was interviewed people or sites selected. Findings and discussion Summarise findings under headings use the criteria headings 3.1 Impact 3.2 Coverage 3.3 Coherence 3.4 Relevance / appropriateness 3.5 Effectiveness 3.6 Efficiency
18

2.

3.

ToR Evaluation Baby Friendly tents programme in Port au Prince

ACF Hati

June 2010

4.

3.7 Sustainability 3.8 Monitoring 3.9 Cross-Cutting Issues Wherever possible, use maps, tables, diagrams, and interpretation of qualitative findings Include examples of what people actually say in the interviews Briefly describe the methods used to analyse the information (statistics, PCA, etc.)

Conclusions Summarise the main conclusions for each of the sections outlined above, and the main lessons learned and more generally for the typology of this response Recommendations What are the main recommendations to be considered for future interventions according to the evaluation? Where appropriate, make recommendations which relate to each of the sections highlighted above.

5.

Annexes Annexes to the report should include, The Evaluation budget Example questionnaires / interview sheets Feedback / outputs from the Debriefings / Workshop

ToR Evaluation Baby Friendly tents programme in Port au Prince

19