Final evaluation of Ethiopia ACT Appeal (as part of overall ACT response to the Horn of Africa crises)

TERMS OF REFERENCE


Final evaluation of Ethiopia ACT Appeal
(as part of overall ACT response to the Horn of Africa crises)


1. BACKGROUND

The worst draught since decades, coupled with conflict in Somalia, Kenya and Ethiopia, has hit the Horn of Africa affecting more than 12.4 million people mainly in Somalia, Ethiopia and Kenya. The United Nations describes the situation in the region as the most severe food security emergency in the world today, with the Famine Early Warning Systems Network warning that the response has remained “inadequate Approximately 120,000 people have crossed the border from Somalia to Ethiopia as a result of draught and ongoing conflict who are currently living in the refugee camps in Dollo Odo.

ACT Alliance has responded to the Horn’s emergency situation with country appeals for Somali, Ethiopia and Kenya, each being implemented by a number of ACT members. Requesting members in the Ethiopia appeal are Lutheran World Federation – Department of World Service (LWF-DWS), Ethiopian Evangelical Church Mekane Yesus-Development and Social Services Commission (EECMY –DASSC), International Orthodox Christian Charities Inc./Ethiopian Orthodox Church- Development and Inter-Church Aid Commission (IOCC/EOC-DICAC), Christian Aid/Action for Development, and Norwegian Church Aid. DCA will provide technical assistance and will work in close collaboration with the ACT Ethiopia forum coordinator to establish a complaint handling mechanism in each district through HAP Complaint Response Mechanism (CRM).In addition, funds were requested to cover the costs for the ACT Ethiopia forum coordination and for a contribution to the external evaluation. The appeal which was issued on 14 July 2011 (and was revised on: 5 August 2011 (rev.1) and on 6 October 2011 (rev.2).


2. ACT EMERGENCY RESPONSE

The goal is of this ACT appeal is to saving lives and contribute to mitigating the risks of drought induced-hunger on targeted communities in nine Districts of Oromia Region of Ethiopia and Somalia refugees in Dollo Odo camps. Specifically, the objectives of Ethiopia ACT appeal are:

1. The lives and livelihoods of drought affected communities saved and protected through food provision, agricultural inputs provision, physical assets rehabilitation, build the resilience of drought affected communities through various rehabilitation interventions and the strengthening of drought coping capacity in the targeted areas, for 12 months.
2. The lives of refugees in Dollo Odo refugee camps (Bokolomayo, Kobe, and Melkadida) saved and protected through provision of food, health & nutrition, shelter materials, water, sanitation and hygiene, literacy program & standard and quality secondary education, psychosocial and skills training.

The implementation activities include provision of food and agricultural inputs, Watson, psycho-social support, nutrition and supplementary feeding, environmental protection, education and livestock and pastoralists support. Support is being provided to ensure sufficient capacity to deliver expected results, effective coordination of ACT Forum and technical support in quality and accountability in humanitarian assistance. The immediate humanitarian assistance will be provided upto December 2011 while some draught response activities will continue to July 2012.


3. DESCRIPTION OF THE ASSIGNMENT

3.1 Evaluation objectives

The main objective of this evaluation is to assess the performance, quality and overall impact of ACT humanitarian response in the Ethiopia’s draught-induced hunger emergency. The evaluation is intended to establish ACT’s commitment to accountability and promote learning of lessons from findings specific to Ethiopia country context and the Horn generally.

The evaluation shall be based on the following specific objectives;

i) Assess the achievement and results of Ethiopia ACT appeal in contributing to improving the living conditions of the most vulnerable groups impacted by the hunger and conflict
ii) Assess the performance of Ethiopia ACT appeal in the aspect of beneficiary accountability,. context of management, coordination, reporting, monitoring and evaluation, visibility, communication and dissemination of information and partnership with local actors.
iii) Identify lessons learnt and best practices which may benefit the community as well as ACT members in improving similar emergency response activities in the future


3.2 Key evaluation questions

1. Achievements
• To what degree were expected results achieved against set objectives and extent of quality?
• What promoted or undermined the achievement of results?
• Were there any unintended benefits and/or harms resulting from the Ethiopia ACT response?
• What impact has the implementation of Ethiopia ACT Appeal created in the lives of affected population (both female and male) and local institutions in relation to providing lon-term solutions to the Horn’s chronic crises?
• How was the performance of the Appeal’s activities in terms of OCED/DAC criteria for humanitarian action, the ALNAP quality proforma and the UNOCHA cluster-based objectives to assessment of results?

2. Quality & Accountability
• Compliance with the principles of the Code of Conduct for the International Red Cross and Red Crescent Movement and NGOs in Disaster Relief, the Sphere minimum standards and the HAP standards? Were accountability issues (including HAP standards) given due consideration? Were specific challenges encountered in meeting the principles or standards? What were reasons for these challenges?
• Beneficiary accountability: Did the affected population (both female and male) have adequate space for informed and adequate participation during the design and implementation stages?
• Impartiality: Did the design and implementation of the programme target the people who were most in need of support? Were procedures used for needs identification and targeting appropriate and transparent?
• Independence: were there any challenges for organizations to operate independently? How were these challenges dealt with?
• Were feedback procedures in place to raise questions or complaints & get answers about services provided and were these systems understood and used by beneficiary groups?
• Were there qquality assessments conducted and findings from them (including earlier assessments, studies, best practices and lessons learnt) utilized to inform programming?
• Flexibility: How did the programme adapt to the changing humanitarian conditions from earthquake to cholera? Were the strategies employed to respond to each and cross-emergency scenarios appropriate?
• Was the response timely?
• Was there appropriate plan in place to support conditions for long term recovery e.g. some intentional LRRD approach
• How cost-effective was the Ethiopia ACT humanitarian response? How have the funds been used in the most efficient way?
• Were the interventions relevant to the local context or were the intervention choices the most appropriate in meeting the desired results, given the context of Ethiopia?
• Did the appeal implementation develop monitoring and evaluation mechanisms to track specific indicators and used the M&E results to make improvements?
• Critical assumptions: did general security and humanitarian access, among others, constrain implementation?

3. ACT policies, coordination and implementing capacities
• How did Ethiopia ACT Forum optimize the value of ACT Alliance’s collaborative appeal system to create greater impact?
• How did the ACT CO and funding members support the Ethiopia ACT forum to mount an effective , fast and credible response, focus on predictability of funding, speed of disbursement of funds, amount of funding vis avis the level of crisis, technical support from ACT CO and make recommendation for ACT
• Was coordination enhanced, supported and managed in a way that contributed to the effective and timely delivery of emergency support to affected population? How did the partners of the forum (international & national) complemented and mutual reinforced each other?
• What measures did the ACT Forum or ACT organizations put in place to comply with the principles of the Code of Conduct for the International Red Cross and Red Crescent Movement and NGOs in Disaster Relief, the Sphere minimum standards and the HAP standards? Were accountability issues (including HAP standards) given due consideration?
• How well prepared were agencies prepared to respond to this slow onset disaster? How well did they help communities to prepare?Did implementing ACT members and partners have adequate technical expertise to match the complex humanitarian conditions of the horn? How did capacity affect or influenced implementation and impact?
• Has the recruitment and staffing (e.g. deployment speed and staff turn-over/retention) constrained or enhanced the response efforts and affected impacted the overall programmes?
• What impact did the Ethiopia response have on local capacities – local NGOs, churches and the state?
• Were the commitment and compliance to ACT policies including visibility in the ACT Horn’s response ensured?
• Were gender issues taken into account? Any specific challenges?


4. METHODOLOGY

Reference to Joint Monitoring Visit: one of the results of JMV is identifying the priority issues (of questions above) to focus on in the external evaluation.

The Horn of Africa’s crisis represents a complex and geographically spread emergency that is linked to the chronic drought in the region and the Somali armed conflict. These have led to cross-border movements of people with humanitarian needs spread across Somali, Kenya and Ethiopia.

The evaluation shall address sector and thematic issues, advocacy and capacity building, within the specific context of Ethiopia but in reference to the wider Horn’s humanitarian crises. The evaluation will also examine how the critical assumptions, notably the humanitarian access and general security, among others, have constrained implementation.

The evaluation shall utilize the following OECD/DAC evaluation criteria to establish the overall performance and results of ACT response in the Horn’s crises: Relevance, Appropriateness, Efficiency, Effectiveness, Impact, and Sustainability.

The consultant shall utilize both qualitative and quantitative research methods to collect and analyze pertinent data to establish the performance and impact of the Ethiopia’s ACT appeal and its contribution to overall ACT response to the Horn’s crises.

The Ethiopia’s ACT evaluation will ensure active participation of the ACT Forum, the individual requesting members, and the direct beneficiaries of the programs. The evaluation process is seen as an opportunity for ACT Forum and individual ACT members to reflect strategically on issues of the Horn’s crises, hence the emphasis for the use of participatory methodology. The Forum’s engagement with the evaluation process shall take the form of initial consultation meeting, consistent representation throughout field data collection and full participation in validation workshop soon after field work. However, participation and learning approaches recommended does not substitute or reduce the value of independent and credible evaluation.

It is emphasized that the evaluation shall take into consideration the humanitarian principles that ACT subscribe to, including; the HAP guidelines for Quality and Accountability, The Code of Conduct for International Red Cross and Red Crescent Movement and NGOs in Disaster Relief, and the SPHERE Standards. Further, to build in the good practice for conduct of evaluation in humanitarian emergencies, the evaluation shall be guided by both the ALNAP Quality Proforma and the UNOCHA cluster-based objectives to assessment of results.

The evaluation will utilize existing M&E data accumulated or analyzed during the response, the progress and coordination reports developed, and the report of Joint Monitoring Visit conducted by ACT members during the response.


4.1 Learning from evaluation

While the evaluation will provide opportunity for ACT members to draw lessons from own experiences, this will further support a more strategic reflection and choices on the horn’s crises (hunger & conflict). Therefore, ACT Alliance plans to conduct a joint horn evaluation debriefing and learning workshop which will bring together the evaluation consultants for individual ACT appeals in Kenya, Ethiopia and Somali. For the case of Ethiopia, the evaluation debriefing shall take place in Addis followed by a joint workshop in Nairobi after the evaluation for all three ACT country appeals are completed. It is envisaged that the joint learning workshop will take place in June 2012 even if the Ethiopia evaluation will need to be completed in February 2012.

Therefore, invitations for this evaluation debriefing will be extended to representatives of all implementing members and partners as well as the funding members for the Horn’s ACT appeals. It is expected that the Horn evaluation will also help to identify synergies and related coordination implications for ACT regional response as well as drawing some strategic choices for the chronic horn’s emergency situation The multi-layered learning opportunity (county-specific and regional) will require that lessons are learnt both at the field level (where possible) for each appeal and at a wider level of the horn crises.

4.2 Evaluation outputs
i) Inception report
ii) Evaluation report
iii) Country debriefing/learning workshop
iv) Joint horn evaluation debriefing/learning workshop

4.3 Intended key users of evaluation

- Implementing ACT members
- Implementing partners working with ACT members
- ACT funding members
- ACT Secretariat
- Other ACT members not part of the response
- UN and Other international organizations in the Horn






4.4 General Dates

Starting February 2012 Evaluation field work Ethiopia

Evaluation Debriefing in Adis

Final evaluation report

30-31 August 2012 Joint Horn (Ethiopia, Somali and Kenya ACT appeals) evaluation debriefing/learning workshop shall be conducted in Nairobi. The result of the learning workshop will be a multi-stakeholder action plan for implementing the evaluation recommendations.


5. CONSULTANTS’ QUALIFICATIONS

The evaluation of this multi-sector and multi-actor ACT response to Horn’s emergency will require a team of 3 well-balanced evaluators, 1 being international consultants (lead and co-consultants) and 2 being a local consultant who will add value of his/her better familiarity with the Horn’s region and Ethiopia context in particular. This team of evaluators will be responsible for the evaluation of the Somali Appeal, however, the lead consultant shall take the responsibility for the evaluation of Somali and Kenya ACT Appeals respectively. This is to ensure coherence in evaluation approach for total ACT appeals for the Horn’s response. . The evaluators must, collectively, have:
• Extensive experience in monitoring and evaluation of emergency projects, especially in draught-induced emergencies coupled with conflict.
• Strong facilitation and diplomatic skills
• The team leader should have at least 10 years of relevant experience, with a minimum of a Master degree
• A broad understanding of the issues concerning the humanitarian response with a sound knowledge of Humanitarian Principles, Red Cross Code of Conduct and Sphere standards and humanitarian accountability mechanisms.
• Strong ability to stimulate and guide participatory processes
• Excellent report writing skills
• Capable of representing what ACT stands for: an international church-based humanitarian joint action
• Be political, religious and cultural sensitive
• Proficiency in English is required and knowledge of Amharic language is desired
• Familiarity with the Horn’s general humanitarian conditions and country Ethiopia specific condition i local condition is considered an important asset


6. APPLICATION

The evaluation requires a team of 3 consultants including 1 lead consultant with clear international evaluation experience in humanitarian emergencies and 2 experienced local/regional consultants. A balance of International, regional and local expertise with male and female composition is preferred. The lead consultant may also take responsibility for the evaluations of Kenya & Somalia ACT appeals that shall follow. It is possible that these are done with different local consultants.

Expression of interest that describes the methodology and approach, the evaluation plan and budget, along with C.Vs for the consultant(s) should be emailed to the address below by 15 January 2012.

NB: For your own planning, please note that the evaluation in Ethiopia will be conducted within the month of February 2012. A separate process for subsequent evaluations in Kenya and Somali shall be agreed based on specific ToRs that will collectively cover the total ACT appeal for the Horn (Ethiopia, Kenya and Somali). Both individual consultants and consultant teams can apply. Applications from individual consultants are preferred.

Wilfred Kibwota
PM&E and Learning Officer
ACT Secretariat, Geneva, Switzerland
Email: Wilfred.Kibwota@actalliance.org