A team of four consultants will be required to undertake the work in the 6 countries that have agreed to take part in this exercise. This is, therefore, an invitation to institutions with adequate expertise and capacities in evaluation, particularly of capacity development initiatives in EPR and DRR, to submit applications to conduct this evaluation study from September to November 2011.
Background:
The Eastern and Southern Africa region is becoming increasingly vulnerable to disasters triggered by both natural and man-made hazards: storms, drought, floods, cyclones, environmental degradation, diseases such as HIV and AIDS, cholera, malaria and tuberculosis, as well as conflict, political instability, poverty, food and livelihood insecurity.
In 2009, as part of the Education in Emergencies and Post-crisis Transition Programme (EEPCT) funded by the government of the Netherlands and the European Commission, UNICEF ESARO launched a strategy for national capacity development in partnership with Save the Children under the aegis of the IASC Education Cluster. The objective was to build and strengthen sustainable national emergency preparedness and response (EPR) capacity in the education sector in ESAR holistically and strategically, by supporting national authorities at all levels.
Since May 2009, over 1,700 people have participated in capacity development workshops at regional, national and sub-national levels. On average, over 75% of participants were from national government partners, including MoE, Disaster Management bodies and other sectors, particularly protection, health and WASH. The remaining participants included representatives of local and international NGOs including the Red Cross, community-based organisations, and UN agencies.
Country teams comprising MoE, UNICEF and Save the Children staff from all 20 ESAR countries who participated in the Regional Training of Trainers (ToT) workshops in April/May 2009 are currently rolling-out the initiative at national, provincial and district levels within their countries (see table in Annex to review activities undertaken in the various countries).
The Education in Emergencies and Post-crisis Transition Programme (EEPCT) four goals are:
•Designated Goal 1 – Improved quality of education response in emergencies and post-crisis transition countries.
•Designated Goal 2 – Increased resilience of education sector service delivery in chronic crises, arrested development and deteriorating contexts.
•Designated Goal 3 – Increased education sector contribution to better Prediction, Prevention and Preparedness for emergencies due to natural disasters and conflict.
•Designated Goal 4 – Evidence-based policies, efficient operational strategies and fit-for-purpose financing instruments for education in emergencies and post-crisis situations.
The proposed study is expected to analyze how the capacity development intervention has contributed to Goal 3 and will contribute to building a solid evidence base upon which to guide future capacity development and institutionalization efforts in accordance with Goal 4 . In terms of a results hierarchy Goals 3 and 4 are seen as contributing to Goals 1 and 2.
This study is expected to be a key advocacy and fund-raising tool at a time when donors are increasingly interested in ensuring more stability in countries, not least in fragile states, linked to growing commitments to supporting climate change mitigation strategies and peace and conflict resolution.
The proposed study is in line with UNICEF global priorities and has been endorsed by the UNICEF EEPCT Manager. The study TORs have also been shared with the ESARO BEGE, M&E and Emergency Sections and with Save the Children as a key partner for their comments and inputs.
The proposed study also contributes to the implementation of the Education Cluster Strategic Plan 2011-2013 and Annual Work Plan 2011 Outcome 1: Education Cluster and national actors have appropriate technical capacities.
It will play a key role in establishing a baseline and robust monitoring and evaluation framework for future interventions in the area of EPR and DRR.
Challenges that are likely to affect the study include:
•Absence of a baseline: Baseline data were not collected before the start of the capacity development/institutionalisation strategy, but are now essential to identify the gaps and ensure that strategic responses are developed;
•Difficulty to qualitatively assess the level of EPR/DRR institutionalisation, preparedness and resilience;
•Availability of key MoE Officials;
•Reduced sample size due to access to trained staff because of staff turnover in UNICEF, Save the Children and partners including Ministries;
•Country Offices evaluation fatigue after a number of previous M&E exercises;
•Short time scale required for the completion of the study.
SCOPE AND FOCUS
The study will look at both individual and institutional capacity development bearing in mind that as a result of the workshops, participants representing the national/provincial/district level were expected to have learned the following competencies/be able to perform the following tasks (which constitute the workshops’ 6 objectives):
•Learn the key technical components of effective emergency response in education through interactive and participatory approaches, from conducting rapid assessments and establishing temporary learning spaces to developing emergency education curricula and devising psychosocial support strategies (knowledge);
•Have the knowledge and skills to apply these technical components for strengthened preparedness at national/provincial/district level. Participants develop action plans for capacity development activities at school and community levels for improved preparedness and disaster risk reduction in the most emergency-prone localities (practice);
•Indicate priority advocacy and policy action to inform national education sector policy, planning and budgeting for strengthened preparedness, response and risk reduction so that education in emergencies is addressed in a more systematic and sustainable manner(practice);
•Map capacity of education actors at national and local levels for emergency education and identify roles and responsibilities for effective coordination through new or existing mechanisms including education clusters (practice);
•Initiate the process of contingency planning for the education sector at local levels in coordination with the relevant disaster management authorities (practice);
•Plan for the roll-out of further capacity development at local levels, as required (practice).
The capacity development strategy adopted addresses both emergency preparedness and response (EPR) by improving UNICEF and its partners’ capacities to respond, but also by mitigating the damage resulting from future crises through the institutionalization of emergency preparedness and disaster risk reduction (DRR) in the education sector’s policy and planning (see the fifth column of the table in the Annex). EPR and DRR institutionalization - meaning the vertical (centralised/decentralised) but also horizontal (all MoE’s departments/sections) mainstreaming of emergency awareness throughout the MoE’s structure, policy and planning and its linkage with overall disaster management structures - was suggested as the best strategy to ensure the overall capacity development intervention was effective and sustainable. For emergency preparedness and DRR to become embedded in ministries’ actions at all levels, a multi-step approach is required. This involves sensitization, awareness raising, mentality change, training, institutionalisation (roles, responsibilities, structures, etc.), behavioural change and ultimately emergency risk informed programming.
Key indicators of impact of the above should be reflected in: MoE contingency planning; integration of EPR/DRR into education sector policy and plans (if these have been recently reviewed), curricula, EMIS, Medium-Term Expenditure Frameworks at national and district levels, school and community plans (DRR trainings and drills); MoE participation in national disaster management; co-ordination mechanisms and identification of roles and responsibilities for EPR/DRR at the national and sub-national levels; sector review documents; MoE participation in education clusters; meeting minutes; utilisation of the training in the event of actual emergencies.
The study will analyse the contribution of capacity development on institutionalisation with the understanding that institutionalisation should contribute to building resilience but this study will not actually measure resilience.
The study is both summative to assess the outcome as donor funding is coming to an end and formative since support for EPR and DRR will continue in the future.
The EEPCT Programme has already been reviewed/evaluated twice but this new proposed study focuses on the Programme’s capacity development and institutionalisation components, responding to recommendations made in the two previous studies (see Annex for details). The proposed study will expand the evidence base by incorporating other countries that were not covered in the two previous studies.
The geographic scope of the evaluation and the selection of countries to be evaluated will be done in cooperation with Country Offices, taking into account the following criteria:
•Balanced inclusion of countries according to potential emergency scenarios (man-made and natural disaster, chronic emergency, complex emergencies);
•Preference for countries with high levels of vulnerability;
•Stratification of sample according to phases of emergencies and transitions (mid-crisis, early recovery, or transitional);
•Stage/progress of the institutionalization efforts as reported by the UNICEF Country Offices;
•Ability of the country to facilitate or absorb an additional external mission at relatively short notice.
•Identifiable list of trainees to which researchers may have access
Countries which have confirmed their participation include: Burundi, Comoros, Ethiopia, Madagascar, Malawi and Rwanda.
The validity of the evaluation is premised on its being conducted by a completely independent source, although UNICEF will provide logistical support and access to government and other partners at the country level.
Responsibilities:
I. PURPOSE OF THE STUDY
The study will systematically and objectively assess the effectiveness, adequacy and sustainability of the EPR/DRR capacity development strategy and its contribution to institutionalization in the education sector in the Eastern and Southern Africa Region and draw lessons for future programming including the development of a robust monitoring and evaluation framework .
The study is both necessary and timely as it is expected to:
•Help country offices identify and understand achievements and challenges arising from capacity development/institutionalisation efforts, including the perspectives of participants and duty bearers;
•Provide the country and regional offices with recommendations to inform decision-making at a critical time i.e. the end of the current EEPCT grant;
•Provide evidence for future advocacy and fund raising efforts;
•Identify a way forward for enhancing the future quality of EPR and DRR capacity development and institutionalisation in the education sector, including the development of robust planning and M&E frameworks.
II. OBJECTIVES
The proposed evaluation study has 4 objectives:
1. To assess the EPR/DRR knowledge of individuals who attended the EPR/DRR trainings provided by UNICEF/partners since 2009.
•What did participants actually learn?
•How far has the cascade training approach gone and what has the impact been at each level, including the community/school level (where applicable)?
•What other ERP/DRR capacity building activities have trainees participated in and what did learn from these?
2. To assess the EPR/DRR practices of individuals who attended the EPR/DRR trainings provided by UNICEF/partners since 2009.
•How have trainees applied what they have learned?
•How did trained staff react to an emergency compared to untrained staff (where applicable)?
•What were the achievements and the challenges in terms of the competencies that those who were trained developed and were they sustained?
•What mechanisms have been put in place to ensure the sustainability of the activities and how effective are they?
•How are the activities monitored and evaluated at the regional and national levels and how can this be enhanced?
3. To assess whether and how the capacity development strategy has contributed to the institutionalization of EPR and DRR at national/education sector level (national policy and planning) and at sub-national level (as applicable) since its launch in 2009.
•How have trainings helped build effective institutional capacities, e.g. being translated into the integration of EPR/DRR into the education sector’s policy and programming?
•What is the role of the education cluster or sector coordination group in this integration?
•To what extent are roles and responsibilities clear and institutionalised at all levels of the system?
•What other capacity development activities, if any, have contributed to EPR/DRR institutionalisation?
Additional questions raised by the EEPCT Programme Review and Evaluability Study or PRES:
•Are issues of conflict vulnerability and disaster risk reduction accounted for within the design and implementation of [MoE] programme activities?
•Are the contextual features of areas affected by chronic crises also appropriately considered?
4. To identify and recommend measures to address capacity in practice gaps and institutionalisation failures to inform future EPR/DRR capacity in ESAR and develop a clear road map and logical framework for the way forward.
•Where are the existing capacity gaps and what are the potential strategic responses including key actions required for effective EPR/DRR institutionalisation?
•What are the principal lessons learned – both positive and negative?
•What key elements need to be incorporated into future planning and M&E?
III. METHODOLOGY
The study will take place in 4 steps:
1. Orientation and Desk Review Phase (Home base)
An initial distance briefing meeting (through Webex or teleconference) will take place with the Regional Education Adviser, the Regional Chief, Monitoring and Evaluation, the DRR Specialist and the Education Specialist (Emergency and DRR) who will comprise the Reference Group for the study.
For the Desk Review, core documents and contacts will be made available by the Education Specialist (Emergency and DRR). These documents will include previous EEPCT Studies, training materials, list of participants, training evaluations, national capacity development roll-out plans and national sectoral policy documents.
2. Inception Phase (Home base)
The Institution will produce an overall roadmap of the consultancy which shows in a work plan the tasks, the deliverables and deadlines against the TOR. This road map will be known as the Inception Report.
The Inception Report will include a thorough description of the scope of the exercise and the methodological approach for data collection and analysis. For the data collection, a combination of tools will be expected, and will include, inter alia, interviews, written or oral tests of trainees, on line surveys, focus groups and multi-stakeholder consultations, using an appreciative inquiry approach. For the analysis, an evaluation framework will be developed that lists and defines each evaluation criteria to be used by the evaluators. Quantitative but also qualitative indicators will have to be developed to measure the EPR/DRR institutionalization by the MoE and its partners. The framework will explicitly analyze if the capacity development/institutionalization strategy adequately insured the inclusion of Human Rights, Gender (including gender-based violence), Equity and Results Based Management, and the extent to which the integration of EPR/DRR into education sector planning was inclusive of these issues. The Inception Report will be the subject of discussion in a meeting with the Reference Group.
The study design will take the following recommendation into account from the EEPCT Programme Review and Evaluability Study: “future evaluative work may attempt to undertake standardised approaches (such as knowledge, attitude and practice (KAP) surveys) to measure the effectiveness of previous or ongoing capacity development activities (including trainings and workshops)”.
3. Implementation Phase/Country Visits
The consultants will visit countries in team of 2. The logistics of data collection will be supported by the UNICEF Education Specialist (Emergency and DRR) and facilitated by country-based National Consultants.
Data collection will include the following steps:
•Initial consultation with UNICEF Country Office staff;
•Structured interviews with key stakeholders (see also list of informants below): programme managers at UNICEF and Save the Children, project managers from (implementing) partners, officials from the MoE and other relevant ministries at the national and sub-national level (where applicable), co-lead and members of education clusters (where cluster approach is activated ) or sector coordination groups;
•Participant profile analysis to define the level of institutional change that the training could have influenced and that which it could not;
•Analysis of any other initiatives at the CO level that have contributed to enhanced EPR and DRR in the education sector;
Analysis of linkages with other sectors (WASH, Protection, Health, Nutrition, etc.)
•Review of key documents – policies, plans and budgets, monitoring and evaluation reports, etc. at national, sub-national and school levels (where applicable);
•Group discussions on draft findings with UNICEF and partners in country.
Consultants, in team of 2, are expected to spend two weeks in each selected country to ensure an in-depth study of capacity development activities at national, but also at the sub-national level. UNICEF will provide logistical support and access to government and other partners at the country level. Consultants will also feedback initial findings as a validation exercise within country before departure.
4. Reporting Phase
A Draft Report will be delivered to the Reference Group on 14 November 2011. The Reference Group will provide feedback by 17 November 2011. The Final Report will be due on 25 November 2011.
The report, written in English, is expected to contain, as a minimum, the following features:
•Executive summary
•Context
•Literature review
•Methodology including indicators, evaluation tools and framework
•Findings
•Recommended measures to address identified capacity gaps
•Lessons Learned to inform future capacity development/institutionalisation work
•Road map and logical framework for the way forward
•Annexes including ToRs
•The final report should not exceed 50 pages (excluding annexes)
Who should be the key informants?
It will be essential that the views of all stakeholders are taken into account:
Direct beneficiaries of the capacity development intervention themselves at the various levels of the intervention (national, sub-national and community):
•UNICEF and partner (particularly Save the Children) staff on the degree to which trainings have reinforced individual EPR skills and influenced the operating procedures and institutional responsiveness within their respective organisations;
•Ministry of Education officials on the degree to which trainings have strengthened individual EPR skills and influenced national policy and planning for EiE (including at school and community level where applicable);
•Disaster Management (DM) bodies and other sectors officials on the degree to which the education sector has been integrated in national DM policies/ planning or linked to other sectors priorities and strategies.
•Colleagues involved in EPR/DRR who were not direct beneficiaries of the training to measure the level of institutionalisation within the organisations and ministries.
ESARO will take the following responsibilities:
•Identifying and recruiting the institution to carry out the proposed study;
•Briefing the consultants;
•Reviewing the inception paper;
•Coordinating field visits with Country Offices;
•Reviewing and validating the Final Report in coordination with the consultants;
•Disseminating the Study’s findings;
•Coordinating the actual implementation of the Study’s recommendations;
•Designing the new phase of the capacity development/institutionalisation strategy.
Participant Country Offices will be responsible for:
•Availing key relevant staff members for interview;
•Providing logistical and administrative support (ESARO to make all efforts to limit required CO support to a minimum);
•Recruiting one national consultant to prepare, assist and accompany the institutional consultant on field visits (funded by ESARO);
•Facilitating interaction with the Ministry of Education, Disaster Management agencies and other stakeholders involved in EPRP and DRR, including access to appropriate co-ordinating mechanisms (e.g. Education Clusters and Ministry Working Groups).
•Following up on, and implementing, the study’s recommendations.
Benchmarks and Perfomance Standards
The study will refer to the following benchmarks:
•Core Commitments for Children in Humanitarian Action (CCCs)
•Inter-agency Network for Education in Emergencies (INEE) Minimum Standards for Education: Preparedness, Response, Recovery
•UNICEF Disaster Risk Reduction (DRR) Programme Guidance Note
•The UN Ethical Guidelines for UN Evaluators in the UN System and the UNICEF technical note #2 on ethics in evaluations (children’s involvement in evaluations) are to be followed.
IV. ORGANISATION AND MANAGEMENT
The evaluation team will consist of 4 consultants working in 2 teams of 2 under the supervision of one Team Leader (one of the 4 consultants). The consultants will report to, and received guidance and supervision from, the Education Specialist (Emergency and DRR).
Reference Group for the study will comprise of the Regional Education Adviser, the Regional Chief, Monitoring and Evaluation, the DRR Specialist and the Education Specialist (Emergency and DRR).
Requirements:
Each team of 2 consultants should combine the following competencies:
•Strong expertise in education sector policy and planning;
•Demonstrated experience in evaluation, particularly of capacity development approaches and of institutional change programmes;
•Understanding of conflict/disaster education programming;
•Some exposure to/knowledge of UNICEF work in education in the humanitarian/post-conflict sector;
•Demonstrated understanding of EPR/DRR capacity mapping and capacity development approaches in the education sector;
•Knowledge of global emergency education issues and how these “translate” in the Eastern and Southern Africa Region;
•Demonstrated technical skills in the field of disasters, particularly the social and human dimensions of disaster impact, and understanding of the concepts of ‘risk’, ‘exposure’ and ‘vulnerability’ and of how disaster impacts are mediated by different factors;
•Analytical and conceptual ability; good communication skills;
•Effective written and verbal communication skills in English;
•Proficiency in French to conduct the study in the 3 Francophone countries (Burundi, Comoros and Madagascar);
•Good inter-personal skills with cultural sensitivity;
•Experience in working for UNICEF and/or other UN or humanitarian organizations an advantage;
•Personal and professional integrity;
•Institutional contractor must provide UNICEF with a Certificate of Incorporation/ documentation that this a registered company or institution
Timeline & Start Date:
September to November 2011
Starting date: 01 September 2011
Application Procedure:
Full Request for Proposal document can be requested from the Supply Assistant – Timothy Wasilwa, (twasilwa@unicef.org) with C.C. to OIC, Supply section – Monika Tschida-Spiers (mtschidaspiers@unicef.org) by 15 July 2011 COB latest. The actual Proposal MUST be hand delivered or sent by courier and received by latest 10:00a.m (East Africa Time) on Wednesday 27th July 2011.
Consultant - Evaluation of EPR/DRR Capacity Development in ESAR
![Consultant - Evaluation of EPR/DRR Capacity Development in ESAR]()
Reviewed by Unknown
on
2:13:00 AM
Rating:
No comments: