Evaluation of the Joint Programme 5 on Capacity Building Support to Zanzibar

Tanzania is among the eight countries piloting the Delivering As One concept in the world. The ‘Delivering as One’ (DaO) approach was conceived by the UN Secretary-General’s High-level Panel on System-wide Coherence in the areas of development, humanitarian assistance, and environment. The DaO approach was initially conceptualised as the ‘Four Ones’ – One Programme, One Leader, One Budgetary Framework, and One Office where appropriate. The UN Country Team in Tanzania added the ‘One Voice’ in 2008 – making it ‘Five Ones’. On 22 November 2007, the Secretary-General decided to establish a series of pilot country initiatives to test the ‘One UN’ approach. The governments of eight countries – including Tanzania – volunteered to pilot the DaO approach. It was agreed that the testing of the DaO approach and principles would be undertaken on a voluntary basis, and that the outcome and lessons learned, based on the experiences, would inform future intergovernmental consultations. The One UN Programme was the first of the ‘One’s’ to be fully developed in Tanzania. The One Programme is implemented through a number of Joint Programmes (JPs). The JPs seek to respond to national priorities and represent sectors in which the UN has expertise to deal with the development gaps. The JPs are thus aligned to the National Strategy for Growth and Reduction of Poverty; MKUKUTA/MKUZA. The Joint Programme approach involves collaborations between various UN agencies and partners. This implies making joint work plans, joint budgets and defining common results. Currently, there are 10 Joint Programmes being piloted in Tanzania (Tanzania Mainland and Zanzibar). Although some Joint Programmes extend their activities to Zanzibar, the Joint Programme on Capacity Building Support to Zanzibar (JP5) is the only UN joint programme framework for development assistance to Zanzibar (Unguja and Pemba isles) which is multi-sectoral and addresses all clusters of the National Strategy for Growth and Reduction of Poverty in Zanzibar-MKUZA. The programme aims to support Government institutions in Zanzibar to improve capacity and service delivery for the purpose of contributing to the realization of pro-poor growth and enhancement of democratic governance in Zanzibar. The programme commenced in January 2008, and is implemented under the leadership of the Revolutionary Government of Zanzibar (RGoZ). The overall budget for the JP5 programme is USD 8,779,543.60. The Capacity Building Support to Zanzibar aims to contribute to the achievement of the goals and objectives of the MKUZA and the MDGs as they relate to Zanzibar. Results to be achieved through the programme includes: increased access to sustainable income opportunities, productive employment and food security in rural and urban areas by 2010; improve access to quality basic social services for all especially the poor and most vulnerable; strengthen democratic structures and systems of good governance, as well as the rule of law and the application of human rights. Strategically, JP5 responds to national priorities and vision 2020 for Zanzibar and MDGs as delineated in the Zanzibar National Development Plans (i.e. the Zanzibar Strategy for Growth and Reduction of Poverty and other sector or agency specific plans). In responding to Vision 2020 and MDGs, JP5 utilizes comparative advantage of the UNs in programming and contributes to the achievement of the MDGs 1, 2, 3, 4, 5, 7, and 8; and UNDAF’s outcomes. With the three-pillar structure of the Joint Programme, JP5 combines efforts of the UN Agencies and its national counterparts to address the three main clusters of MKUZA, that is: 1) Wealth Creation, Employment and Economic Empowerment responding to Growth and Reduction of Poverty of MKUZA cluster I; 2) Reduction of Maternal, Newborn and Child Mortality and Improved Social Services responding to Social Services and Well-being of MKUZA cluster II; and 3) National Capacity Development Management responding to Good Governance and National Unity of MKUZA cluster III. Each pillar is led by a national counterpart with a participating UN agency providing technical assistance and supplementary parallel financial support. The pillar leaders have the overall thematic responsibility to coordinate the pillar partners in planning, implementing, and reporting of pillar outputs and avail the opportunity for the UN and its national partners to work closely together to support capacity development efforts for a greater impact in Zanzibar. The national implementing partners (IP) include: Association Of Non Governmental Organizations (ANGOZA), Ministry Of Agriculture Livestock And Environment, Ministry Of Labour Youth Women And Child Development, Ministry of Constitutional Affairs And Good Governance, Ministry Of Education And Vocational Training, Ministry Of Finance And Economic Affairs, Ministry Of Health And Social Welfare, Ministry Of Tourism Trade And Industries, Office Of Chief Government Statistician (OCGS), Zanzibar Employees Association (ZANEMA), Zanzibar Trade Union Cooperation (ZATUC), Zanzibar Water Authority (ZAWA), Zanzibar Food Security And Nutrition Policy And Programme and Zanzibar National Chamber Of Commerce Industry And Agriculture (ZNCCIA). While the PUNs include Food And Agriculture Organization Of The United Nations, International Labour Organization, United Nations Development Programme, United Nations Population Fund, United Nations Children’s Fund, United Nations Industrial Development Organization and World Health Organization. JP5 provides opportunity for the UN to work in a harmonized, coherent and coordinated manner and become more cost-effective while delivering greater development impact and minimize fragmentation and inefficiency. As a result, UN Agencies have developed a common work plan and budgetary framework and use of common business processes such as procurement, financial reporting system and procedures (HACT) to reduce transaction costs. UNDP is the Managing Agent of the Joint Programme. As a Managing Agent (MA), UNDP has ultimate responsibility and accountability for both the achievement of results and management of funds since it is responsible for overall coordination of the programme. The MA is responsible for reporting matters including (1) preparation of consolidated narrative and financial documents and reports for review by the Joint Programme Committee (and subsequent submission to the Joint Steering Committee); (2) progress reporting to governance structures; and (3) financial reporting to the One UN Fund’s Administrative Agency (AA). The planned terminal evaluation will be carried out to assist the UN and programme stakeholders to draw lessons learned in implementing the programme and improve the quality of future development interventions. As the UN in Tanzania moves towards the UNDAP, the first all-inclusive One UN business plan for Tanzania, the review of the JP5 will also provide opportunity to reflect on the challenges in the management and implementation of such multi-stakeholders programs and inform the formulation and implementation of similar programmes and partnerships which might derive from the UNDAP. Summary of the key outputs of the JP5 is presented below: MKUZA Cluster 1: Growth and reduction of income poverty; Goal 3: reduce income poverty and attain overall food security UNDAF Outcome: National productivity and competitiveness is improved through decent employment opportunities, equitable access to and effective use of productive resources, improved transport and communication networks and greater market access JP Output: Wealth Creation, Employment and Economic Empowerment Strengthened capacity of relevant Government institutions, private sector, and CSOs for implementation of SME policy and enforcement of labour laws from a gender perspective Strengthened entrepreneurship capacity of poor groups of youth, women, PLHA and the people with disability. Gender sensitive employment policy, job creation programme and Youth Action Plan developed Comprehensive diagnostic of Zanzibar national needs for social protection undertaken Strengthen capacity of government to monitor and manage food security and manage nutrition and food safety MKUZA Cluster 1: Social services and well being Goals 2: improved health status including reproductive health, survival and well-being of children, women and vulnerable groups Goal 7:strengthen and expand social security and safety nets for the disadvantaged and most vulnerable population groups UNDAF Outcome: Increased and equitable access to comprehensive reproductive health interventions Effective mechanisms including social protection in place the address institutional barriers and socio-cultural dimensions to promote and protect the rights of the poor and most vulnerable JP Output: Reduction of Maternal, New Born and Child Deaths and Improved Social Services Roadmap to accelerate the reduction of maternal, newborn and child mortality finalized and costed Strengthened capacity for primary and referral and health facilities to provide quality maternal, newborn and child health care Strengthened capacity of District Health Management Teams to monitor maternal, newborn and child health interventions Enhanced capacity of multisectoral stakeholders to prevent and respond to GBV and child rights abuse Improved coverage of social services (health, education, water and sanitation) in Micheweni. MKUZA Cluster 3: A society governed by the rule of law and government that is predictable, transparent and accountable Goal 5: increase the capacity of government institutions and actors Goal 9: provision of timely and reliable information and data for monitoring and evaluating government initiatives UNDAF Outcome: Strengthened MKUKUTA/MKUZA monitoring budget and planning systems that foster participation and gender equality JP Output: National Capacity for Development Management Strengthened MKUZA Monitoring System Strengthened capacity for implementation of core reforms and aid coordination Effective participation of MDAs and NSAs in Policy Dialogue and Public Expenditure Review (PER) process

Duties and Responsibilities

The main purpose of the evaluation is to provide an independent in-depth assessment of the achievements of results as well as the implementation arrangements of the Joint Programme on Capacity Building Support to Zanzibar (JP5). The evaluation will focus on the following criteria: effectiveness, efficiency and sustainability, with a particular focus on effectiveness. Thus the evaluation specific objectives are: Based on planned deliverables of the Project Document (Result and Resource Framework (RRF) the consultants should evaluate the project results (that have/have not been achieved and a special emphasis should be placed on measuring the achievements or non-achievements of the expected results under all components of the project); Assess the relevance, effectiveness, efficiency, impact and sustainability of results/activities; Explore to what extent synergies among UN agencies in particular and among IPs were explored and effected in implementation; What extent the UN was able to go upstream as intended; Whether capacity development results were relevant and assess the likely impact of these; Evaluate the effectiveness and efficiency of the JP modality; To draw experience and lessons learnt from the UN Joint Programme 5 for Capacity Building Support to Zanzibar, its structure, management and implementation arrangements as a strategy for poverty reduction and its relevance for both the One UN pilot project and the RGoZ. Consolidate lessons learned with a view to contribute to improving the future UNDAP implementation strategies and make recommendations to guide future programming for the Delivering As One Scope of the evaluation The evaluation will review and assess the achievement of the programme. The review should include an analysis of (a) an assessment of progress towards achieving the outputs/outcome of the programme, (b) assessment of the key factors that could have affected or could affect the achievement of the outputs/outcomes, (c) assessment of the JP5 strategy in general (d) To discuss additional priorities that could have been included in the programme. The evaluation team will review and assess the achievement of the programme outcomes focusing on: Progress status: What were the basis of the outputs/outcome and its constituent interventions? Were past experiences and lessons as well as dialogue with stakeholders in design of the programme and outputs considered? Assess the adequacy of the background work carried out. Determine the degree to which outputs/outcome was achieved and, if not whether there was progress towards their achievement. Identify the balance effort that were needed and the suitability or otherwise of pursuing the achievement of the outcome/outputs. Underlying factors: An analysis of the underlying factors that have influenced the outputs and achievements. What were the key internal and external assumptions made? Distinguish the substantive design issues from the key implementation and/or management capacities and issues including the timeliness of outputs, the degree of stakeholders and partner’s involvement in pursuit of the outputs, and how processes and activities were managed/carried out. JP5 strategy: Ascertain whether JP5 was appropriate and effective. Assess the initial design modalities and planning process as well as management mechanisms and identify the role of each party involved and how did it function and how has it been sustained? How did each party understand its role? How did the management arrangements contribute (or hinder) to the achievement of the outputs/outcome? What was the level of participation of the stakeholders and the perception of the beneficiaries? Methodology The evaluation should be based on a stakeholder approach, where all groups and individuals, who affect and/or are affected by the achievement of the project objectives, are involved in the analysis. Moreover, the evaluation will take into consideration the institutional, structural and economic context, which affects the overall performance of the project. The approach of the evaluation shall be participatory, that is, be flexible in design and implementation, ensure stakeholder participation and ownership, and facilitate learning and feedback. The Evaluations will utilize both qualitative and quantitative methodology. The consultant will make the best use of the existing documents and conduct individual interviews/group meetings with relevant stakeholders. Thus both primary and secondary data will be utilized. The following data collection methods should be included as minimum. Desk review of all relevant documents of the project e.g., those relating specifically to the project context, the project document, log-frame, implementation plans, monitoring reports, assessment and learning mission reports, progress reports, expenditure reports etc. This could be done prior to any field visit, focus group discussion, or individual interviews; Discussions with the ZPMC (Zanzibar Programme Management Committee), the JWG (Joint Working Group), PUN’s (Participating UN Agencies), Senior Management at UNDP office; UNDP Zanzibar Sub office team, UN Zanzibar program staff, Senior Government officials, Implementing partners (Ministries, NGO’s, Private sector institutions). Briefing and debriefing sessions with the UN-IAPC, the Government of Zanzibar, as well as other main stakeholders Interviews with partners and stakeholders (including gathering the information on what the partners have achieved with regard to the outcome and what strategies they have used); other donors Field visits to selected project sites and discussions with project teams, project beneficiaries The evaluation will be led by one national and one international consultant. The lead consultant (International) after brief orientation, s/he will develop plan of action stating the methodologies and required resources for the end of program evaluation. In the plan of action, areas of evaluation, indicators and data collection should be clearly spelled out. The consultants need to attach interview questionnaires and focus group guide. Consultations will be held among users and other stakeholders to refine and approve evaluation questions. But at the minimum, the evaluation should respond to the following questions: Relevance /Appropriateness – the extent to which the JP pertained to national priorities and the requirements of the target group. Indicative questions might include: To what extent were partners involved in the development and implementation of the JP? To what extent are the objectives of the programme still valid? Were the activities and outputs of the programme consistent with the intended key results? Effectiveness - the extent to which the JP attained its key results. Indicative questions might include: To what extent were the key results achieved? What were the major factors influencing the (non)achievement of the desired results, including institutional, management and fiscal arrangements? Efficiency – the extent to which delivery was undertaken by the most cost-efficient means. Indicative questions might include: Were activities cost-efficient? Were outputs achieved on time? Was the programme implemented in the most efficient way compared to alternatives? Impact - the positive and negative changes produced by the JP (directly or indirectly, intended or unintended), Indicative questions might include: What difference has the activity made to the beneficiaries? How many people have been affected? How did impact differ across key target groups, including vulnerability categories such as gender, age etc? Sustainability - measuring whether the benefits of the JP are likely to continue after the end of programme activities. Indicative questions might include: To what extent will the benefits of a programme continue after activities have ceased? What are the major factors which will influence the sustainability of the programme? The evaluation should also respond to three additional assessment areas. Delivering as One - adherence to the principles and objectives of Joint Programming (include reference to the One UN Process Indicators). Indicative questions might include: To what extent were the principles of Joint Programming in terms of collaborative planning, implementation and monitoring across participating agencies adhered to? What factors facilitated or adversely impacted upon Delivering as One? Cross Cutting Considerations – application of the programming principles of the UN, referencing Human Rights Based Approach, Gender Equality, Environmental Sustainability, Results Based Management and Capacity Development. Indicative questions might include: Were cross-cutting considerations mainstreamed in the implementation of activities? To what extent did the programme involve the host communities and other stakeholders in programme design and implementation? Were capacity development activities informed by a capacity assessment at manifold levels? Lessons and Recommendations for future programming. Indicative questions might include: What additional measures (if any) could have improved the relevance, effectiveness, efficiency, impact or sustainability of the JP? What lessons can be applied in terms of collaborative programme planning, implementation and monitoring to the implementation of the UNDAP 2011-2015? What key measures can the UN in Tanzania, regionally and at HQ, plus the Government of Tanzania and Development Partners (DPs) adopt to improve the implementation and performance of the One Programme under DaO? The evaluator will be selected based on a proposed methodology for the evaluation. The evaluation shall be a result-oriented exercise which involves all key stakeholders. The evaluation shall provide evidence of achievement of expected outputs and outcomes using quantitative and qualitative data. At every stage of the evaluation process, the following principles should be observed: Independence - the evaluation team should be independent from the operational management and decision-making functions of the JP Impartiality – the evaluation information should be free of political or other bias and deliberate distortions Timeliness - evaluations must be designed and completed in a timely fashion Purpose - the scope, design and plan of the evaluation should generate relevant products that meet the needs of intended users Transparency - meaningful consultation with stakeholders should be undertaken to ensure the credibility and utility of the evaluation Competencies - evaluations should be conducted by well-qualified teams. The teams should, wherever feasible, be gender balanced, geographically diverse and include professionals from the countries or regions concerned. Ethics - evaluators must have professional integrity and respect the rights of institutions and individuals to provide information in confidence and to verify statements attributed to them. Evaluations must be sensitive to the beliefs and customs of local social and cultural environments and must be conducted legally and with due regard to the welfare of those involved in the evaluation, as well as those affected by its findings. Quality - All evaluations should meet the standards outlined in the Standards for Evaluation in the United Nations System. The key questions and areas for review should be clear, coherent and realistic. The evaluation plan should be practical and cost effective. To ensure that the information generated is accurate and reliable, evaluation design, data collection and analysis should reflect professional standards, with due regard for any special circumstances or limitations reflecting the context of the evaluation. Evaluation findings and recommendations should be presented in a manner that will be readily understood by target audiences and have regard for cost-effectiveness in implementing the recommendations proposed. Key deliverables (Evaluation Products) The consultant(s) will produce a comprehensive structured evaluation report that provide analysis and evidence, independent in-depth assessment of the achievements of results as well as the implementation arrangements of the Joint Programme on Capacity Building Support to Zanzibar (JP5) as well as lessons learnt. Evaluation inception report—an inception report should be prepared by the evaluators before going into the fully fledged evaluation exercise. It should detail the evaluators’ understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods; proposed sources of data; and data collection procedures. The inception report should include a proposed schedule of tasks, activities and deliverables, designating a team member with the lead responsibility for each task or product. The inception report provides the programme unit and the evaluators with an opportunity to verify that they share the same understanding about the evaluation and clarify any misunderstanding at the outset. Evaluation matrix – An evaluation matrix should be included in the inception report. The evaluation matrix is a tool that evaluators create as map and reference in planning and conducting an evaluation. It also serves as a useful tool for summarizing and visually presenting the evaluation design and methodology for discussions with stakeholders. It details evaluation questions that the evaluation will answer, data sources, data collection, analysis tools or methods appropriate for each data source, and the standard or measure by which each question will be evaluated. Debrief the ZPMC members, ZJWG Members as well as the Managing Agent. A Final evaluation report, the Report should be written fully in line with UNDP quality criteria and standards (UNDP standard evaluation format will be utilized). The structure of the Final Report should be including the following as a minimum (i.e. Hard copy, a soft copy in MS Word and Acrobat reader, Times New Roman, Size 12, Single Spacing) containing: Title and opening pages Table of contents List of acronyms and abbreviations Executive summary Introduction Description of the intervention Evaluation scope and objectives Evaluation approach and methods Data analysis Findings and conclusions Recommendations Lessons learned Report Annex (including interview list, data collection instruments, key documents consulted, ToR) Provide a draft report 10 days before the end of the consultancy period and request for comments to identified stakeholders to allow enough time for incorporation of comments received. The Final Evaluation Report should be presented in a solid, concise and readable form and be structured around the issues in the ToR, 3 days before the end of consultancy period. The consultant should refer to annex 7 of the UNDP Planning, Monitoring and Evaluation handbook for details on reporting template. Supervision, reporting and Management of the assignment Under the overall guidance of the UNDP Deputy Country Director (Programme, the selected consultants shall report to the UNDP Pro-Poor Unit Team Leader. The Head of the UNDP Sub Office in Zanzibar will facilitate the work of the consultants and ensure smooth interaction with the joint UN/RGOZ Task Force as well as other key stakeholders. Management and Implementation arrangements The overall support of the evaluation team lies with the UNDP Pro-Poor Team Leader. In terms of practical management and implementation, the UN Sub-Office Team will coordinate the work of the Evaluation Team, while the UN/RGOZ Task Force will provide the Evaluation the following support: Provide the consultant with all the necessary support (not under the consultant’s control) to ensure that the consultant(s) undertake the study with reasonable efficiency. Appoint a focal point in the programme section to support the consultant(s) during the evaluation process. Collect background documentation and inform partners and selected project counterparts. Meet all travel related costs to project sites as part of the JP5 evaluation cost. Support to identify key stakeholders to be interviewed as part of the evaluation. The programme staff members will be responsible for liaising with partners, logistical backstopping and providing relevant documentation and feedback to the evaluation team Cover any costs related to stakeholder workshops during consultation and dissemination of results Organize inception meeting between the consultants, partners and stakeholders, including Government prior to the scheduled start of the evaluation assignment. Evaluation Ethics The evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’ and should describe critical issues evaluators must address in the design and implementation of the evaluation, including evaluation ethics and procedures to safeguard the rights and confidentiality of information providers. Time Frame The work should be carried out in a total of 30 working days from starting date. Orientation with UN Zanzibar, finalize and agree on ToR , revision of Plan of action ; submission of inception report; avail documents, evaluation of methodologies, Desk review of relevant of documents , Presentation of inception report and briefing with UN-IAPC. (3 working days) Data collection and Field visits Submission of draft report, De-briefings through power point to UN-RgoZ JTF & ZPMC (20 working days) Incorporation of comments and submission of final report with clear set of recommendations, dissemination of findings (7 working days) Main Reference materials The consultants should study the following documents among others: UNDP Handbook on Planning, Monitoring and Evaluating for Development Results; Ethical Code of Conduct for UNDP Evaluations; Guideline for Reviewing the Evaluation Report; UNDP Results-Based Management: Technical Project Documents and relevant reports Documents and materials related to the Joint Programmes Management in Tanzania National Strategy for Growth and Reduction of Poverty (MKUZA I & II) Conditions of Work Consultants will be expected to use their own laptop computers. UNDP will support and facilitate the consultants travel, provide administrative, logistics and facilitate security related issues of the consultancy. The UN Sub Office in Zanzibar will avail working space during the consultancy period. Consultants shall arrange by their own means of accommodation in Zanzibar during consultancy period.

Competencies

With the input of UN participating agencies, the UNDP Country Office will select an independent consultant team based on competitive and transparent process. The evaluation team must be independent from both the policy-making process and the delivery and management of assistance. The evaluation team shall consist of 2 consultants: an International consultant (team leader) (1) and a national consultant (1) with extensive knowledge of the country situation. The Team Leader (International) will have the responsibility for the overall co-ordination of the evaluation and for the overall quality and timely submission of the evaluation report to the UN-RGoZ JTF and ZPMC. The team is expected to have experiences and skills in the following area: Program designing and strategic planning, monitoring and evaluation of the implementations system, designing of work flow Experience in UN business processes and programmes will be an asset

Required Skills and Experience

International Consultant: Advanced Degree in Social sciences, Development studies or any other related or field Proven experience of a minimum of 10 years at the international level, preferably with UN experience. Knowledge and familiarity of the United Nations system, its reform process and UNDP programme policies, procedures. Previous experience in conducting country programme evaluations is an asset. Experience applying participatory monitoring approaches; Experience applying SMART indicators and reconstructing or validating baseline scenarios; Demonstrable analytical skills; Experience with multilateral or bilateral supported capacity development projects; Knowledge of the political, cultural and economic situation in Zanzibar or ability to quickly acquire such knowledge is desirable Excellent writing and analytical skills Ability to meet tight deadlines Fluency in English, knowledge of Swahili will be an asset National Consultant: Masters Degree in Social sciences, Development studies or any other related or field Work experience in relevant areas for at least 5 years; Recent experience with result-based management evaluation methodologies; Experience applying participatory monitoring approaches; Experience applying SMART indicators and reconstructing or validating baseline scenarios; Demonstrable analytical skills; Very good knowledge of the political, cultural and economic situation in Zanzibar as well as institutional arrangements Excellent English communication skills as well as Swahili Proven excellent analytical and written skills Fluency in English and Swahili

UNDP is committed to achieving workforce diversity in terms of gender, nationality and culture. Individuals from minority groups, indigenous groups and persons with disabilities are equally encouraged to apply. All applications will be treated with the strictest confidence.

Evaluation of the Joint Programme 5 on Capacity Building Support to Zanzibar