Independent External Evaluation of the Election Support Project 2010 (Terminal Evaluation), Dar es Salaam

The Constitutions of the United Republic of Tanzania and Zanzibar provide for periodic elections by universal suffrage every five years. The fourth multi-party elections were held in October 2010 for the Parliaments and Presidents of the Union and Zanzibar. The three most recent elections following the reintroduction of multiparty politics were generally considered free and credible on the Mainland but have been problematic in Zanzibar with election-related violence (2000/2005) and allegations of registration rigging (2005). Notwithstanding, in the periods between elections, the country has taken notable steps to consolidate democracy including the revision of legislation and introduction of new laws including the Election Expenses Act 2010.

Acknowledging the centrality of credible and legitimate elections in democratic consolidation and based on the needs identified by the government, its key partners and stakeholders, the Union and Zanzibar governments (through NEC and ZEC, engaged early in the preparations for the 2010 general election. To complement the national effort, the Electoral Management Bodies (EMBs) requested the United Nations to help enhance national capacities for the preparation, management, conduct and observation of the 2010 general election.

In response, and within the framework of global UN election support including the joint EU-UNDP agreement on electoral assistance, the United Nations fielded a joint electoral Needs Assessment Mission (NAM) from the UN Department of Political Affairs/ Electoral Assistance Division and UNDP to assess the pre-electoral conditions including the political environment and to identify the challenges, constraints and needs of various actors in the electoral process. Among the key actors consulted were the EMBs, political parties, civil society organizations, the media, and development partners. Upon satisfaction of the need for UN engagement a project document for Electoral Support was developed.

The main goal of the Election Support Project 2010, which was formally inaugurated in June 2009, was to further strengthen national capacities for the implementation of free, fair and credible elections, with a focus on strengthening the national leadership of Electoral Management Bodies (EMBs) and engagement and capacities of political parties, media and other civil society, as well as national security management institutions, in contributing to this overall goal.

The project was implemented under UNDP Direct Implementation with a dedicated project management team under the leadership and full operational support of the UNDP Country Office. The project operated under the overall guidance and leadership of a high-level Steering Committee that delegated aspects of quality assurance to two Technical Committees. The total project budget of USD

28,300,000 was

funded through a multi-donor basket funding mechanism, which received contributions from Canada, Denmark, the European Commission, Finland, Norway, Netherlands, Sweden, Switzerland, UNDP and the United Kingdom. Cost-sharing agreements signed bilaterally between UNDP and individual Development Partners detailed the contractual obligations of the parties.

Scope and Objectives:

The overall aim of the project final, independent evaluation is to assess the contribution by the UNDP Elections Support Project to the conduct of free, fair and credible elections in Tanzania and Zanzibar, as expressed in its overall goal. This will be done by examining and assessing the extent to which outputs were achieved, the contribution of outputs to project outcomes, and their likely impact, relevance and overall sustainability.

A second key objective is to examine the appropriateness of the form of assistance provided, with a view to draw lessons and make recommendations for future development cooperation around Electoral Assistance and assistance to Electoral Management Bodies in Tanzania. In meeting this objective, the evaluators will examine a range of factors, including the modality of financial support provided (through a 'basket' arrangement); the management and governance arrangements for the project; the quality, relevance and effectiveness of Technical Assistance and other services provided, project and component design, etc. Such findings will influence any assistance provided by UNDP and other development partners to the EMBs in the period between elections, as part of an electoral cycle approach, and for consideration of any support for the 2015 general election.

The evaluation period will include election-related support beginning with the fast track support for the upgrading of the Voters Registration system which was carried out under the auspices of the Deepening Democracy in Tanzania Project in 2008-9, and the full duration of the Elections Support Project from June 2009 until closure.

The Evaluation scope is focused on the following criteria:

1. Relevance (This component will count as 10% of the financial value of the full contract):

The extent to which the project design and implementation of activities was able to respond to and address:

The needs identified in the 2008 Needs Assessment Mission (NAM) by the UN Department for Political Affairs and expressed priorities by the Electoral Management Bodies and other key national stakeholders;

The mandate of UN electoral assistance and UNDP goals in Democratic Governance ;

Recommendations emerging from the 2005 elections observer reports, and related project evaluations from UNDP's assistance (2005 elections and the Deepening Democracy in Tanzania Programme, DDTP), including for example pre-election requirements for voters registration and education;

Gender disparities and differing abilities of other segments of the population to participate in elections.

2. Effectiveness (This component will count as 20% of the financial value of the full contract)

The extent to which project activities yielded expected outputs at the component level and contributed to expected outcomes, for example:

Improved and sustained capacity of the EMBs to plan and manage the elections, including notably: voter registration , pre-election procurement and logistics, voter education, facilitation of observer activities; results dissemination; handling of security arrangements; processing of complaints; operational planning.

Improved understanding of the elections process among voters (as measured by access of potential voters to voter education information);

• Improved 'balanced' coverage by the media of the elections;
• Participation of civil society;

Improved knowledge of political party officials and activists of their rights and responsibilities;

The evaluation shall examine to what extent results were achieved and endeavor to attribute any positive or negative results to the project.

3.

Efficiency (This component will count as 20% of the financial value of the full contract)

The evaluation shall further examine how economically the project used resources at its disposal, including the efficacy of the planned activities within the time frame and budget for project implementation; to what extent resources were used appropriately to produce the intended outputs; and the extent to which project funds, expertise and time were used judiciously to achieve desired project results.

4. Value for Money (This component will count as 20% of the financial value of the full contract)

The evaluators are expected to make a quantitative cost benefit analysis of select major procurements and activities to determine the extent to which;

The financial investments (cost of procurement) made were justified in light of the benefits that the project delivered, including the sustainability of results towards 2015 elections;

Procurements of products and services were conducted in pursuit of best value for money;

Project costs were justified in comparison with similar scale election support projects in other countries;

As a minimum, the evaluation shall seek to examine the value for money of:

The total investment into the Voter's registration system (noting that a bulk part of the investment was made with funding outside of the 28 MUSD channeled as fast-track through another project);

The Voter Education Grant Facility;

Large-scale procurement of elections-related materials such as ballot boxes;

The Incident Reporting and Response System (IRRS) installed with the Tanzania Police Force to support the management of elections-related violence;

At least one activity within each of the project's other components.

5. Sustainability (This component will count as 10% of the financial value of the full contract)

The extent to which the improved systems, processes, and capacities resulting from this project are likely to benefit the EMBs after the life of the project and particularly with a view to application for the next scheduled elections in 2015, as well as the

ability to adapt and use such capacity for other processes, such as potential by-elections, referendum, grassroots elections, etc. S

ustainability should be assessed in particular with regard to the following:

Improvements to the Voter's Register from 2009

• Skills and expertise development;
• Capacities for Voter Education;
• Improvements in the Media professionalization compared to 2005 elections;

Capacities for supporting a peaceful environment, including the Incident Reporting and Response System (IRRS);

Future use of other key procurements;

Overall organizational mandate and capacity of the EMBs.

6. Impact (This component will count as 20% of the financial value of the full contract)

The evaluation should seek to approximate or estimate the project's contribution to the preparations for and accomplishment of the 2010 general election.

Consider project design as well as execution, and examine ESP activities across all the six project components;

Assess the degree to which gender, capacity development, and other cross-cutting issues were effectively mainstreamed and addressed in project design and development;

Assess the design and implementation of ESP in relation to core aid effectiveness principles such as; national ownership and leadership;

Extract the lessons learned and best practices and elaborate specific recommendations to the participating partners and project stakeholders.

Duties and Responsibilities

Methodology

Baseline

The evaluators may construct a baseline on the basis of the project's M&E framework and the assumptions made therein. In addition, the final evaluation of the 2005 election support and the 2008 UN-DPA Needs Assessment Report will be other key references for comparative analysis . Other independent assessments, such as independent observers' reports from the 2005 elections, may be helpful for appreciation of the baseline.

The evaluation should use a combination of the following methods for data collection:

Document Review – three major categories of documentation will be examined:

Documents that constitute formal agreement among project partners and/or record progress; such as the ESP project document, meeting minutes, the monitoring and evaluation framework, the approved project Work Plan, periodic reports, as well as reports prepared by contractors to document the progress of their commissioned work;

Publicly available information such as opinion polls, reports from independent observers that document electoral performance (including some that were funded by the project), including e.g. reports from Media monitoring and international and domestic observer reports, documentation prepared by political parties, the courts and other interested parties during and in the immediate aftermath of the elections;

Internal working documents or other documents produced during the course of implementation, such as monitoring reports, training materials, mission reports, consultancy reports, speeches, presentations, etc. (as applicable).

The consultants may also be guided by documents describing recognized international standards for free and fair elections, in order to assess the project's results. These documents are also available in the public domain.

Key Participant Interviews – the evaluation should include interviews with key stakeholders:

• Senior management and other key focal points in NEC and ZEC;

Other national stakeholders with a mandate related to the elections (e.g. political parties, the Police, the RPP, etc.);

• Representatives of sub-contracted parties of major components;
• National electoral expert commentators, for example domestic observers
• International long-term observers in Zanzibar;

Members of the technical committees and project steering committee, representatives of project donors;

• UNDP staff and management, including key project personnel;

Quantitative/Cost-benefit analysis – the evaluation will analyze data on the cost and benefits of project results, to enable conclusions to be drawn about the project's contribution to value for money of the funds provided.

Structured surveys and semi-structured interviews – project staff and key informants no longer in the country could be approached by telephone, e-survey or both. The evaluators may also wish to use this tool for other data collection purposes.

Stakeholder Consultations – Consultations with groups of stakeholders i.e. media, CSOs, Development Partners, Political Parties, National Observer Organizations, etc.

Identification of Best Practices and lessons Learned – the evaluation shall seek to extract good practices, innovation, and other elements that deserve replication; and identify any lessons learned in support of future assistance to elections.

In conducting data analysis and presenting the findings, the evaluation should use a combination of qualitative and quantitative methods. While the evaluation is expected to estimate project impact, it is understood that the evidence of impact may not be available for all components; and further that the methodological rigour usually associated with impact evaluation may not be applicable to this evaluation.

Outputs and Deliverables

The evaluation team shall produce the following deliverables through the course of the assignment:

Work plan – within two days of the start of the assignment. The document will include a detailed approach and methodology, schedule, and a draft data collection protocol. In particular, the work plan will require a clear approach to data collection and work organization to examine the project in its full scope. The work plan will be evaluated by UNDP for quality and content, upon which clearance to proceed with the full evaluation will be granted.

Preliminary findings report – within 20 days of the start of the assignment, the consultants will share a draft report indicating the scope of assessment in key components for validation by UNDP and other stakeholders. The purpose of this report is to demonstrate progress on the assignment and adherence to the TORs, and will identify any evaluation issues that may need further clarification before completion of the in-country mission.

Presentation of findings, and draft evaluation report – within 28 days of the start of the assignment, a presentation of findings and preliminary recommendations to key stakeholders orally and in writing will be made at or prior to completion of the in-country mission. The purpose of this session is to provide opportunity for initial validation and support further elaboration of the evaluators' findings and recommendations. The quality of the report and draft findings will be assessed by stakeholders.

Final evaluation report – within two weeks of receiving the consolidated comments from stakeholders, the Evaluation Team will submit a final document that addresses relevant comments and provides comprehensive reporting on all elements of the assignment. This report will be submitted to the ESP Steering Committee for clearance.

A lump sum payment will be effected on the basis of fully satisfactory and timely delivery on all deliverables. The report will be considered satisfactory if it addresses all elements in the TOR and has been endorsed by the project Steering Committee, with representation of key stakeholders.

Evaluation Report Outline

As a minimum, the Evaluation Report (draft or final) shall include the following components (the exact structure of the report may be influenced by the project components and components of the Evaluation TOR):

• Executive Summary
• Introduction
• Project outline
• Methodology
• Analysis
• Findings
• Best Practices and Lessons Learned
• Recommendations
• Relevant Annexes, for example:

List of people interviewed

List of acronyms

Evaluation work plan and TOR

List of key reference documents

Time Frame for the Evaluation Process

The Evaluation is expected to start in September 2011 and have an expected total duration of 35 working days. The final work plan will be confirmed by the Evaluation Team and UNDP upon submission of a draft work plan for discussion.

Indicative Work Requirement

Orientation to the assignment, initial document review, and preparation/discussion of the Evaluation Plan (2 days)

Detailed document review at component level and interviews with key project personnel (7 days)

Stakeholder consultations, external and internal, including the preparation of surveys etc. (8 days)

• Quantitative analysis (5 days)

Analysis and preparation of draft evaluation findings (6 days)

• Debriefings (2 days)

Preparation of Final Evaluation Report, including addressing comments from the stakeholders on the first draft (5 days)

Total: 35 working days

It is expected that the mission will spend 3-4 weeks on in-country work (including domestic travel time), and that remaining time will be allocated for the production of evaluation deliverables.

Competencies

UNDP wishes to contract experienced evaluators to undertake the evaluation. The A team of at least three experts will undertake the evaluation. The Team Leader will lead, organize, and supervise the work of the evaluation team, ensuring a division of labor that is commensurate with the skills profiles of the individual team members. He or she will have overall responsibility for the production of deliverables, in particular the evaluation report, and is ultimately accountable for its quality. The Team Leader is also responsible for ensuring adequate consultations with all stakeholders and for reporting to UNDP on progress.

As a minimum requirement for the team composition, at least one of the members of the evaluation team must have demonstrated strong understanding of electoral assistance; at least one of the members must have demonstrated experience conducting independent evaluation of large-scale donor projects, at least one member must have strong knowledge of broader democratic governance issues, and at least one of the members must have demonstrated proficiency in value for money analysis.

The evaluators must not have had any involvement in the design or implementation of this elections support project and have no present affiliation with UNDP, its funding partners, electoral management bodies in Tanzania, or other key project stakeholder organizations that in any way could jeopardize their objectivity in relation to the assignment.

Contracts will be output-based and payment issued only upon delivery of satisfactory outputs.

Evaluation Principles and Ethics

The evaluation should be conducted in accordance with the principles laid out in UNDP Evaluation Policy and the principles outlined in the UNEG 'Ethical Guidelines for Evaluation'

Implementation Arrangements

The Team Leader will report to UNDP Senior Management, who will provide guidance in the development of the work plan and ensure the monitoring of satisfactory completion of evaluation deliverables. UNDP may provide office space and access to standard office services as needed. The Evaluation Team should provide their own computer and communications equipment.

In consultation with the Evaluation Team Leader and as requested, UNDP personnel will make available all relevant documentation and provide contact information to key project partners and stakeholders, and facilitate contact where needed.

Time Frame for the Evaluation Process

Required Skills and Experience

Evaluation Team Leader:

University degree in a related area (e.g. political science, international development) with minimum 15 years post-degree work experience.

Demonstrated capability as an effective evaluation manager with experience leading and conducting international development evaluation of large development projects in the area of democratic governance.

Experience from implementation or evaluation of least one election support project and familiarity with project implementation in complex multi donor-funded projects.

Experience with value for money assessment or similar cost-benefit analysis.

Strong competencies in Monitoring and Evaluation methods for development projects; knowledge of UNDP's results-based management orientation and practices.

Fluency in English and excellent oral and written communication skills.

Experience in Sub-Saharan Africa an asset.

International Expert, Democratic Governance :

At least 5 years experience in the design, implementation, management, and/or evaluation of democratization projects in developing countries including at least one election support project.

Strong knowledge in some other key areas related to democratic governance such as political parties, the media, civil society, advocacy/education, human rights, women's empowerment, legal sector , conflict prevention , and/or peace and development.

Demonstrated knowledge of capacity development methods and approaches.

Familiarity with multi donor-funded projects.

At least a Master's Degree in a relevant area with minimum 7 years post-degree work experience.

Fluency in English and excellent oral and written communication skills.

Experience in Sub-Saharan Africa an asset.

International Expert, Project Financial management:

Master's Degree in Finance or Business Administration or relevant area with minimum 5 year's post-degree work experience.

Demonstrated experience in value-for-money financial analysis.

Demonstrated knowledge and experience of project management and design and/or evaluations.

Good knowledge of monitoring and evaluation methods

Fluency in English and excellent oral and written communication skills.

UNDP is committed to achieving workforce diversity in terms of gender, nationality and culture. Individuals from minority groups, indigenous groups and persons with disabilities are equally encouraged to apply. All applications will be treated with the strictest confidence.