Evaluation of Joint Programme 11 on Environment with Focus on Climate Change, Land Degradation/Desertification and Natural Resources Management

Tanzania’s natural resources are the main source of peoples’ livelihoods and backbone of the country’s main productive sectors. The relationship between economic development and rational management of the environment and its natural resources is emphasized in the National Environmental Policy, (NEP) 1997 and the National Strategy for Growth and Reduction of Poverty (NSGRP) or MKUKUTA/MKUZA in Kiswahili. The NEP identifies land degradation, loss of wildlife habitat and biodiversity, ecosystem deterioration and deforestation as critical environmental problems, requiring urgent intervention. The impacts of poverty and climate change are likely to further aggravate these problems, thus increasing the vulnerability of both urban and rural communities, as well as that of the natural resources and the environment providing livelihood to the people. The Millennium Development Goal 7 also recognizes the importance of addressing the adverse impact of environmental degradation on people’s livelihoods. To respond to these critical problems, the Joint Programme (JP11) aimed at increasing the capacity of the Vice President’s Office (VPO), the Prime Ministers’ Office Regional Administration Local Government (PMO-RALG) and the Ministry of Finance and Economic Affairs (MoFEA) to coordinate and lead the implementation of key national environmental policies and plans. The Joint Programme therefore focused on mainstreaming environmental sustainability and climate change - including disaster preparedness and risk reduction - in the policies, strategies, programmes and plans of key sectors: Energy, Water, Natural Resource Management, Agriculture and Livestock Development &Fisheries. Recognizing the importance of working with local communities and local government in order to achieve concrete results, this Joint Programme ensured that LGA’s plans and budgets address local environmental priorities and integrate environmental management issues. The UN capitalizes on its extensive experience in policy and technical support and propose various strategies for the delivery of the objectives of the JP 11. In addition, the programme promoted linkages with other Joint Programmes, particularly the JP on Wealth Creation, Capacity Development and Disaster and Risk Reduction. Tanzania is one of eight countries worldwide to pilot the One UN reform initiative in 2007-2008. In Delivering as One (DaO) context, the UN had a far greater impact on the development and humanitarian challenges that faced Tanzania, avoiding the fragmentation and duplication of efforts seen in the past. By combining the skills and resources of UN Agencies active across the country and by jointly addressing key development issues alongside others, the UN complemented the efforts of Government, civil society, the private sector and other development partners by focusing on what it does best, using available resources as judiciously as possible and reinforcing the leadership of other actors through technical support and modest financial contributions. All UN agencies and offices worked more closely together, also with external partners, to achieve joint results in relation to growth and poverty reduction. The vision of ‘One UN’ in Tanzania is therefore about being a more effective partner for Tanzania’s development; focusing on the UN’s comparative advantage; being strategic about the development areas where the UN prioritizes support; and continuing to support Government on humanitarian assistance in a way that over time ensures national capacity is strengthened on the same. JP11 provided opportunity for the UN to work in a harmonized, coherent and coordinated manner and become more cost-effective while delivering greater development impact and minimize fragmentation and inefficiency. As a result, UN Agencies developed a common work plan and budgetary framework and use of common business processes such as procurement, financial reporting system and procedures (HACT) to reduce transaction costs. The UN Joint Programme on Environment supports the Government and civil society to build capacity to overcome ecosystem degradation and loss of biodiversity in Tanzania. The support will cover three main areas of national priorities as articulated in the MKUKUTA/MKUZA and JAST to ensure: Capacities of the national and local government to coordinate, lead and implement environmental policies and plans are developed; Mechanisms for environmental sustainability and climate change in key sectors, strategies and policies, including disaster risk reduction are in place; Capacity for government and the private sector to access international funding for adaptation and mitigation of climate change is built; The Joint Programme assisted Tanzania’s progress towards achieving goals of MKUKUTA/MKUZA, as well as the MDGs, by supporting government’s efforts to address challenges as articulated in the following key policy and programming documents which as outlined below MKUKUTA CLUSTER I (Goal 2 and 4): Broad based and equitable growth is achieved and sustained MKUKUTA CLUSTER II (Goal 3): Improved Quality of Life and Social Well-Being with a particular focus on the poorest and most vulnerable groups MKUKUTA CLUSTER I & II (Goal 1 and 2): Structures and systems of governance as well as rule of law are democratic, participatory, representative, accountable and inclusive and equitable allocation of public resources with corruption effectively addressed. As the Managing Agent (MA), UNDP has ultimate responsibility and accountability for both the achievement of results and management of funds since it is responsible for overall coordination of the programme. The MA is responsible for reporting matters including (1) preparation of consolidated narrative and financial documents and reports for review by the Joint Programme Committee (and subsequent submission to the Joint Steering Committee); (2) progress reporting to governance structures; and (3) financial reporting to the One UN Fund’s Administrative Agency (AA). Joint Programme which started in 2007 with funded budget (from One UN fund) of US dollar 3,474,000 comes to an end in June 2011. A terminal evaluation is planned to be conducted to assist the UN and programme stakeholders to draw lessons learned in implementing the programme and improve the quality of future development interventions. As the UN in Tanzania moves towards the UNDAP, the first all-inclusive One UN business plan for Tanzania, the review of the JP11 will also provide opportunity to reflect on the challenges in the management and implementation of such multi-stakeholders programmes and inform the formulation and implementation of similar programmes and partnerships which will derive from the UNDAP.

Duties and Responsibilities

The main purpose of the evaluation is to provide an independent in-depth assessment of the achievements of results as well as the implementation arrangements of the Joint Programme on environment with a focus on climate change, land degradation/desertification and natural resources management (JP11) with a particular focus on effectiveness. Thus the evaluation specific objectives are: Based on planned deliverables of the Project Document (Result and Resource Framework (RRF) the consultants should evaluate the project results (that have/have not been achieved and a special emphasis should be placed on measuring the achievements or non-achievements of the expected result of outputs under all components of the project) of the amended and prioritized activities; Explore to what extent synergies among UN agencies in particular and among IPs were explored and effected in implementation; What extent the UN was able to go upstream as intended; Assess whether capacities was indeed imparted and evaluate the likely impact the improved capacities; Evaluate the effectiveness and efficiency of the JP modality used in the implementation of JP11 To draw experience and lessons learnt from JP11 with respect to its structure, management, and implementation arrangements as a strategy for supporting capacity strengthening in Tanzania. Consolidate lessons learned with a view to contribute to improving the future UNDAP implementation strategies and make recommendations to guide future programming for the Delivering As One Scope of the Evaluation The evaluation will review and assess the achievement of the programme. The review should include an analysis of an assessment of (a) the key factors that could have affected the progress towards achieving the outputs/outcomes, (b) the programme (JP11) strategy in general (c) Relevance of priorities that have been included in the programme and what is missing. The evaluation team will review and assess the achievement of the programme outcomes focusing on: Progress status: What was the basis of the outputs/outcome and its constituent interventions? Were past experiences and lessons as well as dialogue with stakeholders in design of the programme and outputs considered? Assess the adequacy of the background work carried out. Determine the degree to which outputs/outcomes were achieved and, if not whether there was progress towards their achievement. Underlying factors: An analysis of the underlying factors that have influenced the outputs and achievements. What were the key internal and external risk and assumptions made? Distinguish the substantive design issues from the key implementation and/or management capacities and issues including the timeliness of outputs, the degree of stakeholders and partner’s involvement in pursuit of the outputs, and how processes and activities were managed / carried out. JP11 strategy: Ascertain whether JP11 was appropriate and effective. Assess the initial design modalities and planning process as well as management mechanisms and identify the role of each party involved and how did it function and how has it been sustained? How did each party understand its role? How did the management arrangements contribute to (or hinder) the achievement of the outputs/outcome? What was the level of participation of the stakeholders and the perception of the beneficiaries? In addition, the evaluation should help to address and endeavour to respond to the industry standard OECD/DAC evaluation criteria, encompassing five key areas of assessment. Relevance /Appropriateness – the extent to which the JP pertained to national priorities and the requirements of the target group. Indicative questions might include: To what extent were partners involved in the development and implementation of the JP? To what extent are the objectives of the programme still valid? Were the activities and outputs of the programme consistent with the intended key results? Effectiveness - the extent to which the JP attained its key results. Indicative questions might include: To what extent were the key results achieved? What were the major factors influencing the (non)achievement of the desired results, including institutional, management and fiscal arrangements? Efficiency – the extent to which delivery was undertaken by the most cost-efficient means. Indicative questions might include: Were activities cost-efficient? Were outputs achieved on time? Was the programme implemented in the most efficient way compared to alternatives? Impact - the positive and negative changes produced by the JP (directly or indirectly, intended or unintended), Indicative questions might include: What difference has the activity made to the beneficiaries? How many people have been affected? How did impact differ across key target groups, including vulnerability categories such as gender, age etc? Sustainability - measuring whether the benefits of the JP are likely to continue after the end of programme activities. Indicative questions might include: To what extent will the benefits of a programme continue after activities have ceased? What are the major factors which will influence the sustainability of the programme? The evaluation should also respond to three additional assessment areas. Delivering as One - adherence to the principles and objectives of Joint Programming (include reference to the One UN Process Indicators). Indicative questions might include: To what extent were the principles of Joint Programming in terms of collaborative planning, implementation and monitoring across participating agencies adhered to? What factors facilitated or adversely impacted upon Delivering as One? Cross Cutting Considerations – application of the programming principles of the UN, referencing Human Rights Based Approach, Gender Equality, Environmental Sustainability, Results Based Management and Capacity Development. Indicative questions might include: Were cross-cutting considerations mainstreamed in the implementation of activities? To what extent did the programme involve the host communities and other stakeholders in programme design and implementation? Were capacity development activities informed by a capacity assessment at manifold levels? Lessons and Recommendations for future programming. Indicative questions might include: What additional measures (if any) could have improved the relevance, effectiveness, efficiency, impact or sustainability of the JP? What lessons can be applied in terms of collaborative programme planning, implementation and monitoring to the implementation of the UNDAP 2011-2015? What key measures can the UN in Tanzania, regionally and at HQ, plus the Government of Tanzania and Development Partners (DPs) adopt to improve the implementation and performance of the One Programme under DaO? Methodology The evaluation will utilize both qualitative and quantitative methodologies. Data and information to be analysed during the evaluation will come from various related documents and key stakeholders. The evaluation will therefore involve:- Inception – includes creation and approval of a detailed evaluation workplan and budget Data Collection – encompasses both document review and field visits Desk review of (i) all relevant documents relating to the programme e.g. the programme document, annual work plans, progress reports, monitoring reports, expenditure reports (ii) all relevant documents relating to the Delivering as One (DaO) initiative and joint programming e.g. One programme document, DaO report (iii) national and Government documents that provide the context to the overall capacity issues in Tanzania e.g. MKUKUTA, capacity development strategy. Interviews with key partners and stakeholders. These will include participating UN agencies, relevant donors, implementing partners (which include Government) and other beneficiaries, interviews with the Joint Programme Committee (JPC) Field visits to selected programme sites and discussions with beneficiaries Analysis and Reporting – incorporates presentation of findings, delivery of a draft, capturing of feedback and production of final written report At every stage of the evaluation process, the following principles should be observed: Independence - the evaluation team should be independent from the operational management and decision-making functions of the JP Impartiality – the evaluation information should be free of political or other bias and deliberate distortions Timeliness - evaluations must be designed and completed in a timely fashion Purpose - the scope, design and plan of the evaluation should generate relevant products that meet the needs of intended users Transparency - meaningful consultation with stakeholders should be undertaken to ensure the credibility and utility of the evaluation Competencies - evaluations should be conducted by well-qualified teams. The teams should, wherever feasible, be gender balanced, geographically diverse and include professionals from the countries or regions concerned. Ethics - evaluators must have professional integrity and respect the rights of institutions and individuals to provide information in confidence and to verify statements attributed to them. Evaluations must be sensitive to the beliefs and customs of local social and cultural environments and must be conducted legally and with due regard to the welfare of those involved in the evaluation, as well as those affected by its findings. Quality - All evaluations should meet the standards outlined in the Standards for Evaluation in the United Nations System. The key questions and areas for review should be clear, coherent and realistic. The evaluation plan should be practical and cost effective. To ensure that the information generated is accurate and reliable, evaluation design, data collection and analysis should reflect professional standards, with due regard for any special circumstances or limitations reflecting the context of the evaluation. Evaluation findings and recommendations should be presented in a manner that will be readily understood by target audiences and have regard for cost-effectiveness in implementing the recommendations proposed. There will be briefing and debriefing sessions with the UN agencies, Implementing Partners and other key stakeholders. The evaluation will led by one national and one international consultant. The lead International consultant, after brief orientation, will develop a plan of action stating the methodologies and required resources for the end of programme evaluation. In the plan of action, areas of evaluation, indicators and data collection methods should be clearly spelled out. Key Deliverables (Evaluation products) The evaluation team will be accountable for producing the following products:- An Inception Report, providing A detailed Evaluation Workplan and Budget Literature Review Evaluation Questions and observations on their evaluability Identification of key informants Data collection methods and reasons for selection (with reference to the key informants) The inception report should detail the evaluators’ understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods; proposed sources of data; and data collection procedures. The inception report should include a proposed schedule of tasks, activities and deliverables, designating a team member with the lead responsibility for each task or product. A presentation of initial findings and provisional recommendations- at the end of the field work, the evaluation team will present their draft findings and provisional recommendations. The evaluation team will also make a PowerPoint presentation summarizing the main findings recommendations and lessons learned and conclusions A draft report for comments by stakeholders. This should incorporate: Title and opening pages Table of Contents List of acronyms and abbreviations An Executive Summary Introduction Details of the JP activities and desired key results Scope of Evaluation Evaluation Methodology and Guiding Principles Data analysis Findings Lessons Learned Recommendations Methodological constraints Additional background data-Annexes (including interview list, data collection instruments, key documents consulted, ToR) A draft report should be provided 10 days before the end of the consultancy period requesting for comments from identified stakeholders to allow enough time for incorporation of comments received. A final evaluation report, encompassing all key sections required in the draft report and additional stakeholder feedback. The final report should be clear, understandable to the intended audience and logically organized based on the comments received from stakeholders. The evaluation team shall submit four hard copies and an electronic version of the final report The final evaluation report should be presented in a solid, concise and readable form and be structured around the issues in the Terms of Reference (ToR), four hard copies and an electronic version of the final report, 3 days before the end of consultancy period. The consultant should refer to annex 7 of the UNDP Planning, Monitoring and Evaluation handbook for details on reporting template. Management and Implementation Arrangements The evaluation will be under the supervision of the Team Leader, Energy and Environment Unit – UNDP, and the evaluation team will report to him/her. The UNDP Deputy Country Director (Programmes) will provide overall guidance to the process. UNDP for that matter will be responsible for supporting the evaluation team in the following:- Appoint a focal person in the programme section to support the consultant(s) during the evaluation process. Collect background documentation and inform partners and selected programme counterparts (including Government) Meet all travel related costs to programme sites outside Region (Dar es Salaam) Provide background document which will help to identify key stakeholders to be interviewed as part of the evaluation. Organize and meet costs related to stakeholder workshops during consultation and dissemination of results Organize consultative meetings between the consultants and stakeholders, including Government Provide office/working space in the course of the assignment. The consultants will however have to use their own computers/laptops In addition, it should clarify who is responsible for: Assessment of tenders and management of recruitment process Liaison with the evaluation team Providing technical guidance and appropriate supporting documents Co-ordinating the stakeholders involved, including scheduling of visits and briefing events Selection and orientation of team members Approval of intermediate and final products, including quality assurance Preparation of a management response to the final Evaluation report Provision of logistical support (and type of) Ensuring adherence to UNEG norms and standards Evaluation Ethics The evaluation will be conducted in accordance with the principles outlined in the UNEG ‘Ethical Guidelines for Evaluation’. Critical issues that evaluators must safeguard include the rights and confidentiality of information providers in the design and implementation of the evaluation. Time Frame The evaluation exercise will involve a total number of 30 paid consultancy days: Preparation and submission of the Evaluation Inception report to stakeholders (4 working days) Presentation of preliminary findings (1 working day) Collection of data/information, analysis, preparation of the Draft Evaluation report and Submission of draft report, de-briefings to UN Agencies and key stakeholders (18 working days) Incorporation of stakeholder comments and submission of Final Evaluation report (7 working days) Responsibilities Designing the evaluation according to the specific terms of reference; gathering data from different sources of information; analyzing and systematizing the information; identifying patterns and causal linkages that explain current performance; drafting evaluation reports at different stages (inception, drafts, final); responding to comments and factual corrections from stakeholders and incorporating them, as appropriate, in subsequent versions; addressing comments by the external Quality Assurance Panel; and making briefs and presentations ensuring the evaluation findings, conclusions and recommendations are communicated in a coherent, clear and understandable manner once the report is completed. All reports shall be written in English. The Evaluation team is responsible for editing and quality control. The final report should be presented in a way that directly enables publication.

Competencies

Through consultations with the Government and PUNs, UNDP will recruit a team of independent consultants on a competitive and transparent basis as per standard UNDP procedures. The evaluation team will be independent from the policy-making process the delivery and management of assistance. The evaluation team shall consist of 2 consultants: an international consultant (Team leader) and a national consultant with extensive knowledge of the country situation. The Team Leader will have the overall responsibility for the co-ordination of the evaluation process, quality and timely submission of the evaluation report.

Required Skills and Experience

The international consultant should possess: A Masters degree in international development, public administration, social sciences, evaluation or a related field A minimum of 10 years of professional experience specifically in the area of evaluation of international development initiatives and development organizations Substantial international track record of conducting different types of evaluations, including process, outcome and impact evaluations in different countries and organizations Experience in monitoring and evaluating cross-cutting issues such as human rights and gender Knowledge and experience of the UN System and the UN Reform process Understanding of the development context in Tanzania and/or other ‘Delivering as One’ countries would be a clear advantage Excellent communication and interview skills Excellent report writing skills Demonstrated ability to deliver quality results within strict deadlines Proficiency in English (written and spoken) The national consultant should have the following qualifications: A Bachelors degree in international development, public administration, evaluation or related field. A minimum of seven years of professional experience, specifically in the area of monitoring and evaluation of international development initiatives and development organizations. A track record of conducting various types of evaluations, including process, outcome and impact evaluations in Tanzania and preferably in the region. Experience in monitoring and evaluating cross-cutting issues such as human rights and gender Knowledge and experience of the UN System and the UN Reform process. In-depth understanding of the development context in Tanzania. Excellent communication and interview skills. Excellent report writing skills. Demonstrated ability to deliver quality results within strict deadlines Proficiency in English (written and spoken)

UNDP is committed to achieving workforce diversity in terms of gender, nationality and culture. Individuals from minority groups, indigenous groups and persons with disabilities are equally encouraged to apply. All applications will be treated with the strictest confidence.

Evaluation of Joint Programme 11 on Environment with Focus on Climate Change, Land Degradation/Desertification and Natural Resources Management