EVALUATION OF THE JOINT PROGRAMME 3 - SUPPORT TO THE NATIONAL RESPONSE AGAINST HIV AND AIDS

This United Nations Joint Programme of Support to Tanzania on HIV and AIDS 2007-2011 was prepared based on the United Nations Development Assistance Framework (UNDAF) in Tanzania and Government priorities as expressed through the national development framework of MKUKUTA and MKUZA, as well as the National Multi-Sectoral Strategic Framework (NMSF) on HIV and AIDS for Tanzania mainland developed by Tanzania Commission for AIDS (TACAIDS) and Zanzibar’s National HIV and AIDS Strategic Plan (ZNSP) developed by Zanzibar Commission for AIDS (ZAC). The Joint Programme reflects the joint UN support to the national response on HIV and AIDS for Tanzania in five main areas as provided in the NMSF for Tanzania mainland and ZNSP for Zanzibar: prevention; care, treatment and support; impact mitigation; enabling environment and monitoring and evaluation. The interventions to be addressed are articulated in the various Annual Work Plans on HIV and AIDS developed jointly by the UN joint team and government under the leadership of TACAIDS for Tanzania Mainland and ZAC for Zanzibar As outlined in the Secretary General’s High Level Panel Report – on ‘Delivering as One’, and the commitment of the UN system in Tanzania the joint programme was formulated to improve development effectiveness and impact through inter-agency collaboration and synergy. With Tanzania being selected as one of the eight countries globally to put into practice the concept of ONE UN, the agencies decided to work together to implement a UN Joint Programme on HIV & AIDS. The vision of the Joint Programme is to “build partnerships to provide an effective and sustained national response to HIV and AIDS as ONE UN in line with MKUKUTA and MKUZA to support universal access to prevention, care, treatment and support in Tanzania”. Tanzania declared HIV & AIDS a national disaster at the end of 1999. As a result responses to contain the epidemic were developed and integrated in the first generation PRSP for mainland and Zanzibar, as well as, the current NSGRP/MKUKUTA and ZPRP/MKUZA, together with Gender and environmental measures. Another step was the establishment of the “Three Ones” - One coordination authority, One national multi-sectoral plan, and One M&E framework – that have been put in place, both on Tanzania Mainland and Zanzibar, and being implemented by both Governments with support from development partners. Despite all these efforts, new infections were on the rise, particularly among the youth, women and girls, in urban as well as rural areas, putting in danger the attainment of the MDGs and national poverty reduction targets. Following agreements among UN Agencies and its national partners, the joint programme was formulated with a bias on HIV prevention. Specifically, the programme and work plans for the mainland and Zanzibar will pursue an evidence-based approach towards the prevention of HIV transmission through support to: 1) advocacy and policy dialogue; 2) communications for behaviour change; 3) capacity development for HIV prevention; 4) reproductive health and commodity security; and 5) life skills development and livelihood security. While the joint programme focus on combating new HIV infections, the UN will continue, through individual agencies, to offer support in the areas of Care, Treatment and Support, Impact Mitigation, Enabling Environment and cross-cutting issues in two ways: 1) through the ongoing programme activities of its agencies and their partners in Government, civil society organizations, and the private sector that fall outside of the Joint Programme; and 2) through its continued attention to advocacy and fund-raising across all four areas. Key expected results as per UNDAF Outcome(s): Increased access to comprehensive prevention, care and treatment and impact mitigation of HIV & AIDS and other major diseases Effective mechanisms, including social protection in place, that address institutional barriers and socio-cultural dimensions to promote and protect the rights of the poor and most vulnerable including those affected by HIV & AIDS Enhanced and accessible systems of justice, legal, public information and education that promote and protect human rights and freedoms Increased and equitable access to comprehensive reproductive and child health facilities Increased and equitable access to quality formal and non-formal education including for those affected by HIV & AIDS Expected Output(s): Gender sensitive strategies and programmes to combat HIV & AIDS at the workplace based upon ILO Code of Conduct effectively implemented in public, private and informal sector Advocacy and Communication Strategies developed and implemented to address stigma & discrimination; and gender relations that render women and girls vulnerable to HIV infection Increased capacity and opportunities for women, the most vulnerable children and adolescents as well as PLHAs to participate in governance at all levels, including the Three Ones, and have their views heeded. Increased awareness, knowledge, skills and services of HIV & AIDS prevention among most vulnerable communities achieved through district and community-owned HIV & AIDS interventions More outputs were developed and defined in each AWP in line with priorities of TACAIDS and ZAC and other partners. The management of the JP3 was entrusted to UNDP as Managing Agent (MA) and overall coordination to UNAIDS. A joint UN team was tasked with technical oversight of the JP3 under UNAIDS. Lead agencies were identified as per DoL and agencies’ mandates: WHO for Care and Treatment, UNICEF for Impact mitigation, UNFPA for Prevention, UNAIDS for Enabling Environment and UNDP as Managing Agent (MA). A Joint Programme Steering Committee was established under the co-leadership of TACAIDS and ZAC with UNAIDS providing the secretariat for policy guidance, direction and decision making. The JP3 planning is done jointly with participating UN agencies and government under leadership of TACAIDS and ZAC. Annual Work Plans (AWP) are prepared jointly every year and approved by Government and MA. The funding of the JP3 was at first through a pooling system by contribution of the participating UN agencies to the MA. With the establishment of the One Fund under DaO, the MA became responsible for management of One Fund only and each participating UN agencies was left to manage their own funds through a parallel mechanism.

Duties and Responsibilities

The JP3 evaluation is an independent evaluation to be completed prior to the completion of the programme in order to provide inputs to the development of the United National Development Assistance Program (UNDAP 2011-2015) as well as any specific activities or programs which might succeed the JP3 in support of the national response against HIV/AIDS in Tanzania. The purpose of this evaluation is to assess the achievement of results (outputs and outcomes) and management of the JP3 on support of the HIV and AIDS national response in Tanzania mainland and Zanzibar. More specifically, the evaluation aims to: Assess the appropriateness, relevance, effectiveness, efficiency, impact and sustainability of the Joint Programme 3; Identify lessons learned and draw conclusions which may inform the new UNDAP programme on HIV/AIDS and which can contribute to decision making processes of the UN, GoT and partners. Assess the planning and monitoring of the JP3 and its AWP: the process and methodology used, providing evidences and quality of involvement of each of the 2 Governments aiming at measuring the gained capacity; Assess the coordination of the sub recipients by ZAC and TACAIDS for all JP3 related activities; Assess the funds management/allocation by UNDP for the One Fund and UN Agencies for their parallel resources: system used, its constraints and its advantages. The primary audience for the JP3 evaluation is the Government of Tanzania (GoT), other responsible partners (MDAs, LGAs, CSOs including PLHIV and the private sector) and the UN in Tanzania. Secondary audiences include interested parties in Tanzania (implementing partners and development partners), other UN DaO Pilot countries and UNHQ. The evaluation involves the systematic and objective assessment of the entire JP3 in both Tanzania mainland and Zanzibar, encompassing the key Implementing Partners – TACAIDS and ZAC respectively, as well as the responsible partners on mainland and Zanzibar such as the Ministry of Health and Social Welfare, Ministry of Education and Vocational Training, Prime Minister’s Office Regional Administration and Local Government, LGAs in Kigoma, Kagera, Shinyanga, Tabora, Coast, Lindi and Dar Es Salaam; CSOs (NACOPHA, ZAPHA+, AMICAALL, ABCT, ABCZ, etc), its design, implementation and results since its inception. This assessment is made of the entire programme regardless of funding source (One Fund or PUN parallel funds). Based on the objectives of the evaluation, the following criteria and questions are expected to guide without limiting the evaluation team in creation of the evaluation framework, information collection and actual assessment: Relevance/Appropriateness How relevant were the JP3 outcomes and outputs in assisting the achievement of Tanzania’s national development goals? To what extent were partners involved in the development and implementation of the JP3? What evidence is there for evidence based programming (i.e. information generated from situation analysis translated into a response in an appropriate manner). Efficiency To what extent did UN agencies plan together? Did the plans demonstrate coherence and delivering as one? To what extent were the management and administrative set up necessary and adequate to deliver the Joint Programme? To what extent was the Joint Programme cost effective, in terms of reducing transaction costs internally as well as externally? Were the institutional and management arrangements supportive to attainment of the intended objectives? Did the PUNs position the appropriate human resource capacity to reflect the strategic orientation of the programme? How efficiently has the JP3 delivered its outputs and outcomes? How were the funding modalities used by the UN appropriate, including use of national systems for disbursement? Effectiveness To what extent were the key results achieved? What were the major factors influencing the (non)achievement of the desired results, including institutional, management and fiscal arrangements? Programme Outcome/Impact: What results (outputs and outcomes) has the JP3 achieved since its commencement? Has the programme made a difference to the lives of host communities? How are impacts different according to gender, age and other vulnerability category (e.g. women, girls, children and key populations such as drug users and sex workers)? Sustainability: How sustainable is the progress made and achievements of the JP3? What are the major factors which have influenced the relevance, effectiveness, efficiency and sustainability of the JP3? What approaches are the JP3 PUNs taking to the building of national capacities? How effective are these approaches? Partnerships and Coordination What are the key lessons learned from the JP3 and how can they inform future UN, GoT and DP decision making processes? What factors favorably or adversely affected the spirit of delivering as one? What links exist between national (TACAIDS/ZAC) and field level coordination (including the CMACs and Regional Facilitation Teams)? Are there any critical technical gaps in the coordination structures? Cross-cutting issues: Gender, Human Rights, Capacity Development Were cross – cutting considerations mainstreamed in the implementation of activities? To what extent did the programme involve the host communities and other stakeholders in programme design and implementation? Were the capacity development activities informed by capacity assessment at different levels? To what extent was the local context (e.g. gender) taken into consideration when the JP3 was designed? Delivering as One: To what extent were the principles of Delivering as One in terms of joint programming, planning, implementation and monitoring across PUN agencies adhered to? To what extent did the program adhere to the performance indicators outlined in the One Programme document and the Delivering as One vision? What factors facilitated or adversely impacted upon Delivering as One? Lessons and Recommendations for future programming: What additional measures (if any) could have improved the relevance, effectiveness, efficiency, impact or sustainability of the JP3? What lessons can be applied in terms of collaborative programme planning, implementation and monitoring to the implementation of the UNDAP 2011-2015? What key measures can the UN in Tanzania, regionally and at HQ, plus the Government of Tanzania and Development Partners (DPs) adopt to improve the implementation and performance of the One Programme under DaO? Evaluation Methodology and Guiding Principles Outlined below are the expected evaluation process, timeframe, methodology and costs. Evaluation Process The evaluation process is expected to contain three phases: inception, data collection; and analysis and reporting. Inception Phase - the evaluation team will review existing documentation, meet with the Reference group members and prepare a brief inception report (which includes a clear evaluation work plan and tools). Data Collection – the evaluation team will gather data from sources such as interviews with key informants and review of relevant documents. Analysis and Reporting – the evaluation team will analyze data and prepare the evaluation report over a three week period. This involves preparing the preliminary report based on the analysis of findings, submitting the report to key stakeholders for factual review as well as presenting the findings and conclusions to stakeholders and gathering feedback. From this process the final report with sections e.g. a minimum 2 pages executive summary containing key findings and recommendations; a maximum of one page background; an Introduction; the approach employed for data collection and analysis; findings, conclusions; recommendations; Lessons Learned and annexes will be prepared and presented to relevant stakeholders in Dar Es Salaam. Evaluation methods During the Inception phase, the evaluation team will formulate in detail the methodology for the evaluation which will be informed by the key methodological principles below: The evaluation is results focused; The evaluation will adopt a consultative, iterative and transparent approach with stakeholders; Triangulation of information and data across groups of stakeholders and individuals will be the key method to validate evidence, throughout the whole evaluation process; The evaluation is cost-effective; The evaluation will strictly adhere to the UNEG Norms & Standards. The evaluation will use a wide range of methods and tools, adapted to the national context and to the evaluation questions. They may include, but are not limited to, the following: Debriefing sessions with the stakeholders (inception, data collection, analysis phases); Evaluation matrix relating evaluation issues and questions to evaluation criteria, indicators, sources of information and methods of data collection (inception phase); Mapping exercise of the main focus areas of the Joint Programme work (inception phase); Desk review of reference documents (inception and data collection phase); Individual and group interviews with stakeholders including representatives from Government, JP Steering Committee (and other JP management structures), JP Implementing partners, DPs, UN Agencies, UN Resident Coordinator’s Office, One Fund Administrative Agent (data collection phase); Checklists or semi-structured interview protocols for each type of interview (inception phase); Establishment of historical causality: a time-line and narrative about the milestone events in the DaO process at country and international level (inception and data collection phase); Field observation (data collection phase); Thematic studies on specific areas of focus of the Joint Programme, as relevant and appropriate (data collection phase). Lessons learning workshop which brings together all the key stakeholders and implementers of the programme (what worked well and why; what didn’t work well and why; what can we learn and what can we do about it in future). Deliverables The following deliverables are expected: An inception report: the evaluation team shall prepare an inception report containing a clear work plan, interpretation of the suggested evaluation questions and observations on their evaluability, specific data collection methods, assessment tools such as questionnaires and the specific key informants. Presentation of and preliminary findings: At the end of the field work, the evaluation team will present their draft findings and provisional recommendations. The evaluation team will also make a PowerPoint presentation summarizing the main findings recommendations and lessons learned and conclusions Draft Report: The evaluation team will prepare a draft report for comments by stakeholders. This should be a focused document containing an executive summary, background, introduction findings, conclusions, recommendations, lessons learned and annexes. It should also clearly state the limitations of the methodology followed. Final Report: Based on the comments received from stakeholders, the evaluation team shall submit four hard copies and an electronic version of the final report within 2 weeks of receiving comments. Timeframe It is estimated that the evaluation team will conduct their work over 30 working days: Activity: Preparation Preliminary consultations with stakeholders, preparation and circulation of draft terms of reference, mapping of data and information sources and identification and recruitment of consultants Output: Final terms of reference. Recruitment of evaluation team. Activity: Inception, Initial Briefing, Review of documents, Drafting Inception Report Output Inception Report containing a detailed work plan, review of literature, interpretation of the suggested evaluation questions and observations on their evaluability, detailed methods, draft instruments and an evaluation matrix. Presentation to Quality. Assurance Team Timeline: 5 working days Activity: Fieldwork, interviews with stakeholders, field Visits , reviewing & analyzing information, mid evaluation debriefing Output: Interview Coverage completed, agreed field stations visited, debriefing with the quality assurance team Timeline: 20 days Activities: Reporting and presentation (findings), drafting the report, presenting the report to stakeholders, finalizing the report. Output: Draft Report, Presentation to Stakeholders, Inputs from stakeholders and Final Report Timeline: 5 days Stakeholder Involvement There are a range of country-level stakeholders with an interest in the JP3 including UN agencies and staff, implementing partners including the Government of Tanzania and non-government organizations, and development partners who are providing funds to the One Fund. Stakeholders will be asked to support the evaluation by facilitating access to information (such as documents and people) and providing information directly to the evaluation team. The UNJT under the coordination of UNAIDS will also provide additional logistical support to the evaluation team as per the contract and contribute in-kind support (for example, office space). Accountabilities: The following section outlines the evaluation management responsibilities as well as those of the evaluation team. In its role as Managing Agent and thus as facilitator, the UNDP Country Office will, in consultation with partners commission the evaluation and oversee the process, assuring the independence of the evaluation and that the final product complies with the highest standards in evaluation. The consultants will report to the Deputy Country Director (Programs). UNDP will provide secretariat support to the evaluation team, and be responsible for: Assessing the tenders and managing the consultant recruitment process of the independent evaluation team Overseeing the evaluation process throughout its entire duration Managing the evaluation budget including drafting financial reports. Facilitate visits in consultation with UNAIDS, PUNs, TACAIDS, ZAC and relevant MDAs, Regions and Districts. Facilitating the work of the independent evaluation team by ensuring that all relevant contacts and information are available. Coordinating any stakeholder workshops in consultation with key stakeholders, and the evaluation team including, among other things: drafting the agenda, identifying materials for consultation and distribution, coordinating with participants; and drafting the stakeholder report. In similar consultation with UNAIDS, TACAIDS, ZAC and PUNs as already mentioned above, the UNDP Country Office will also prepare a management response on the final evaluation report. There will be a quality assurance committee whose members will be from UNAIDS, UNICEF, UNFPA, UNDP, WHO and TACAIDS and ZAC and whose main duty will be to put in place a plan for quality assurance; facilitate the review of the evaluation process products including ensuring that the views of stakeholders are considered during the evaluation; and to ensure adherence to the norms and standards of UNEG.

Competencies

The evaluation team will be comprised of two (2) evaluation professionals (1 international and 1 national consultant) who, as a team, have a solid understanding of the national context, the JP3 focus area and a proven track record of conducting evaluations in a professional manner. To the extent possible the composition of the evaluation team will be gender balanced and utilize regionally based expertise. Full competency in English (written and spoken) is required. Kiswahili will be considered an advantage. The evaluation team is expected to be fully self-sufficient in terms of IT/office equipment, stationary, communication, office space, accommodation, transport and other logistics. The evaluation team will be responsible for conducting the evaluation. This entails among other responsibilities designing the evaluation according to the specific terms of reference; gathering data from different sources of information; analyzing, organizing and triangulating the information; identifying patterns and causal linkages that explain current performance; drafting evaluation reports at different stages (inception, drafts, final); responding to comments and factual corrections from stakeholders and incorporating them, as appropriate, in subsequent versions; addressing comments by the external Quality Assurance Panel; and making briefs and presentations ensuring the evaluation findings, conclusions and recommendations are communicated in a coherent, clear and understandable manner once the report is completed.

Required Skills and Experience

The international consultant should have the following qualifications: Master’s degree in Public Health, international development, monitoring & evaluation or related field. A minimum of 10 years of professional experience specifically in the area of evaluation of international development initiatives and development organizations. Substantial international track record of conducting different types of evaluations, including process, outcome and impact evaluations in different countries and organizations. Experience in M&E of cross-cutting issues including HIV/AIDS, human rights, gender and capacity development. Must not have been part of the design, implementation of JP3 or employed by any of the partner organizations during the time of implementation of JP3. Knowledge and experience of the UN System and the UN Reform process. Understanding of the development context in Tanzania and/or other ‘Delivering as One’ countries would be a clear advantage. Excellent communication and interview skills. Excellent report writing skills. Demonstrated ability to deliver quality results within strict deadlines. The national consultant should have the following qualifications: Master’s degree in Public Health, Epidemiology of HIV/AIDS and programming, monitoring and evaluation or related field. A minimum of seven years of professional experience, specifically in the area of monitoring and evaluation of international development initiatives and development organizations. A track record of conducting various types of evaluations, including process, outcome and impact evaluations in Tanzania and preferably in the region. Experience in M&E of cross-cutting issues including HIV/AIDS, human rights, gender and capacity development. Must not have been part of the design, implementation of JP3 or employed by any of the partner organizations during the time of implementation of JP3. Knowledge and experience of the UN System and the UN Reform process. In-depth understanding of the development context in Tanzania. Excellent communication and interview skills. Excellent report writing skills. Demonstrated ability to deliver quality results within strict deadlines.

UNDP is committed to achieving workforce diversity in terms of gender, nationality and culture. Individuals from minority groups, indigenous groups and persons with disabilities are equally encouraged to apply. All applications will be treated with the strictest confidence.

EVALUATION OF THE JOINT PROGRAMME 3 - SUPPORT TO THE NATIONAL RESPONSE AGAINST HIV AND AIDS