Terms of Reference for an End of Project Evaluation for G.R.A.C.E Africa's New Partners Initiative Program

Terms of Reference for an End of Project Evaluation for G.R.A.C.E Africa's New Partners Initiative Program

Program title:- Engaging a Network of Indigenous Youth Groups and Community Based Organizations in the Support of Orphans and Vulnerable Children”.

Grant No:- GHO-A-00-09-00004-00

Evaluation period: December 1st, 2008 - September 30th, 2011

1.0 Background

Grassroots Alliance for Community Education (G.R.A.C.E) Africa is a non-profit organization registered in 2001 under Kenya’s NGO Act.

The organization’s mission is to enhance the capacity of community-based organizations for self-determined, high impact and sustainable initiatives leading to better health and development.

G.R.A.C.E’s key approach to development work entails strengthening the capacity of local grassroots organizations to design and implement programs aimed at serving poor and marginalized groups and communities.

Since December 2008, and with support from the United States Agency for International Development (USAID) through the New Partners Initiative (NPI), G.R.A.C.E has been implementing a three year program in three of Kenya's eight provinces namely Nyanza, Eastern and Central.

1.1 Purpose of the NPI Program

Through the NPI Program, G.R.A.C.E contributes to the strengthening of community based responses to the plight of OVC as well as the prevention of HIV in Kenya.

This is in line with the Kenya National HIV/AIDS Strategic Plan priority area on mitigation and social impact, whose key focus priority areas include: strengthening social mechanisms for orphan care; ensuring access to social services by OVC (including food & nutrition, education, health, shelter & care, protection and supporting OVC caregivers to provide quality care to OVC under their care); and strengthening the legal and policy framework for protecting the rights of OVC in the country.

1.2 Program Geographical Areas of Coverage

Nyanza Province: Mbita, Siaya, Migori, Kisumu town, Borabu and Oyugis

Central Province: Kerugoya, Kiambu, Kikuyu and Kamahuha

Eastern Province: Isiolo, Maua, Chuka and Embu

1.3 Program Goal:

The goal of NPI program is to improve quality of orphan care in the targeted areas.

1.4 Program Objectives:

The following are the strategic objectives of the NPI program:-

• Provide Early Childhood Education and related services to 1,800 orphans and vulnerable children aged 3 to 6 years by enrolling them in ECD centers.
• Conduct skills training workshops to 1,200 OVC caregivers and community members on psychosocial support, entrepreneurship, HIV Prevention and food security.
• Build and strengthen the capacity of 15 sub partners for quality programming through institutional strengthening assessments/initiatives and provision of technical assistance.
• Increase information sharing and referral systems by participating in key national fora and hosting regional meeting with stakeholders.
• Provide HIV prevention messages to 20,000 community members.

1.5 Program activities and implementing partners

NPI program activities include provision of education and related services to OVC aged 3-6 years in community ECD centres; strengthening the capacity of caregivers and other vulnerable community members in economic well being, Child Protection, Psychosocial Support, food and nutrition; and promoting HIV Prevention through public education and Behaviour Change Communication (BCC).

The program is implemented in collaboration with four organizations namely the Trust for Indigenous Culture And Health (TICAH), UZIMA Foundation, Population Services International (PSI) and SAHAYA Deaf Kenya.

These partner organizations have been engaged by G.R.A.C.E to provide skills and technical support supervision to the 15 local Community Based Organizations that implement the program at the various target districts.

Several line ministries and government departments have also supported the implementation of the program mainly at the district level. These include the ministries of Education, Agriculture, Public Health and Sanitation, Gender and Children’s Services and the local government.

2.0 Purpose and Scope of Work for the End Term Program Evaluation
2.1 Purpose and rationale for the assignment

The NPI Program is scheduled to come to an end at the end of November 2011. In this regard, G.R.A.C.E is sourcing for an external consultant or consultancy firm to undertake the end of program performance evaluation in the course of October, 2011.

The overall purpose of the end of program evaluation is to measure the project outcome of the strategic objectives; determine the relevance, efficiency and effectiveness of the interventions; and highlight program achievements and gaps.

In addition, it will gauge the level of community and other stakeholder participation in and ownership of program implementation processes; the degree of sustainability for the various program interventions; highlight and document promising practices, key challenges encountered and lessons learnt for policy influencing or future programming by G.R.A.C.E and other agencies undertaking similar work.

2.2. Evaluation Objectives

The specific objectives of this evaluation will be to assess the following:

1. Outputs and outcomes: Outputs and outcomes generated by the program in relation to the stated goal, objectives and desired results. In particular, the evaluation will assess the effects of the program activities on the targeted beneficiaries including the OVC, their caregivers, partner CBOs, and community members reached with the various services.

2. Relevance - whether the program interventions met needs of the beneficiaries; the appropriateness of results in relation to the needs of the communities, national policies and priorities.

3. Effectiveness: In particular assess the extent to which program interventions achieved the desired outcomes, factoring in issues of program management including decision making processes, risk management, institutional arrangements and partnerships and their effect on the program results.

4. Efficiency - the relationship between the quantity, quality, and timeliness of program inputs, including personnel, consultants, travel, training, office equipment and financial sub grants to partner CBOs. In addition, determine the quantity, quality, and timeliness of the outputs generated and whether the resources were spent as economically as possible.

5. Sustainability – assess the readiness of partner CBOs and other stakeholders to sustain program interventions, in particular assess the infrastructure and systems of partner CBOs, resources available to sustain the activities and services, collaborative links and referral networks with other service providers, and the level of community ownership.

6. Program Improvement areas: capture key successes, best practices lessons learnt, implementation challenges, constraints, strengths and weaknesses and provide recommendations for possible scale up or replication of the program in totality or in part.

2.3 Evaluation Approach

The evaluation will follow the G.R.A.C.E and USAID Norms and Standards. This implies inter alia that the evaluation must be guided by the principles and goals of community empowerment and focus on Orphans and Vulnerable Children.

Although the consultant will be expected to come up with a comprehensive evaluation design for the exercise, the following principles will be relevant while considering the methodology of the evaluation:

a. Use of participatory approaches including children to measure performance.

b. Evaluation processes should engender greater accountability, transparency, and help build capacity, attempt systematic and objective assessment(s), guide decision-makers and/or program managers and provide information on whether underlying strategies and assumptions used in program implementation were valid (what worked and what did not work and why)

c. The measurement of effects/benefits of program interventions;

d. Giving stakeholders the opportunity to clarify issues related to program delivery

2.4 Evaluation questions:

The following broad questions will need to be addressed:

• Were all program activities implemented as planned? In cases of deviation from the original plans, what were the reasons for deviation?
• Were program activities and interventions appropriate and in line with the National and USAID policies or guidelines in the various service areas?
• Were the activities relevant, and were they implemented in an appropriate, effective and efficient manner?
• What are the key program outputs and outcomes, and to what extent did the program activities contribute the outcome?
• To what extent did external factors influence the outputs and outcome of the program?
• What capacity building activities were undertaken to strengthen implementing partners and how did they contributed to:
• Project success or hindered progress,
• Organizational growth in general,
• Were there any lessons learned about what should have been done differently in the capacity building interventions.
• To what extent did G.R.A.C.E coordinate with both in-country USAID team and Government of Kenya (e.g. partner meetings, membership on technical working groups, involvement on government working groups, participation in trainings, provincial government meetings and coordination, etc.).
• What referral networks did the project develop or work with?
• Are the results sustainable? (Will the outputs and outcome(s) lead to benefits beyond the life of the existing NPI program?)

1. What are the notable achievements, challenges, promising practices and lessons learnt from the program?
2. How can G.R.A.C.E. do things better in the future? (Which findings may have relevance for future programming or for other similar initiatives elsewhere?)

2.5 Evaluation Methods

The evaluator is expected to employ a variety of data collection and analysis techniques for both quantitative and qualitative data to ensure a comprehensive evaluation exercise.

In essence, the process has to factor in the participation of children who either benefited from or took part in program activities.

The key methods for the exercise will include but will not necessarily be limited to the following:

• Desk review of relevant program documents (these include the program proposal, annual work plans and budgets, program progress reports, minutes of technical meetings, progress reports from CBO partners and other documents related to the NPI program);
• Review of the data collected by G.R.A.C.E staff and CBO partners using the Child Status Index (CSI) and other routine monitoring tools.
• Review of the G.R.A.C.E database as well as other databases and registers kept and maintained by the CBO partners at their offices and the various ECD centres supported by G.R.A.C.E.
• Interviews with various informants including G.R.A.C.E management team, Program and Finance staff, staff of partner CBOs, opinion leaders, government officials involved in project implementation, caregivers and child beneficiaries.
• Observation and focus group discussions.

3. Key Responsibilities of the Consultant

3.1. Tasks and Responsibilities

• Work with the G.R.A.C.E Program team to develop an appropriate evaluation design and data collection tools based on indicators as documented in the program proposal and performance monitoring and evaluation framework.
• Develop a Detailed Implementation Plan (DIP) for the evaluation process.
• Review key program documents including those produced by program partners; such documents shall include but are not limited to the background project document, annual work plans, progress reports, and other documents related to the Project.
• Work with the G.R.A.C.E Program team and implementing partners in carrying out the necessary field activities for the end term evaluation, including a pre-testing and refinement of the developed tools and data collection.
• Conduct data analysis as appropriate and generate a draft evaluation report detailing evaluation methodology, process, key achievements, challenges, lessons learnt, promising practices etc.
• Present the draft evaluation report to G.R.A.C.E for input, discussion and revision if need be.
• Review the draft report to include feedback from G.R.A.C.E and her partners, and prepare and submit three hard copies and one soft copy of the report to G.R.A.C.E.

Deliverables

• An end of term program evaluation design detailing the methodology, tools and approaches to be used in the exercise.
• A Detailed Implementation Plan for the evaluation exercise.
• Three (3) bound hard copies of the end term Evaluation Report.
• One soft copy of the End-Term Evaluation Report in a CD.
• The consultant will also be required to submit to G.R.A.C.E. NPI Project all study materials including:

1. Soft copies of all data sets both quantitative and qualitative
2. All filled quantitative data collection tools and qualitative data recording materials.
3. Any other non-consumable documents/items that will be used in the course of the planned consultancy

The contents of the final report will be in line with USAID Evaluation Policy and as agreed upon together with the G.R.A.C.E Management Team.

However, the evaluation report must include any significant unresolved differences of opinion on the part of funders, implementers and/or members of the evaluation team where those arise.

4.0 Timeframe

The assignment is expected to take place between October, 3rd and October 29th 2011 with a maximum of 20 consultancy days

Consultant’s Profile: Skills, Experience and Qualifications

• Proven experience in research, monitoring and evaluation of non-governmental community development structures, process and integrated programs.
• Sound knowledge and understanding of programs related to OVC support and HIV Prevention.
• Sound knowledge and understanding of OVC service provision national standards, health sector policies and systems, and HIV&AIDS response framework in Kenya.
• The Lead Consultant must possess a Masters degree from an internationally recognized university and at least 5 years experience in programming at a senior level, as well as experience in evaluating complex programs related to OVC, HIV and AIDS.

Other skills necessary include:

• Quantitative and qualitative research methods
• Program analysis, evaluation and multi-districts/community study design
• Knowledge on gender main streaming, gender analysis and cross-cultural studies
• Organizational development and systems strengthening
• Data analysis and report writing
• Strong communications skills -oral, written and presentation skills

5. Expression of Interest

Interested consultants or firms are requested to submit an Expression Of Interest detailing their interpretation of the TOR, proposed methodology, work schedule and proposed budget (in Kenya Shillings), a capability statement and copies of the relevant Curriculum Vitae.

These should be submitted on or before the 29th of October 2011 to the following email addresses;

Email: pascal.mailu@gmail.com and, onyaloj@gmail.com

Hard copies should be addressed to

The Program Coordinator,
G.R.A.C.E. Africa,
P.O.Box 13993 -00100,
Ole Odume Road, 2 doors past Riara Road,
Nairobi, Kenya.

Tel +254 (0) 20 387 2856.