External Evaluation of a DFID funded project - Mwanza, Tanzania

Railway Children Africa is looking for an independent consultant or firm to conduct a high quality external evaluation of a 3 year project funded by DFID's Global Poverty Action Fund, implemented in Mwanza, Tanzania.

Railway Children (RC)

Railway Children Africa is registered in Tanzania as a Limited Company (77698) under the Companies Act, with NGO compliance (1563) under the Non-Governmental Organisations Act. Railway Children Africa is a subsidiary of Railway Children. Railway Children is registered in the UK as both a Charity (1058991) and a Company Limited by Guarantee (3265496).

Railway Children exists to help vulnerable children in grave circumstances. Our objective is to provide relief to children and young people who are in conditions of need, hardship or distress and in particular to those living on the streets.

As an organisation our primary aim is to enable and create positive sustainable change in the lives of children living alone and at risk on the streets. Our ‘3 Step Change Agenda’ embodies our vision of real change being achieved through success in three interwoven, inextricable areas:

1. Meeting the immediate needs of children on the streets
2. Shifting local perception
3. Holding governments responsible

Railway Children is receiving funds from DFID's Global Poverty Action (GPAF) Fund for a 3 year period - January 2013-March 2016 - and the external evaluation outlined in this ToR will aim to verify and learn about impact achieved in implementing the project supported by GPAF in Mwanza, Tanzania.

The Global Poverty Action Fund (GPAF)

DFID provides significant funding to civil society organisations (CSOs) annually in line with its overall strategy to alleviate poverty and promote peace, stability and good governance. The Programme Partnership Arrangement (PPA) and Global Poverty Action Fund (GPAF) are two of DFID’s principal funding mechanisms and have provided £480 million to approximately 230 CSOs between 2011 and 2013. The current political climate and results-based agenda demand a rigorous assessment of the effectiveness of funds disbursed to ensure that they are managed to provide value for money.

TripleLine is the Fund Manager for the GPAF and is responsible for assessing performance of grantees at the project level. Coffey International Development is the Evaluation Manager and is responsible for assessing the performance of the funding mechanisms as a whole. For more information on the fund level valuation, please see the 2012 Evaluation Strategy.

Railway Children Project Description

Project title: ‘Provision of support, reintegration and referral services for 1,200 children and youth on the streets of Mwanza, Tanzania’

This project has aimed to create sustainable support structures for children on the streets in Mwanza; the major urban centre for a radius of several hundred kilometres and the destination for the rural population migrating to the city including children escaping poverty and abuse. Government agencies have few resources to respond to the needs of children and local agencies are under resourced in terms of funds available and appropriate skills. We are developing quality practice across agencies in the city which work with children on the streets, which includes contact, short term care and effective reintegration with families for children already on the street and support for vulnerable families to prevent their children going to the street. We provide opportunities for homeless youth to learn new skills to enable them to improve their livelihoods and leave the street. The project is also working with local government to help build effective linkages with service providers to encourage sustainable structures and processes.

Purpose of the independent final evaluation for GPAF grantees

The evaluation report will ultimately be submitted by Railway Children to DfID, and will be used to inform the GPAF Fund Manager’s understanding of Railway Children's performance at the project level and will also be used to inform the Evaluation Manager’s assessment of performance at the GPAF fund level. The independent final evaluation report needs to be a substantial document that (a) answers all the elements of the Terms of Reference (ToR); (b) provides findings and conclusions that are based on robust and transparent evidence; and (c) where necessary supplements the grantee’s own data with independent research.

Key objectives of the evaluation

The evaluation has two explicit objectives that are explained below:

1 - To independently verify (and supplement where necessary), Railway Children's record of achievement as reported through its Annual Reports and defined in the project logframe;

2 - To assess the extent to which the project was good value for money, which includes considering:

• How well the project met its objectives;
• How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;
• What has happened because of DFID funding that wouldn’t have otherwise happened; and
• How well the project aligns with DFID’s goals of supporting the delivery of the MDGs.

Verification of grantee reporting

The first task of the final evaluation is to verify Railway Children's achievement. The record of achievement will be presented in past Annual Reports and progress against the project logframe. This exercise could include verifying information that was collected by the grantee for reporting purposes and possibly supplementing this data will additional information collected through primary and secondary research. Verifying the results from the project log frame will begin to capture what the project has achieved. However, there will be other activities and results that occur outside of the logframe that may require examination in order to respond to the different evaluation questions. Verifying reporting will also necessarily include a review of the data and systems that were used to populate results.

Assessment of value for money

The evaluation should assess the extent to which the delivery and results of the project are good value for money. Value for money can be defined in different ways, but at minimum the evaluation report should include an assessment against:

• How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;
• What has happened because of DFID funding that wouldn’t have otherwise happened.

Evaluation questions

To ensure comparability across the final evaluation reports, the evaluator(s) should respond to the questions below. Please note that the attention given to each evaluation question may vary depending on the objectives of certain projects and the availability of data, so the independent evaluator(s) should use his/her discretion in the level of effort used to respond to these questions.

The evaluator is encouraged to structure their research questions according to the OECD-DAC criteria of relevance, effectiveness, efficiency, sustainability and impact.

GPAF's generic Evaluation Questions:

Relevance

• To what extent did the Railway Children support achievement towards the MDGs, specifically off-track MDGs?
• To what extent did the project target and reach the poor and marginalised?
• To what extent did the project mainstream gender equality in the design and delivery of activities (and orother relevant excluded groups)?
• How well did the project respond to the needs of target beneficiaries, including how these needs evolved over time?

Effectiveness

• To what extent are the results that are reported a fair and accurate record of achievement?
• To what extent has the project delivered results that are value for money? To include but not limited to:
• How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;
• What has happened because of DFID funding that wouldn’t have otherwise happened; and
• To what extent has the project used learning to improve delivery?
• What are the key drivers and barriers affecting the delivery of results for the project?

Efficiency

• To what extent did the grantee deliver results on time and on budget against agreed plans?
• To what extent did the project understand cost drivers and manage these in relation to performance requirements?

Sustainability

• To what extent has the project leveraged additional resources (financial and in-kind) from other sources?
• What effect has this had on the scale, delivery or sustainability of activities?
• To what extent is there evidence that the benefits delivered by the project will be sustained after the project ends?

Impact

• To what extent and how has the project built the capacity of civil society?
• How many people are receiving support from the project that otherwise would not have received support?
• To what extent and how has the project affected people in ways that were not originally intended?

Railway Children's additional questions

• CSO capacity building: To what extent did Railway Children succeed in building the institutional and programmatic capacity of the implementing organisations? What were the conditions in each partner and/or our support that led to success or acted as a barrier?
• To what extent did emotional and relational support provided by staff have a positive impact on families and children?
• How well have we balanced the pressure to deliver on project plans against the individual needs of beneficiaries - and what have we learnt from this.
• Youth: To what extent has the economic empowerment been successful (grants / placements) in realising youth working in their selected field? What are the main changes that the youth have experienced as a result of our intervention?
• Economic empowerment across the programme: Analysis of failure and success rates across the programme: What were the factors or conditions caused these differences?
• Government and stakeholder collaboration (output 4) - What have been the barriers of the partnership in building sustainable government collaboration and referral pathways with other organisations?

METHODOLOGY

The consultant will be responsible for designing the methodology, with input from Railway Children. To fulfil the objectives of this exercise both quantitative and qualitative assessment methods should be employed. We require a participatory methodology whereby the work engages all key stakeholders including beneficiaries; children, youth, parents/guardians, community members, partner staff and Railway Children staff.

Selected consultants will be expected to provide an overview of the proposed methodology as part of the tendering process including evaluation tools to be used, progress reports and debriefing meetings and a detailed work plan should be established before the work begins.

Evaluation Outputs

A report should be produced that documents findings of the evaluation in relation to the key questions outlined and be consistent with the reporting guidelines below. The draft report will be shared with Railway Children and respective partner organisations for their comments prior to finalising the report.

REPORTING

The report will be prepared within 20 working days following the end of the evaluation and is not expected to be more than 40 pages, excluding appendices.

Report structure

Front and cover and title page

• Title of the evaluation.
• Project location
• Name of the partner organisations.
• Names of the evaluation team and any support staff
• Dates and duration of the evaluation
• Date report completed

Acknowledgements

• Beneficiaries, Communities, advisers, team members, funders etc.

Contents page

• Numbered contents page

Executive summary

• A summary of the most important parts of the report
• Who was involved in the evaluation
• The purpose and objectives of the evaluation
• How the evaluation was carried out, where and when
• Key findings
• Key recommendations

Background information

• Only information directly relevant to the report's analysis and conclusion
• The context of the project - social, economic, political background along with the existing legal framework and particular problems faced by the target communities

Evaluation Purpose and Methodology

• What methods were chosen and why?
• Which stakeholders were involved and why were they chosen?
• How was the information collected and by whom, and which methods were used?
• How reliable and valid did the methods prove to be?
• Include any evaluation timetable or schedule in an appendix.

Findings and Analysis, {only information directly relevant to the report's analysis and conclusion}

The report will include:

• Key findings – Responses to each of the key questions
• Challenges and learning

Lessons Learnt and Recommendations:

This section should reflect and recommend on the project design and outline any improvements to this implied by the findings of the evaluation.

Appendices

This should include technical information referred to in other sections, for example details of methods used, questionnaires, timetables and schedules, a list of informants and the team's work schedule.

SCHEDULE:

A schedule should be provided with the tender application detailing time spent for each part of the assignment that fits broadly within the following framework:

• Consultant selected and TOR signed - Before 27th November 2015
• Design, methodology and tools are finalised - By end of December 2015 In consultation with RC.
• Field work – carrying out surveys. January 2015 Mwanza - Tanzania
• Data compilation, analysis & reporting February 2015 First draft of the report is ready to be sent to partners and RC for feedback and comment.
• Sharing of report and finalisation March 2015 Railway Children and partners review the report Incorporation of feedback from all stakeholders

PROJECT FINAL EVALUATION TEAM

The composition of the evaluation team will be agreed in advance by Railway Children.

PROFESSIONAL FEES AND LOGISTICS

Full details of all Professional Fees and additional costs in order to complete the assignment should be in the tender document. The total budget for this assignment is £15,240.

RC will pay 30% advance payment at the time of commissioning of the exercise and 30% after the completion of the field work. The remaining 40% would be paid on the completion of the evaluation process and the submission of the report.

CONFIDENTIALITY

The engagement will be carried out confidentially. The consultants will neither use, nor appear to use, information acquired during the course of this engagement for either personal advantage or the advantage of a third party. The consultants will have to abide by Child Protection Policy of Railway Children and the partner organisations.

HOW TO APPLY:

Interested organisations, research institutes or independent organisations should submit a tender document outlining the approach that you would take to complete the assignment and your commitment to meet the objectives, timelines and obligations laid out in this document.

As stated the budget available is £15,240. Your application should include a budget that clearly breaks down expenses incurred in carrying out the assignment separately to the fee charged. Fees charged should also be laid out clearly for each person that will work on the assignment and a brief resume of each person that will work on this assignment should be included in the tender application. Skills and previous experience of this type of assignment should be submitted.

The application should not be more than four pages long with an Arial font size 11, excluding CV and budget. Applications should be sent to w.brawn@railwaychildren.org.uk with the title ‘GPAF Evaluation – Mwanza’in the subject line, no later than Tuesday 10th November 2015. Shortlisted candidates will be contacted by the Tuesday 17th November 2015.