Consultancy for Trachoma End of Project Evaluation

Evaluation TERMS OF REFERENCE

KENYA TRACHOMA ELIMINATION PROJECT (KTEP)

END OF PROJECT EVALUATION

Introduction

The Fred Hollows Foundation (The Foundation) is a secular non-profit public health organisation based in Australia, which was founded in 1992 by eminent eye surgeon Professor Fred Hollows. The Foundation focuses on strengthening eye health systems and the treatment and prevention of avoidable blindness caused by Cataract, Trachoma, Diabetic Retinopathy, and Refractive Error. It operates in more than 20 countries across Australia, The Pacific, South and South-East Asia, and Africa. The Foundation was named The Australian Charity of the Year 2013 at the inaugural Australian Charity Awards.

The Foundation began working in Kenya in 2004, partnering with the Ministry of Health (MoH) to deliver comprehensive eye health care in the three hitherto districts (now counties) of Nakuru, Homa Bay, and Migori, evolving to eventually cover 13 counties based on the needs identified in the National Plan for Blindness Prevention (NPBP). Working in partnership with the MoH, The Foundation strives to improve access to high quality eye health services through community empowerment, development of appropriate infrastructure to enhance coverage of effective eye health programmes, and improvement of the quality of eye health workforce education and training through capacity strengthening of health professional training institutions.

Project background

Phase II of the 5year Kenya Trachoma Elimination Project (KTEP) funded by Queen Elizabeth Diamond Jubilee Trust (the Trust) and The Fred Hollows Foundation ended on March 31st, 2019. The July 2017 and March 2018 Trachoma Impact Assessment (IA) report, the outcome of the Trust YR4 review and planning meeting, and the December 2017 FHF SAFE review reports identified gaps post the TRUST funded KTEP project. The gaps were contributed majorly by delayed MDAs due to delay in signing of an agreement between MoH and ITI, delay in implementation of impact assessments, lack of agreement on methodology to use for the surveys and delay in commencement of F and E activities. By the end of the Trust funding, three Counties (Turkana, West Pokot and Baringo-East Pokot) had an estimated number of 1,771 cases to threshold. Initial reports on sweeping indicated that the sweeping coverage in Kitui and Embu counties was at close to 100%. In Baringo however, there were no interventions in 2 sub counties of interest. Specifically, the 2018/ 2019 impact assessments conducted in Kitui, Embu and Baringo revealed that the three counties had a back log of over 4,000 TT cases and 2824 TT cases to manage to reach to elimination. Therefore, urgent interventions were required to address these cases and reach the recommended elimination threshold.

The current Trachoma project was therefore to address these gaps between 2019 and 2023. The project outcomes remained the same with an emphasis to strengthen the sustainability of interventions beyond elimination period. The planned activities targeted communities living in 6 Trachoma endemic counties (Turkana, West Pokot, Baringo, Kitui, Narok and Embu) which had not reached elimination by March 31, 2019. Specific SAFE activities were implemented in different counties with transition activities implemented in all the six counties. Gender and disability inclusivity was a key focus during the implementation of the various SAFE components.

Specific Interventions

The table below shows the specific interventions that were carried out in each sub county in the 6 counties.

KITUI

Sub county - Kitui Central, Kitui West, Kitui Rural, Kitui East, Mwingi West, Kitui South, Mwingi Central, Mwingi North

Interventions - Surgery and F&E

EMBU

Sub county - Mbeere North; Mbeere South

Interventions - Surgery and F&E

WEST POKOT

Sub County - Pokot North, Pokot Central, Pokot South, West Pokot

Interventions - Surgery, Mass drug administration and F&E

BARINGO

Sub County - Baringo North, Baringo South

Interventions - Surgery and F&E

Sub County - Tiaty East, Tiaty West

Interventions - Surgery, Mass drug administration and F&E

TURKANA

Sub County - Turkana North, Kibish, Turkana South, Turkana East

Interventions - Surgery, Mass drug administration and F&E

NAROK

Sub County - Transmara

Interventions - Surgery and mass drug administration

This project intended to achieve the following outcomes and outputs.

1. Outcome 1: TT prevalence reduced to WHO recommended threshold of below 0.2% in persons 15 year and above in 5 counties by 2023.

• Output 1: 4,595 TT cases managed in 5 counties.

1. Outcome 2: Trachoma inflammation follicular (TF) Prevalence reduced to acceptable WHO elimination goal of <5% in children 1-9 years old in 3 counties.

• Output 1: Treat communities with antibiotics for Trachoma control with each evaluation unit achieving a coverage of over 80%

1. Outcome 3: Demonstrated change in behaviour towards facial cleanliness and environmental improvement amongst communities.

• Output 1: Communities demonstrating positive change in behaviour towards to facial cleanliness and improved sanitation.

1. Outcome 4: Strengthened County governments’ engagement to sustain the interventions beyond control and elimination period.

• Output 1: Detailed transition plans developed, implemented, and reviewed for sustainability of trachoma control gains post elimination.
• Output 2: Innovative approaches targeting access to services for women, girls and people with disability are developed and implemented.

Project partners

MOH including National government and Directorates, agencies, OSU, KEMRI, CHMTs

Kenya Medical Supplies Authority (KEMSA)

National consortia - Health & Eye health NGOs

Council of Governors/County governments

NTD department

Purpose of the evaluation

The purpose of this evaluation is to determine the achievement of the project across the four objectives. This will provide the project with information that can be used identify which areas the project successfully achieved and areas that will need to be followed up after the project term.

The primary intended users of this of the end of project evaluation will be The Foundation’s:

• Programs Division (including Program Development Team and country program teams)
• Kenya Team
• Country Support Network and other country teams
• Development Effectiveness Team

These teams provide advice and/or make decisions about the choice and design of programs across The Foundation.

The evaluation is also likely to be of interest to a range of secondary users, including:

• International eye health sector NGOs, the Australian Department of Foreign Affairs and Trade.
• Ministry of Health in Kenya and 6 counties where Trachoma is being implemented.

Evaluation scope and key questions

This is an end of project evaluation of the ‘Kenya Trachoma Elimination Project’ in 6 counties of Kenya to be implemented between 2019 – 2023. The following indicative evaluation questions will be used to assess the status of the project. However, the consultant will have latitude to expand the set of questions to suit the evaluation.

Evaluation Criteria

Key evaluation questions

Relevance

1. To what extent was the design of the Trachoma project appropriate in responding to the needs and priorities in selected trachoma endemic regions?

Effectiveness

1. What key changes have occurred in behaviour towards facial cleanliness and environmental improvement amongst schools and communities in the project areas?
2. To what extent did the project achieve TT surgery outcomes in terms of numbers, quality, capacity building, case finding and strengthening referral pathways? How well did partners work together through Trachoma task force /TWG/ CHMTs to effectively coordinate trachoma services?
3. To what extent were mass drug administration activities effective in reaching the communities and achieving the required coverage?
4. Did the project achieve the WHO recommended high coverage rates for key interventions, especially Mass drug administration and F&E promotion?
5. To what extent was data collection captured and documentation done to support dossier preparation?

Impact

1. Compared to baseline, what are the current TT and TF prevalence levels and what strategies contributed to this?
2. What was the impact of the project on reducing the TF and TT prevalence in the targeted evaluation units (target population)?

Sustainability

1. To what extent were the project interventions sustainable to bring about lasting improvements in Trachoma control?
2. To what extent have the counties set up a transition and surveillance system capable of detecting and managing incident cases of TT?

Equity

1. To what extent was the project able to reach the “hard to reach” communities and “leave no one behind”

Approach

FHF Kenya will recruit a research consultant/institution to conduct the end of project evaluation. The selected evaluation consultant will be required to prepare detailed research methodology in collaboration with FHF Kenya and head office team. The consultant will be required to propose suitable methods for the evaluation but at the minimum the chosen methodology should be broad enough to accommodate both quantitative and qualitative methods.

5.2: Proposed methods for data collection

5.2.1: Quantitative data collection

The evaluation will collect quantitative data to answer relevant evaluation questions. Possible key respondents or data sources are initially determined by project team. However, the detailed data source and methods, and a complete set of questionnaires will be developed by the consultant.

5.2.2: Qualitative data collection

The evaluation will use the following indicative qualitative methods of data collection, but the consultant will be free to apply a range of other methods that will be relevant to the evaluation.

Document reviews

The consultant will review the relevant project documents which includes Project Development Document (PDD), project agreements, project reports and any other relevant materials on eye health and relating to the Trachoma project.

Key informant interviews (KIIs)

Key Informant Interviews (KIIs) will be conducted to collect data from key persons engaged in eye health across the partners and other relevant stakeholders working in Trachoma. Specifically, the health officials in the 6 counties shall be targeted. Besides, KIIs will also gather data by engaging respective members of County Health Management Teams (CHMT) and officials within target counties.

Review of Kenya Health Information System (KHIS)

The consultant shall review and analyse eye health data from the Kenya Health Information System (KHIS) to collect data that will be relevant in answering the evaluation questions.

Deliverables

The following deliverables are expected from the consultant in completing the assignment:

• Evaluation Plan and methodology, including timeline/ Inception Report
• Completed data collection tools in line with evaluation questions.
• Presentation and validation of preliminary findings
• Draft and final evaluation reports including summary report, summarized slide decks for dissemination of findings.
• Submit to The Foundation raw data collected/generated during the evaluation provided in an agreed format.

The Foundation will provide the following:

• Focal person to support in undertaking the assignment.
• Provide documents required for desk review.
• Give feedback on the evaluation plan and methodology.
• Provide feedback on draft data collection tools, reports, and other deliverables.
• Support coordination between the consultant and partners during the entire evaluation activities.
• Any field visits to obtain information from the beneficiaries?

Schedule

The evaluation is expected to start in July 2023 for an estimated duration of 30 working days. This will include desk reviews, fieldwork, interviews, and report writing and final presentation to FHF. Data collection will take place in the counties of Embu, West Pokot, Turkana, Kitui, Narok and Baringo being the regions where Trachoma project has been under implementation. The evaluation is scheduled to take place between July to September 2023.

Evaluation team & qualifications

This evaluation will be contracted to an independent evaluator or team who will work closely with The Foundation staff in the design and implementation of the evaluation. The Foundation seeks to engage the services of an independent individual or registered organization, that have the following experiences and expertise in project/program evaluation.

• At least a master’s degree in public health, Ophthalmology, or related health sciences.
• Demonstrable experience in evaluation of public health programs including health systems assessments and eye health especially Trachoma.
• Experience in development and use of quantitative and qualitative data collection tools and evaluation methods
• Strong knowledge and experience working with the health care systems and overall understanding of health sector policies including Trachoma elimination programs.
• Strong analytical skills
• Excellent spoken and written communication skills.
• Proficiency in English language.

Confidentiality

The consultant (s) agree to not divulge confidential information to any person for any reason during or after completion of this contract with The Foundation. Upon completion or termination of this contract, the consultant (s) undertake to return to The Foundation any materials, files or property in their possession that relate to the work of The Foundation. The consultant will be responsible for safety, security and administration of primary and secondary data collected from FHF or otherwise.

Intellectual Property

All intellectual property and/or copyright material produced by the evaluator/s whilst under contract to The Foundation remain the property of The Foundation and will not be shared with third parties without the express permission of The Foundation. The evaluator/s are required to surrender any copyright material created during the term of the contract to The Foundation upon completion or termination of the contract.

Safeguarding People

The Fred Hollows Foundation is committed to ensuring that its activities are implemented in a safe and productive environment which prevents harm and avoids negative impacts on the health and safety of all people, particularly children, vulnerable people, and disadvantaged groups. The Foundation has a zero-tolerance approach to sexual exploitation, abuse, and harassment of any kind. All personnel including contractors/consultants are expected to uphold and promote high standards of professional conduct in line with The Foundation’s Safeguarding People Policy including Code of Conduct. Contractors/consultants will be expected to sign and adhere to The Foundation’s Safeguarding Code of Conduct and provide any background checks as required.

INSURANCE

Any consultants involved in this evaluation will be required to have in place insurance arrangements appropriate to provision of the requirements in this Terms of Reference including travel insurance.

Ethical and other Considerations

The evaluator and evaluation team are expected to maintain high professional and ethical standards and comply with The Foundation’s Research and Evaluation Policy. The Foundation is committed to ensuring a safe environment and culture for all people, including children, with whom we come in contact during our work. All members of the evaluation team will be required to comply with The Foundation’s Safeguarding People Policy and sign the Safeguarding Code of Conduct.

How to apply

This evaluation is being commissioned by The Fred Hollows Foundation Kenya. The Kenya program team will work closely with selected consultant (s).

Individual or institutions interested should submit a cover letter, resume, technical and financial proposals to The Foundation office in Kenya by 27th June 2023. Please reference “Consultancy for Trachoma End of Project Evaluation” and send to the following address: procurementkenya@hollows.org. Only submissions with complete documents as stated in the TOR and those meeting minimum requirements will be considered. Qualified consultants may be subjected to a background check on child protection as a condition for engagement.