Evaluation report : midterm evaluation of the digital health activity (DHA)

USAID/Ethiopia contracted the Global Health Evaluation and Learning Support Activity (GH EvaLS) to conduct a midterm performance evaluation of the Digital Health Activity (DHA) implemented by John Snow International and its partners. Purpose. The purpose was to obtain independent observations about Activity achievements, challenges, and lessons learned during implementation to inform course corrections and fine-tune program plans and approaches for the remaining years of the Activity. Methods. The evaluation employed a mixed-methods approach using quantitative (112 health facilities assessments, and secondary data review) and qualitative (desk review, 38 key informant interviews, and five focus group discussions) data collection methods. Key Findings and Conclusions. Data indicated that DHA is being implemented as designed to contribute to the country’s information revolution, provide support on information technology systems and use of a data repository at all levels, and build a culture of data use and capacity-building that governs the Ethiopian Ministry of Health (MOH) health information system. DHA interventions address government priorities and demand in relation to digitization, data-use culture, and institutional capacitybuilding and governance. The digital tools supported by DHA improved program implementation and evidence-based decision-making at different levels (health post, health center, hospital, woreda health office, and Ethiopian Pharmaceutical Supply Service). Data quality, including completeness/consistency, and use improved, however, infrastructure (internet connectivity, power, and system installation), human resources attrition, and security concerns hindered wider implementation. Key informants (MOH and regional health bureaus) reported lack of transparency in communicating its work plan. The Activity is working to address the existing gender gap in Ethiopia’s digital space by developing a gender mainstreaming strategy in collaboration with MOH. DHA’s various interventions have focused on empowering youth and women, such as youth enterprises, digital hackathons, and internship programs, however, financial skillbuilding efforts are needed. DHA’s local ownership and sustainability approaches included: engagement with MOH and local partners, strengthening the capacity of youth enterprises, strengthening the capacitybuilding/mentorship program by partnering with local universities and colleges, providing support virtually through call centers, making e-learning platforms available, and providing facility-level support through supportive supervision and mentorship. Key Recommendations. USAID and the Government of Ethiopia (GOE) should continue to synergistically support the current momentum to sustain the necessary national integration and alignment of digital tools. A comprehensive resource mapping of different partners will help to further enable better integration of digital tools and ensure targeted interventions from DHA remain relevant. DHA should continue to support youth enterprises and strengthen them through creating market linkages and building their capacity in writing business proposals. DHA, in collaboration with the MOH, needs to ensure usability of digital tools by strengthening the capacity of regional health bureaus through coaching, mentoring, and follow-up and step-by-step implementation of the tools. The GOE should play a major role in bringing all stakeholders together to mitigate the infrastructure and human resource challenges. DHA should strengthen working relationships among the MOH, local universities, and the Ministry of Information and Technology regarding e-learning and the use of adapted technology in all digital health implementation areas, including conflict zones.