Kenya Resilient Arid Lands Partnership for Integrated Development (RAPID) Activity: Impact Evaluation: Overview and Key Findings

Funded jointly by the United States Agency for International Development (USAID), Swiss Agency for Development and Cooperation (SDC), private sector partners, and Millennium Water Alliance members, the activity aimed to improve water management in arid lands. One key component of the RAPID Activity involved the installation of sensors on water borehole pumps to transmit information to data dashboards in real-time. This information and communication technology (ICT) intervention was paired with budget facilitation for borehole pump repairs and clarification of roles and responsibilities to create dedicated pump maintenance teams. A subset of 69 sensors was installed on "strategic" water boreholes, which local authorities identified as important due to the risk of drought in the borehole area. The USAID/Kenya and East Africa (USAID/KEA) mission, in conjunction with SDC, commissioned a quasi-experimental impact evaluation to understand the effect of the RAPID Activity's ICT intervention on strategic borehole pump use and functionality during drought periods. The evaluation team used a propensity-score matching design to match strategic boreholes in RAPID's five implementation counties with a set of 132 strategic boreholes across eight comparison counties. The evaluation team installed sensors on the comparison county borehole pumps to compare how long borehole pumps ran on average in a day. The team supplemented this quantitative analysis with qualitative interviews and focus group discussions to understand how water managers perceived the impact of the sensor-based system and to see whether user perceptions of borehole pump functionality and access changed. The results of the evaluation suggest that strategic boreholes that received the RAPID Activity's ICT intervention performed similarly to strategic boreholes in comparison counties. Across multiple analyses, the results indicate that the RAPID Activity did not have a statistically significant or meaningful impact on borehole pump functionality during the drier months of the intervention, from 2018 to 2020, relative to comparison county strategic boreholes. At most, the analysis suggests that the sensor-based intervention resulted in less than an hour of additional borehole pump on-time per day in RAPID counties compared to comparison counties. Water managers reported similar timelines for borehole repairs in RAPID and comparison counties, but officials in RAPID counties had a positive perception of the ICT intervention. County- and sub-county-level water managers in RAPID counties viewed the sensor-based system favorably and said it provided useful data to support water management activities. However, these officials also pointed out that lack of access to repair resources continues to limit borehole functionality in RAPID counties. Officials in Garissa—a RAPID county—reported that they did not yet have full access to the data dashboard, and others reported that a lack of office internet and issues with network connectivity for using mobile devices limited access to the data dashboard. A lack of dedicated resources for borehole repairs remains a key barrier to improved functionality in both RAPID and comparison counties. The sensor-based intervention did not improve users' perceptions of borehole functionality and water access in RAPID counties relative to comparison counties. All strategic borehole users continued to identify a range of water access and supply issues not directly addressed by the intervention, including breakages in distribution pipes and taps bringing water from boreholes to people's homes and villages. Crowding and congestion at boreholes lead to long waiting times. Access issues vary seasonally and have a particularly large effect on vulnerable groups, including women and the elderly. The evaluation team offers several recommendations to USAID, including continuing to focus on water system governance, clarifying roles and responsibilities for water management, and establishing dedicated and sustainable funding sources for water system maintenance and repairs. The team also recommends addressing community concerns carefully in planning for delivery of water services and considering rural water ICT intervention costs and context. Additionally, the team suggests that USAID should consider implementation costs and systemic challenges in thinking about the theory of change for ICT interventions given large structural constraints such as limited budgets and climate change.