Using routine and program monitoring data for social and behavior change evaluation: An application, lessons learned, and recommendations

The study aims to guide global-, regional-, and country-level SBC program implementers, evaluators, and monitoring and evaluation teams in documenting the impact of their programs using routinely collected data. Breakthrough ACTION/Nigeria, a project led by Johns Hopkins University Center for Communications Programs (JHU CCP), leverages and expands the Health Communication Capacity Collaborative activities in Nigeria in 11 targeted states. The project focuses on three states: Bauchi, Kebbi, and Sokoto, where Breakthrough RESEARCH's work is concentrated. The official launch of the expanded, integrated SBC activities in the targeted states occurred on September 19, 2019, in Abuja, with staged implementation rolling out between early and late September in select wards within local government areas across the three states. Breakthrough ACTION/Nigeria used a life stage approach to inform integrated SBC activities, targeting women planning for pregnancy and those in early pregnancy, as well as women in late pregnancy and those with a newborn. The project aimed to increase the proportion of pregnant women attending first antenatal care (ANC) as soon as they become aware of pregnancy and at least four times during pregnancy, as well as increase the proportion of pregnant women taking three or more doses of intermittent preventive treatment in pregnancy (IPTp) as prescribed by health workers during ANC. The case study application draws on three types of routine data shared with Breakthrough RESEARCH by Breakthrough ACTION/Nigeria through a cooperative data sharing agreement. These data include SBC-focused community events, household visits, and health facility data. The analyses in this applied example focused on three primary health areas: family planning (FP), maternal health, and malaria. The objectives of the analysis were to assess the impact of SBC integration on FP service uptake, ANC utilization, and use of IPTp. However, there are two important caveats about the data to note, directly affecting the analysis. First, data on FP service utilization collected pre-integration is not fully comparable to the data collected post-integration due to a change in the data collection form that Breakthrough ACTION/Nigeria made at the start of program integration. Second, the data collection form for facility data used pre-integration contains more limited information, with data on ANC visits and IPTp use not available pre-integration and limiting the methodologies that can be used to leverage routine data for evaluation purposes. The study highlights the importance of using routine and program monitoring data for SBC evaluation, particularly in low-resource settings where data collection can be challenging. The findings and limitations of this applied example provide valuable insights for SBC implementers and particularly those working on the monitoring and evaluation of SBC projects. The recommendations shared in this case study can also be used by donors funding SBC programming to guide conversations, set and calibrate expectations, and inform work plans.