Collaborating, Learning, and Adapting Impact Measurement Learning Network: Findings on how to measure CLA

Funded by USAID, the network consisted of five implementing organizations: Counterpart International, Global Knowledge Initiative, MarketShare Associates, Mercy Corps, and Pollen Group. Over an 18-month period, the network conducted five separate research agendas in partnership with seven development projects to address the research question: "How do we effectively measure the contribution (or lack thereof) of CLA to improved organizational effectiveness and development outcomes?" The network grantees employed a mix of innovative quantitative and qualitative methods to investigate the link between CLA practices and development outcomes. These methods included the USAID Maturity Matrix, CLA Action Chains, Pathway/Decision Mapping, Pivot Logs, Participatory Quality Assessment (PQA), surveys, focus group discussions, participant observation, and key informant interviews. The grantees developed specific questions aligned with the overarching research question and contributed to a general theory of change (ToC) to articulate different levels of measurement. Despite facing several challenges, including defining CLA, influencing environmental factors, confirmation bias, aggregating findings, recall bias, absence of a counterfactual, and differing definitions of development outcomes, the grantees produced important and useful contributions to the evidence base for CLA. The methods tested have improved the collective understanding of how to address difficulties of measuring CLA, its enabling conditions, and identified some methods and tools that work well. These include a general ToC and a detailed ToC for each agenda, defining CLA upfront to facilitate measurement, self-assessments as effective tools for generating partner buy-in, quantitative analysis for analyzing cases with large numbers of observations, and key informant interviews of staff. The grantees also identified lessons for future efforts to measure CLA and its impact on development outcomes. These include improving the scope and reliability of findings from pivot logs, employing a flexible research design, and focusing research efforts on different types of programs, sectoral areas, and/or without deliberately adaptive approaches. The network's research agendas have provided valuable insights into the challenges and opportunities of measuring CLA and its contribution to development outcomes. The findings from these research agendas can inform future efforts to develop and implement effective CLA practices and measurement approaches. The network's research has highlighted the importance of defining CLA upfront to facilitate measurement, using a mix of quantitative and qualitative methods, and employing a flexible research design. The grantees have also emphasized the need to consider the enabling conditions for CLA, such as partner organization's ability to adapt, resources available, and organizational incentives. The research has also shown that self-assessments can be effective tools for generating partner buy-in, and that quantitative analysis can be useful for analyzing cases with large numbers of observations. The findings from the network's research agendas have implications for the design and implementation of CLA practices and measurement approaches. The research has highlighted the need for a more nuanced understanding of CLA and its contribution to development outcomes, and the importance of considering the enabling conditions for CLA. The findings also suggest that a flexible research design and a mix of quantitative and qualitative methods can be effective for measuring CLA and its impact on development outcomes.