Let's Stop Trying to Quantify Household Vulnerability: The Problem With Simple Scales for Targeting and Evaluating Economic Strengthening Programs

These programs are typically funded by the United States Agency for International Development (USAID) and the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) for orphans and vulnerable children (OVC) affected by HIV. Interventions are designed to improve economic status in order to help households better access HIV-related services, reduce HIV risk, and improve child well-being. Practitioners are increasingly seeking data collection tools that will help them target vulnerable households, classify them in terms of economic vulnerability, and match them to appropriate economic interventions. To meet this demand, the ASPIRES project, funded by USAID and PEPFAR and managed by FHI 360, has experimented with vulnerability assessment methods relevant for economic strengthening projects. Here, we discuss three different efforts to quantify and classify individual and household economic status: a large-scale survey in Côte d'Ivoire using rigorous psychometric methods and two validation exercises conducted with monitoring and evaluation (M&E) tools developed for different economic strengthening projects in Uganda and South Africa. Across disciplines, vulnerability is generally understood as the risk of falling below an accepted benchmark of welfare. Economists typically analyze household survey data using econometric methods to predict economic vulnerability at a population level, answering questions such as what is the extent of vulnerability, who is vulnerable, what are the sources of vulnerability, how do households respond to shocks, and what gaps exist between risks and risk management mechanisms. Economic strengthening practitioners need vulnerability assessment tools that can help them target households and individuals and determine if their vulnerability has decreased during an intervention. The most intuitive basis for targeting and monitoring economic strengthening interventions is to measure poverty levels. Household poverty, however, can be difficult to measure accurately. One popular tool for doing so is the Progress out of Poverty Index (PPI), a validated, 10-question scorecard that includes indicators derived from national survey data to predict the likelihood that a household falls beneath a given poverty line. The PPI is easy to use and transparent in its accuracy for targeting at the individual level, which varies depending on the poverty cutoff selected. However, it encompasses several indicators that are unlikely to change over time, such as household size, and may not be very sensitive to monitoring economic change over the time frame of an average project. Several OVC programs have approached this challenge by developing short scorecards with indicators related to program objectives. Because data are usually collected by community volunteers acting as case managers, the tools must be simple and easy to use for data collectors with limited education. This need, in addition to the large scale of many programs, places a premium on streamlining data collection efforts, so implementers often prefer a single tool to serve the functions of targeting, M&E, matching households to appropriate interventions, and determining readiness for graduation. When these types of tools are scored and aggregated as an index measure of vulnerability, they can give implementers a false sense of accuracy and result in failure to target the neediest households. An influential example of this kind of tool is the Vulnerability Assessment Tool (VAT) used by the Sustainable Comprehensive Responses (SCORE) for Vulnerable Children and their Families project, implemented by AVSI, in Uganda. This tool includes questions on household assets and income, child protection, food security, parental status, basic services, and the enumerator's impression to classify households according to PEPFAR's categories of household economic status: destitute, struggling to make ends meet, or prepared to grow. However, a test of the VAT demonstrated that it was not sufficiently sensitive to identify households with the most critical needs, limiting its utility as a targeting tool. The Government of Uganda has since adapted an updated version of the VAT called the Household Vulnerability Assessment Tool (HVAT). The HVAT collects information on child and household well-being, including indicators related to program objectives, and is designed to be simple and easy to use for data collectors with limited education. In Côte d'Ivoire, a cross-sectional survey with 3,749 households was conducted to develop a scale based on the definition of HIV-related economic vulnerability from the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) for the purpose of targeting vulnerable households for PEPFAR-funded programs for orphans and vulnerable children. The vulnerability measures examined did not cluster in ways that would allow for the creation of a small number of composite measures, and thus a scale was not developed. In Uganda, the validity of a vulnerability index developed to classify households according to donor classifications of economic status was assessed by measuring its association with a validated poverty measure. The results showed only a modest correlation, indicating that the index was not a reliable measure of economic vulnerability. In South Africa, monitoring and evaluation tools were developed to assess economic status of individual adolescent girls and their households. The results showed no significant correlation with the validation measures, which included a validated measure of girls' vulnerability to HIV, a validated poverty measure, and