Development Collaborative Assessment and Feedback Engine (DevCAFE) 1.0: A Participatory Platform for Assessing Development Initiatives in the Field

DevCAFE is a customizable assessment platform that collects and integrates quantitative and qualitative data, as well as peer-to-peer collaborative filtering. The platform enables rapid exploratory analysis and is optimized for use in remote regions with intermittent or non-existent Internet connectivity. DevCAFE addresses two major data filtering challenges: qualitative data filtering, where researchers must quickly identify relevant open-ended responses, and quantitative data filtering, where researchers must identify which quantitative variables correlate significantly with demographics or qualitative features. The platform uses collaborative filtering, where participants evaluate each other's responses, and dimensionality reduction, where Principal Component Analysis (PCA) is used to find features that best differentiate participants. DevCAFE features a visual, interactive interface that presents a 2D map showing a small sample of ideas to evaluate. Evaluations and quantitative data are collected through visual modalities such as sliders and keypads that can be adapted to different local settings. The platform is designed to be flexible and can be modified to fit the needs of a participatory assessment application. Two case studies were conducted to evaluate the initial version of DevCAFE. The Uganda Case Study involved applying DevCAFE to assess the effectiveness of family planning trainings at three Nutrition Education Centers in rural Uganda. Responses were collected from 137 women who attended these trainings. The Mexico Case Study involved applying a Spanish-language DevCAFE to evaluate the Mexican Government's performance on timely policy issues. Data were collected from 4,518 participants from throughout the country. DevCAFE was compared to existing digital data collection tools, including Open Data Kit (ODK) and SMS-based data collection tools such as FrontlineSMS and RapidSMS. While ODK and related tools are well-established, they do not enable interaction between participants. DevCAFE's peer-to-peer interactions and visualization features make it a more suitable tool for participatory assessment applications. The platform's ability to operate in unreliable networks with less fear of losing data is a significant advantage. DevCAFE's use of transactions and logging allows it to manage data longitudinally and provide instant feedback to participants. The platform's flexibility and adaptability make it a valuable tool for development researchers and practitioners. DevCAFE's initial version was supported by the Blum Center for Developing Economies and the Development Impact Lab (USAID Cooperative Agreement AID-OAA-A-12-00011), part of the USAID Higher Education Solutions Network; UC Berkeley's Algorithms, Machines, and People Lab; and the UC CITRIS Connected Communities Initiative.