Cost of Evaluations Brief FS-24-660 D4I

The primary types of evaluations conducted by D4I include process, outcome, impact, real-time, and economic evaluations, as well as implementation science and operations research. D4I also conducts outcome monitoring surveys and formative studies to aid in intervention design and implementation. When planning evaluations, cost is a crucial consideration that may influence the decision to conduct or plan for an evaluation or study. The cost of an evaluation depends on various factors, such as objectives, design, methods, sample size, geographic scope, and local context. The number of objectives and the type of objective can also affect cost, with a large number of questions or objectives typically increasing the sample size and the number of different data-collection methods needed. Planning a study can involve extensive coordination and collaboration with clients, programs, governments, and other local stakeholders to develop and finalize a study protocol. This process can take six months to a year or more, depending on the complexity of the design, tools, and context. The type and number of methods used in an evaluation or study can also impact the cost, with household surveys often being more expensive than facility-based surveys. D4I uses existing and routine data where feasible, which can decrease costs compared to primary data collection. However, efforts to abstract or account for missing data can be costly. The inclusion of biomarkers can substantially add to costs, and study questions and objectives play a key role in determining the most appropriate type of evaluation study design or methods. Sample size is another critical factor that affects cost, with larger sample sizes coming with higher costs. Country context also plays a significant role in data collection costs, with costs varying across countries depending on country capacity for data collection, transport costs, ethics costs, and costs for other approval board reviews. Institutional strengthening is an essential aspect of D4I's work, and the associated costs depend on the extent of such activities, including costs for assessments, if required, and work planning. Informal mentoring and learning-by-doing approaches are the least costly, while formal training and more intensive mentoring efforts increase costs. Data use is also a critical consideration, and planning should incorporate costs for engaging stakeholders in design and intended data use. Data-use activities may include stakeholder sensitization meetings, assessments of data needs and use, data-use dissemination workshops, and the development of knowledge products such as briefs and infographics. Table 1 provides examples of costs from recent evaluations and studies conducted by D4I, including a thematic evaluation of family planning strategic transitions in Indonesia, Honduras, Peru, and Morocco, which cost approximately $249,000. The Zimbabwe Assistance Program in Malaria (ZAPIM) Assessment cost $250,000, while the Evaluation of the Private Health Sector Project (PHSP) in Ethiopia cost $350,000. The My Forest, My Livelihood, My Family program (FUTURES) project evaluation in Ethiopia cost $842,000, and the Malawi Secondary Education Expansion for Development (SEED) evaluation cost $1,500,000. The USAID Integrated Health Program (IHP) Program evaluation in the Democratic Republic of the Congo was the most expensive, costing $6,725,000.