Data for Impact: What’s the cost of evaluations and other surveys?

A core principle of D4I is to strengthen the technical and organizational capacity of local partners to collect, analyze, and use data to support their move to self-reliance. D4I conducts various types of evaluations, including process, outcome, impact, and economic evaluations, as well as implementation science and operations research. The primary types of evaluations in D4I's scope are process, outcome, impact, and economic evaluations, and implementation science and operations research. D4I also conducts outcome monitoring surveys and implements formative studies to aid in intervention design and implementation. When planning any of these investigations, cost is an important consideration that may help determine the decision to do or plan for an evaluation or study. Cost is dependent on many factors, including objective, design, method, sample size, geographic scope, and local context. Objectives play a significant role in determining the cost of an evaluation or study. The number of objectives and the type of objective affect cost. A large number of questions or objectives typically increases the sample size and the number of different data-collection methods needed. Research questions that require comparing different combinations of activities imply multiple evaluation arms, which increases cost. Questions on differential impact of interventions on different target populations typically require a larger sample size and have a higher cost. Methods also affect the cost of an evaluation or study. The type and number of methods affect the cost of an evaluation or study. For example, household surveys tend to be more expensive than facility-based surveys. Where feasible, D4I uses existing and routine data. Use of existing data has the potential to decrease costs compared with primary data collection, but efforts to abstract data or account for missing data can be costly. In some cases, different methods are combined in a single study. Sample size is another factor that affects the cost of an evaluation or study. Larger sample sizes come with higher costs. Keep in mind that to detect statistically significant change in an outcome, you typically need a larger sample size than you would need for a point estimate at the same level of precision. This is particularly true if the outcome is relatively rare or is likely to change slowly. Estimating the difference in change in an outcome between program and nonprogram areas typically requires even larger sample sizes. Country context also affects the cost of data collection. Data-collection costs vary across countries, depending on local capacity for data collection, transport costs, ethics costs, and costs for other approval board review, etc. Institutional strengthening is also an important consideration, and the associated costs depend on the extent of such activities, including costs for assessments if required and work planning. Data use is another critical factor in the cost of an evaluation or study. It is essential to highlight the commitment to disseminate and act on the evaluation findings, which means that planning should incorporate costs for engaging stakeholders in design and intended data use. Data use activities may include stakeholder study sensitization meetings, assessments of data needs and use, and data use workshops as part of dissemination. Examples of costs from recent evaluations and studies conducted under MEASURE Evaluation, D4I's leader award, are provided in Table 1. These costs include both direct and indirect costs. The table is meant to illustrate the range of likely costs, but each evaluation or survey must be budgeted based on its own parameters.