How Worried Should We Be? The Implications of Fabricated Survey Data for Political Science

A prevailing view is that faked interviews affect the inferences drawn from compromised datasets. Researchers have generated theories about how fabrication might affect inferences, but speculation has outpaced systematic testing. A rare dataset from a national face-to-face survey in Venezuela in 2016/17 provides an opportunity to address this empirical gap. The survey was conducted in a nationally representative sample, and an unusually high volume of falsified interviews was detected. The researchers canceled and replaced these interviews while fieldwork was in progress, allowing for direct comparisons between a validated dataset and the compromised one. The dataset includes both the canceled and replaced interviews, which enables the examination of the effects of fabrication on inferences. Descriptive inference is sometimes affected by fabrication, but correlational results hold in some typical applications, even in a dataset with an unusually high proportion of fabricated cases. Replication with a second dataset from a similar study in Peru in 2017 yields similar results. Enumerators largely seem to fabricate plausible data, which tamps down on the likelihood that faked interviews severely threaten political science research. The study's findings have implications for research on politics. Scholars have long relied on survey data to generate insights into public opinion and political behavior. The implication of this study is that analyses of opinion datasets compromised by fabrication may not be as severely threatened as previously thought. Enumerators' fabrication of plausible data may not constitute a grave threat to political science research. The study's methodology differs significantly from most approaches to detecting fabricated data, which remove fraudulent cases after fieldwork is complete. The researchers used dozens of automated and manual quality control checks to cancel and immediately replace fraudulent interviews while they were in the field. This approach allows for the examination of the effects of fabrication on inferences in real-time. The study's results have implications for the study of politics and the use of survey data in research. The findings suggest that while fabrication is a critical issue, it may not be as severe a threat as previously thought. The study's methodology provides a valuable approach for detecting and addressing fabrication in survey data, which can help to improve the validity and reliability of research findings.