Bot detection in online studies and experiments

Piehlmaier, Dominik M (2022) Bot detection in online studies and experiments. SAGE Research Methods.

[img] PDF - Accepted Version
Download (439kB)


Most experimental and online studies in the empirical social sciences rely on online panels from crowdsourcing platforms, such as Amazon Mechanical Turk (MTurk), Prolific, Qualtrics Online Panel, and their lesser known competitors. The key benefit of all of these services is an easy and affordable access to a large pool of diverse participants, a privilege that was previously reserved for globally leading and financially independent universities. However, this newly achieved leveled playing field comes at a cost. Semi- or fully automated response tools, also called bots, decrease data quality and reliability. This case describes how two online studies were conducted on a crowdsourcing platform in anticipation of bot responses. Specifically, the case offers insights into the study design process, the selection of appropriate survey questions and bot traps, as well as the ex-post analysis and filtering of bot responses. Best practices are identified, and potential pitfalls explained. The description should aid readers in designing anticipatory online studies and experiments to increase their data quality, validity, and reliability.

Item Type: Article
Additional Information: Online ISBN: 9781529601312
Keywords: Bot Detection, Online Study, MTurk, Survey Bot
Schools and Departments: University of Sussex Business School > Strategy and Marketing
SWORD Depositor: Mx Elements Account
Depositing User: Mx Elements Account
Date Deposited: 01 Oct 2021 09:17
Last Modified: 23 Mar 2022 12:13

View download statistics for this item

📧 Request an update