Data reliability in citizen science: learning curve and the effects of training method, volunteer background and experience on identification accuracy of insects visiting ivy flowers

Ratnieks, Francis L W, Schrell, Felix, Sheppard, Rebecca C, Brown, Emmeline, Bristow, Oliver E and Garbuzov, Mihail (2016) Data reliability in citizen science: learning curve and the effects of training method, volunteer background and experience on identification accuracy of insects visiting ivy flowers. Methods in Ecology and Evolution, 7 (10). pp. 1226-1235. ISSN 2041-2096

[img] PDF - Accepted Version
Available under License Creative Commons Attribution.

Download (898kB)


• Citizen science, the involvement of volunteers in collecting of scientific data, can be a useful research tool. However, data collected by volunteers are often of lower quality than that collected by professional scientists.
• We studied the accuracy with which volunteers identified insects visiting ivy (Hedera) flowers in Sussex, England. In the first experiment, we examined the effects of training method, volunteer background and prior experience. Fifty-three participants were trained for the same duration using one of three different methods (pamphlet, pamphlet + slide show, pamphlet + direct training). Almost immediately following training, we tested the ability of participants to identify live insects on ivy flowers to one of 10 taxonomic categories and recorded whether their identifications were correct or incorrect, without providing feedback.
• The results showed that the type of training method had a significant effect on identification accuracy (P = 0.008). Participants identified 79.1% of insects correctly after using a one-page colour pamphlet, 85.6% correctly after using the pamphlet and viewing a slide show, and 94.3% correctly after using the pamphlet in combination with direct training in the field.
• As direct training cannot be delivered remotely, in the following year we conducted a second experiment, in which a different sample of 26 volunteers received the pamphlet plus slide show training repeatedly three times. Moreover, in this experiment participants received c. 2 minutes of additional training material, either videos of insects or stills taken from the videos. Testing showed that identification accuracy increased from 88.6% to 91.3% to 97.5% across the three successive tests. We also found a borderline significant interaction between the type of additional material and the test number (P = 0.053), such that the video gave fewer errors than stills in the first two tests only.
• The most common errors made by volunteers were misidentifications of honey bees and social wasps with their hover fly mimics. We also tested six experts who achieved nearly perfect accuracy (99.8%), which shows what is possible in practice.
• Overall, our study shows that two or three sessions of remote training can be as good as one of direct training, even for relatively challenging taxonomic discriminations that include distinguishing models and mimics.

Item Type: Article
Keywords: Batesian mimicry, Citizen science, Data quality, Insect identification, Learner effect
Schools and Departments: School of Life Sciences > Evolution, Behaviour and Environment
Subjects: Q Science > QL Zoology > QL0360 Invertebrates > QL0434 Arthropoda > QL0463 Insects
Related URLs:
Depositing User: Mihail Garbuzov
Date Deposited: 26 Apr 2016 15:41
Last Modified: 15 Mar 2017 13:55

View download statistics for this item

📧 Request an update
Project NameSussex Project NumberFunderFunder Ref
Honey Bee DonationsG0168ROWSE HONEY LTDUnset