It’s All Fun and Games until Someone Annotates: Video Games with a Purpose for Linguistic Annotation
Abstract
Annotated data is prerequisite for many NLP applications. Acquiring large-scale annotated corpora is a major bottleneck, requiring significant time and resources. Recent work has proposed turning annotation into a game to increase its appeal and lower its cost; however, current games are largely text-based and closely resemble traditional annotation tasks. We propose a new linguistic annotation paradigm that produces annotations from playing graphical video games. The effectiveness of this design is demonstrated using two video games: one to create a mapping from WordNet senses to images, and a second game that performs Word Sense Disambiguation. Both games produce accurate results. The first game yields annotation quality equal to that of experts and a cost reduction of 73% over equivalent crowdsourcing; the second game provides a 16.3% improvement in accuracy over current state-of-the-art sense disambiguation games with WordNet.- Anthology ID:
- Q14-1035
- Volume:
- Transactions of the Association for Computational Linguistics, Volume 2
- Month:
- Year:
- 2014
- Address:
- Cambridge, MA
- Editors:
- Dekang Lin, Michael Collins, Lillian Lee
- Venue:
- TACL
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 449–464
- Language:
- URL:
- https://aclanthology.org/Q14-1035
- DOI:
- 10.1162/tacl_a_00195
- Cite (ACL):
- David Jurgens and Roberto Navigli. 2014. It’s All Fun and Games until Someone Annotates: Video Games with a Purpose for Linguistic Annotation. Transactions of the Association for Computational Linguistics, 2:449–464.
- Cite (Informal):
- It’s All Fun and Games until Someone Annotates: Video Games with a Purpose for Linguistic Annotation (Jurgens & Navigli, TACL 2014)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/Q14-1035.pdf
- Data
- ImageNet