Abstract
Working with a wide range of annotators with the same attributes is crucial, as in real-world applications. Although such application cases often use crowd-sourcing mechanisms to gather a variety of annotators, most real-world users use mobile devices. In this paper, we propose “FAST,” an annotation tool for application tasks that focuses on the user experience of mobile devices, which has not yet been focused on thus far. We designed FAST as a web application for use on any device with a flexible interface that can be customized to fit various tasks. In our experiments, we conducted crowd-sourced annotation for a sentiment analysis task with several annotators and evaluated annotation metrics such as speed, quality, and ease of use from the tool’s logs and user surveys. Based on the results of our experiments, we conclude that our system can annotate faster than existing methods while maintaining the annotation quality.- Anthology ID:
- 2021.emnlp-demo.41
- Volume:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
- Month:
- November
- Year:
- 2021
- Address:
- Online and Punta Cana, Dominican Republic
- Editors:
- Heike Adel, Shuming Shi
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 372–381
- Language:
- URL:
- https://aclanthology.org/2021.emnlp-demo.41
- DOI:
- 10.18653/v1/2021.emnlp-demo.41
- Cite (ACL):
- Shunyo Kawamoto, Yu Sawai, Kohei Wakimoto, and Peinan Zhang. 2021. FAST: Fast Annotation tool for SmarT devices. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 372–381, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- FAST: Fast Annotation tool for SmarT devices (Kawamoto et al., EMNLP 2021)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2021.emnlp-demo.41.pdf
- Code
- cyberagent/fast-annotation-tool