Abstract
Emotion detection from social media posts has attracted noticeable attention from natural language processing (NLP) community in recent years. The ways for obtaining gold labels for training and testing of the systems for automatic emotion detection differ significantly from one study to another, and pose the question of reliability of gold labels and obtained classification results. This study systematically explores several ways for obtaining gold labels for Ekman’s emotion model on Twitter data and the influence of the chosen strategy on the manual classification results.- Anthology ID:
- 2021.ranlp-1.151
- Volume:
- Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
- Month:
- September
- Year:
- 2021
- Address:
- Held Online
- Venue:
- RANLP
- SIG:
- Publisher:
- INCOMA Ltd.
- Note:
- Pages:
- 1350–1359
- Language:
- URL:
- https://aclanthology.org/2021.ranlp-1.151
- DOI:
- Cite (ACL):
- Sanja Stajner. 2021. Exploring Reliability of Gold Labels for Emotion Detection in Twitter. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1350–1359, Held Online. INCOMA Ltd..
- Cite (Informal):
- Exploring Reliability of Gold Labels for Emotion Detection in Twitter (Stajner, RANLP 2021)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2021.ranlp-1.151.pdf