Exploring Reliability of Gold Labels for Emotion Detection in Twitter

Sanja Stajner


Abstract
Emotion detection from social media posts has attracted noticeable attention from natural language processing (NLP) community in recent years. The ways for obtaining gold labels for training and testing of the systems for automatic emotion detection differ significantly from one study to another, and pose the question of reliability of gold labels and obtained classification results. This study systematically explores several ways for obtaining gold labels for Ekman’s emotion model on Twitter data and the influence of the chosen strategy on the manual classification results.
Anthology ID:
2021.ranlp-1.151
Volume:
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
Month:
September
Year:
2021
Address:
Held Online
Editors:
Ruslan Mitkov, Galia Angelova
Venue:
RANLP
SIG:
Publisher:
INCOMA Ltd.
Note:
Pages:
1350–1359
Language:
URL:
https://aclanthology.org/2021.ranlp-1.151
DOI:
Bibkey:
Cite (ACL):
Sanja Stajner. 2021. Exploring Reliability of Gold Labels for Emotion Detection in Twitter. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1350–1359, Held Online. INCOMA Ltd..
Cite (Informal):
Exploring Reliability of Gold Labels for Emotion Detection in Twitter (Stajner, RANLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl-24-ws-corrections/2021.ranlp-1.151.pdf