Language Models (Mostly) Do Not Consider Emotion Triggers When Predicting Emotion

Smriti Singh, Cornelia Caragea, Junyi Jessy Li

Abstract
Situations and events evoke emotions in humans, but to what extent do they inform the prediction of emotion detection models? This work investigates how well human-annotated emotion triggers correlate with features that models deemed salient in their prediction of emotions. First, we introduce a novel dataset EmoTrigger, consisting of 900 social media posts sourced from three different datasets; these were annotated by experts for emotion triggers with high agreement. Using EmoTrigger, we evaluate the ability of large language models (LLMs) to identify emotion triggers, and conduct a comparative analysis of the features considered important for these tasks between LLMs and fine-tuned models. Our analysis reveals that emotion triggers are largely not considered salient features for emotion prediction models, instead there is intricate interplay between various features and the task of emotion detection.
Anthology ID:
2024.naacl-short.51
Volume:
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
603–614
Language:
URL:
https://aclanthology.org/2024.naacl-short.51
DOI:
Bibkey:
Cite (ACL):
Smriti Singh, Cornelia Caragea, and Junyi Jessy Li. 2024. Language Models (Mostly) Do Not Consider Emotion Triggers When Predicting Emotion. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 603–614, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
Language Models (Mostly) Do Not Consider Emotion Triggers When Predicting Emotion (Singh et al., NAACL 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/corrections-2024-07/2024.naacl-short.51.pdf
PDF Search