SemEval 2024 - Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF)

Shivani Kumar, Md. Shad Akhtar, Erik Cambria, Tanmoy Chakraborty

Abstract
We present SemEval-2024 Task 10, a shared task centred on identifying emotions and finding the rationale behind their flips within monolingual English and Hindi-English code-mixed dialogues. This task comprises three distinct subtasks – emotion recognition in conversation for code-mixed dialogues, emotion flip reasoning for code-mixed dialogues, and emotion flip reasoning for English dialogues. Participating systems were tasked to automatically execute one or more of these subtasks. The datasets for these tasks comprise manually annotated conversations focusing on emotions and triggers for emotion shifts.1 A total of 84 participants engaged in this task, with the most adept systems attaining F1-scores of 0.70, 0.79, and 0.76 for the respective subtasks. This paper summarises the results and findings from 24 teams alongside their system descriptions.
Anthology ID:
2024.semeval-1.270
Volume:
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
1933–1946
Language:
URL:
https://aclanthology.org/2024.semeval-1.270
DOI:
Bibkey:
Cite (ACL):
Shivani Kumar, Md. Shad Akhtar, Erik Cambria, and Tanmoy Chakraborty. 2024. SemEval 2024 - Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF). In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 1933–1946, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
SemEval 2024 - Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF) (Kumar et al., SemEval 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/bionlp-24-ingestion/2024.semeval-1.270.pdf
Supplementary material:
 2024.semeval-1.270.SupplementaryMaterial.txt
PDF Search Supplementary material