Knowledge Distillation from Monolingual to Multilingual Models for Intelligent and Interpretable Multilingual Emotion Detection
Yuqi Wang, Zimu Wang, Nijia Han, Wei Wang, Qi Chen, Haiyang Zhang, Yushan Pan, Anh Nguyen
Abstract
Emotion detection from text is a crucial task in understanding natural language with wide-ranging applications. Existing approaches for multilingual emotion detection from text face challenges with data scarcity across many languages and a lack of interpretability. We propose a novel method that leverages both monolingual and multilingual pre-trained language models to improve performance and interpretability. Our approach involves 1) training a high-performing English monolingual model in parallel with a multilingual model and 2) using knowledge distillation to transfer the emotion detection capabilities from the monolingual teacher to the multilingual student model. Experiments on a multilingual dataset demonstrate significant performance gains for refined multilingual models like XLM-RoBERTa and E5 after distillation. Furthermore, our approach enhances interpretability by enabling better identification of emotion-trigger words. Our work presents a promising direction for building accurate, robust and explainable multilingual emotion detection systems.- Anthology ID:
- 2024.wassa-1.45
- Volume:
- Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Orphée De Clercq, Valentin Barriere, Jeremy Barnes, Roman Klinger, João Sedoc, Shabnam Tafreshi
- Venues:
- WASSA | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 470–475
- Language:
- URL:
- https://aclanthology.org/2024.wassa-1.45
- DOI:
- 10.18653/v1/2024.wassa-1.45
- Cite (ACL):
- Yuqi Wang, Zimu Wang, Nijia Han, Wei Wang, Qi Chen, Haiyang Zhang, Yushan Pan, and Anh Nguyen. 2024. Knowledge Distillation from Monolingual to Multilingual Models for Intelligent and Interpretable Multilingual Emotion Detection. In Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 470–475, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Knowledge Distillation from Monolingual to Multilingual Models for Intelligent and Interpretable Multilingual Emotion Detection (Wang et al., WASSA-WS 2024)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2024.wassa-1.45.pdf