Multi-modal Multi-label Emotion Detection with Modality and Label Dependence
Dong Zhang, Xincheng Ju, Junhui Li, Shoushan Li, Qiaoming Zhu, Guodong Zhou
Abstract
As an important research issue in the natural language processing community, multi-label emotion detection has been drawing more and more attention in the last few years. However, almost all existing studies focus on one modality (e.g., textual modality). In this paper, we focus on multi-label emotion detection in a multi-modal scenario. In this scenario, we need to consider both the dependence among different labels (label dependence) and the dependence between each predicting label and different modalities (modality dependence). Particularly, we propose a multi-modal sequence-to-set approach to effectively model both kinds of dependence in multi-modal multi-label emotion detection. The detailed evaluation demonstrates the effectiveness of our approach.- Anthology ID:
- 2020.emnlp-main.291
- Volume:
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3584–3593
- Language:
- URL:
- https://aclanthology.org/2020.emnlp-main.291
- DOI:
- 10.18653/v1/2020.emnlp-main.291
- Cite (ACL):
- Dong Zhang, Xincheng Ju, Junhui Li, Shoushan Li, Qiaoming Zhu, and Guodong Zhou. 2020. Multi-modal Multi-label Emotion Detection with Modality and Label Dependence. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3584–3593, Online. Association for Computational Linguistics.
- Cite (Informal):
- Multi-modal Multi-label Emotion Detection with Modality and Label Dependence (Zhang et al., EMNLP 2020)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/2020.emnlp-main.291.pdf