MUSER: MUltimodal Stress detection using Emotion Recognition as an Auxiliary Task

Yiqun Yao, Michalis Papakostas, Mihai Burzo, Mohamed Abouelenien, Rada Mihalcea


Abstract
The capability to automatically detect human stress can benefit artificial intelligent agents involved in affective computing and human-computer interaction. Stress and emotion are both human affective states, and stress has proven to have important implications on the regulation and expression of emotion. Although a series of methods have been established for multimodal stress detection, limited steps have been taken to explore the underlying inter-dependence between stress and emotion. In this work, we investigate the value of emotion recognition as an auxiliary task to improve stress detection. We propose MUSER – a transformer-based model architecture and a novel multi-task learning algorithm with speed-based dynamic sampling strategy. Evaluation on the Multimodal Stressed Emotion (MuSE) dataset shows that our model is effective for stress detection with both internal and external auxiliary tasks, and achieves state-of-the-art results.
Anthology ID:
2021.naacl-main.216
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2714–2725
Language:
URL:
https://aclanthology.org/2021.naacl-main.216
DOI:
10.18653/v1/2021.naacl-main.216
Bibkey:
Cite (ACL):
Yiqun Yao, Michalis Papakostas, Mihai Burzo, Mohamed Abouelenien, and Rada Mihalcea. 2021. MUSER: MUltimodal Stress detection using Emotion Recognition as an Auxiliary Task. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2714–2725, Online. Association for Computational Linguistics.
Cite (Informal):
MUSER: MUltimodal Stress detection using Emotion Recognition as an Auxiliary Task (Yao et al., NAACL 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2021.naacl-main.216.pdf
Video:
 https://preview.aclanthology.org/ingestion-script-update/2021.naacl-main.216.mp4
Data
OMG-Emotion