CMTD: Cognitive Modeling with Traits and Distortions for Multimodal Emotion Recognition in Conversations

Minh-Tien Nguyen, Huu-Loi Le, Manh-Cuong Phan, Hajime Hotta


Abstract
This paper introduces a new multi-agent framework, CMTD (Cognitive Modeling with Traits and Distortions), for multimodal emotion recognition in conversations (MERC). Instead of relying on shallow analysis of emotions, CMTD reconstructs a cognitive model by taking advantage of stable personality traits, dynamic cognitive distortions, visual and acoustic features of interlocutors to enhance the emotional intelligence of LLMs. CMTD includes trait, distortion detection, vision, and speech agents that provide psychological and multimodal indicators for the fusion agent to make the final prediction. Experimental results on MELD and IEMOCAP show that traits temper negativity bias from distortions, and cognitive modeling with psychological, visual, and acoustic information can improve the performance of MERC.CMTD is flexible and easy to adapt to advanced emotional AI systems (Github link: https://github.com/Shaun-le/CMTD.git).
Anthology ID:
2026.findings-acl.41
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
839–854
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.41/
DOI:
Bibkey:
Cite (ACL):
Minh-Tien Nguyen, Huu-Loi Le, Manh-Cuong Phan, and Hajime Hotta. 2026. CMTD: Cognitive Modeling with Traits and Distortions for Multimodal Emotion Recognition in Conversations. In Findings of the Association for Computational Linguistics: ACL 2026, pages 839–854, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
CMTD: Cognitive Modeling with Traits and Distortions for Multimodal Emotion Recognition in Conversations (Nguyen et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.41.pdf
Checklist:
 2026.findings-acl.41.checklist.pdf