Ning Yuan
2026
JX4MEI: Multimodal Semantically-Enhanced LLM for Joint Multimodal Emotion-Intent Explanation and Classification
YiJie Huang | Xiaocui Yang | Shi Feng | Daling Wang | Yifei Zhang | Ning Yuan | Zhuoyue Jia | Wen Zhang
Findings of the Association for Computational Linguistics: ACL 2026
YiJie Huang | Xiaocui Yang | Shi Feng | Daling Wang | Yifei Zhang | Ning Yuan | Zhuoyue Jia | Wen Zhang
Findings of the Association for Computational Linguistics: ACL 2026
Existing multimodal emotion and intent recognition tasks predominantly focus on classification, overlooking the underlying rationale and intrinsic connections between these states. Bridging this gap, we propose Joint Multimodal Emotion-Intent Explanation and Classification, JX4MEI, a novel task requiring the model to jointly predict emotion and intent, while generating natural language explanations for why they co-occur. To support this, we present XMEI-dataset, a large-scale benchmark of 15,461 multimodal samples spanning 7 emotion and 9 intent categories across text, audio, and visual modalities. Unlike prior works, our dataset provides fine-grained rationales for emotion, intent, and their causal interplay, curated via a rigorous pipeline involving Chain-of-Thought generation and strict human refinement to eliminate model artifacts. Furthermore, we propose XMEI-Qwen, a model equipped with a novel Language-Query Former (LQ-Former). By leveraging modality-specific captions as semantic queries, LQ-Former injects explicit semantic guidance into feature alignment, significantly enhancing reasoning capabilities. Empirical experiments demonstrate that XMEI-Qwen sets a new state-of-the-art on the JX4MEI task, outperforming competitive baselines in both prediction and explanation generation. Code: https://github.com/OrangeYeah1027/JX4MEI.