Richang Hong
2026
Beyond Semantic Similarity: Appraisal-Guided Chain-of-Thought Reasoning and Retrieval for Multimodal Emotional Support Conversations
Yuqi Chu | Lizi Liao | Jinggui Liang | Boyang Li | Richang Hong
Findings of the Association for Computational Linguistics: ACL 2026
Yuqi Chu | Lizi Liao | Jinggui Liang | Boyang Li | Richang Hong
Findings of the Association for Computational Linguistics: ACL 2026
Emotional support conversation systems strive to emulate the empathetic depth of human therapists, yet current approaches often fail due to the "Cognitive Gap"—the inability to discern the latent psychological evaluations driving a user’s distress. Existing retrieval-augmented generation paradigms exacerbate this by relying on semantic similarity, frequently retrieving historical dialogues that are surface analogous but therapeutically incongruent. To bridge this gap, we introduce Appraisal-Guided Chain-of-Thought Reasoning & Retrieval (AG-CTR²) for better emotional support. Specifically, we bootstrap the MLLM to generate appraisal-guided reasoning chains and apply a dual-signal verification mechanism using ground-truth emotion labels and a teacher model to verify and correct them. Under such instance-level guidance, we finetune the MLLM to internalize such reasoning capability. At inference, the model utilizes its generated appraisal chain as a structured query to help retrieve historical therapeutic responses based on psychological situation similarity rather than content surface proximity. Extensive experiments and analyses on two ESC benchmarks demonstrate that AG-CTR² significantly outperforms state-of-the-art baselines.
QueueEDIT: Structural Self-Correction for Sequential Model Editing in LLMs
Taolin Zhang | Haidong Kang | Dongyang Li | Qizhou Chen | Xiaofeng He | Chengyu Wang | Richang Hong
Findings of the Association for Computational Linguistics: ACL 2026
Taolin Zhang | Haidong Kang | Dongyang Li | Qizhou Chen | Xiaofeng He | Chengyu Wang | Richang Hong
Findings of the Association for Computational Linguistics: ACL 2026
Recently, large language models (LLMs) have demonstrated impressive performance but still suffer from hallucinations. Model editing has been proposed as a means to correct factual inaccuracies. A challenging scenario is sequential model editing (SME), which aims to rectify errors continuously, rather than a one-time task. During SME, the general capabilities of LLMs can be negatively affected due to the introduction of new parameters. In this paper, we propose a queue-based self-correction framework, QueueEDIT, that not only enhances SME performance by addressing long-sequence dependencies but also mitigates the impact of parameter bias on the general capabilities of LLMs. Specifically, we first introduce a structural mapping editing loss to map editing triplets to knowledge-sensitive neurons within the Transformer layers. We then store the located parameters for each piece of edited knowledge in a queue and dynamically align previously edited parameters. At each edit, we select parameters in the queue that are most relevant to currently located parameters to determine whether knowledge associated with previous edits requires realignment. Irrelevant parameters in the queue are frozen, and we update the parameters at the queue head into the LLM to ensure they do not harm general capabilities. Experiments show that QueueEDIT significantly outperforms strong baselines across various SME settings, while maintaining competitive performance in single-turn editing. Resulting LLMs also preserve high performance on general NLP tasks throughout the SME process.
Taming "Zombie" Agents: A Markov State-Aware Framework for Resilient Multi-Agent Evolution
Taolin Zhang | Pukun Zhao | Qizhou Chen | Jiuheng Wan | Chen Chen | Xiaofeng He | Chengyu Wang | Richang Hong
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Taolin Zhang | Pukun Zhao | Qizhou Chen | Jiuheng Wan | Chen Chen | Xiaofeng He | Chengyu Wang | Richang Hong
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Recent advancements in LLM-based multi-agent systems have demonstrated remarkable collaborative capabilities across complex tasks. To enhance the overall efficiency, existing methods often rely on aggressive graph topology evolution for agents (e.g., node or edge pruning), which risks prematurely discarding valuable agents due to transient issues such as hallucinations or temporary knowledge gaps. However, such hard pruning overlooks the potential for "zombie" agents to recover and contribute in subsequent discussion rounds. In this paper, we propose AgentRevive, a Markov state-aware framework for resilient multi-agent evolution. Our approach dynamically manages agent collaboration through soft state transitions, implemented via two key components: (1) State-Aware Policy Learning: Agent states are divided into "Active", "Standby", and "Terminated", selectively propagating messages based on agent memory. The policy employs a risk estimator to optimize agent state transitions by assessing hallucination risk, minimizing the influence of unreliable nodes while safeguarding valuable ones. (2) State-Aware Edge Optimization: Subgraph edges are pruned according to states learned from the policy, permanently removing "Terminated" nodes and retaining "Standby" nodes for subsequent rounds to observe potential future contributions. Extensive experiments on general reasoning, domain-specific, and hallucination challenge tasks show that our method consistently outperforms strong baselines and significantly reduces token consumption through state-aware agent scheduling.
AMATA: Adaptive Multi-Agent Trajectory Alignment for Knowledge-Intensive Question Answering
Taolin Zhang | Dongyang Li | Chen Chen | Qizhou Chen | Jiuheng Wan | Xiaofeng He | Chengyu Wang | Richang Hong
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Taolin Zhang | Dongyang Li | Chen Chen | Qizhou Chen | Jiuheng Wan | Xiaofeng He | Chengyu Wang | Richang Hong
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Despite substantial advances in large language models (LLMs), producing factually consistent responses for knowledge-intensive question answering remains challenging. These difficulties are primarily due to hallucination and the limitations of LLMs in bridging long-tail knowledge gaps. To address this, we propose AMATA, an Adaptive Multi-Agent Trajectory Alignment framework that dynamically integrates external knowledge to improve response interpretability and factual grounding. Our architecture leverages six specialized agents that collaboratively perform structured actions for complex question reasoning. We formalize multi-agent collaboration with external tools as a trajectory preference alignment problem, incorporating question-aware agent customization and inter-agent preference harmonization. AMATA introduces two principal innovations: (1) Intra-Trajectory Preference Learning, which learns objective-oriented preferences to prioritize critical agents, and (2) Inter-Agent Dependency Learning, which captures cross-agent tool dependencies through a novel dependency-aware direct preference optimization technique. Empirical results show that AMATA consistently outperforms baseline approaches, knowledge-augmented frameworks, and LLM-based trajectory systems on five established knowledge-intensive QA benchmarks. Further analysis demonstrates the efficiency of our method in reducing token consumption.
2025
SURE: Safety Understanding and Reasoning Enhancement for Multimodal Large Language Models
Yuxin Gou | Xiaoning Dong | Qin Li | Shishen Gu | Richang Hong | Wenbo Hu
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Yuxin Gou | Xiaoning Dong | Qin Li | Shishen Gu | Richang Hong | Wenbo Hu
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Multimodal large language models (MLLMs) demonstrate impressive capabilities by integrating visual and textual information. However, the incorporation of visual modalities also introduces new and complex safety risks, rendering even the most advanced models vulnerable to sophisticated jailbreak attacks. This paper first analyzes the impact of inserting safety reasoning prompt on various aspects of the model. We find that this external method can help the model resist jailbreak attacks to some extent, but the model still fails to distinguish specific semantic scenarios, resulting in a significantly increased refusal rate for benign queries. Inspired by this, we propose a novel training framework, SURE (Safety Understanding and Reasoning Enhancement for Multimodal Large Language Models), designed to help models internalize chain-of-thought-based safety decision-making capabilities. Extensive experiments demonstrate that SURE significantly improves model safety while effectively avoiding over-defense, achieving a good balance between safety and generality. Finally, we create a large-scale multimodal safety reasoning dataset, MLLM-SCoT-Plus, to facilitate research on safety alignment in multimodal models.Our code and the dataset are publicly available at https://github.com/hfutml/SURE.
Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models
Zijun Chen | Wenbo Hu | Guande He | Zhijie Deng | ZHeng ZHang | Richang Hong
Proceedings of the 31st International Conference on Computational Linguistics
Zijun Chen | Wenbo Hu | Guande He | Zhijie Deng | ZHeng ZHang | Richang Hong
Proceedings of the 31st International Conference on Computational Linguistics
Multimodal large language models (MLLMs) combine visual and textual data for tasks like image captioning and visual question answering. Proper uncertainty calibration is crucial but challenging for reliable use in areas like healthcare and autonomous driving. This paper investigates several MLLMs, focusing on their calibration across various scenarios, including before and after visual fine-tuning as well as before and after multimodal training of the base LLMs. We observed miscalibration in their performance, and at the same time, no significant differences in calibration across these scenarios. We also highlight differences in uncertainty between text and the impact of the integration of these two types of information in uncertainty. To better understand MLLMs’ miscalibration and their ability to self-assess uncertainty, we developed the IDK (I don’t know) dataset, which is key for evaluating how they handle unknowns. Our findings reveal that MLLMs tend to give answers rather than admit uncertainty, but this self-assessment improves with prompt adjustments. Finally, to calibrate MLLMs and enhance model reliability, we propose techniques such as temperature scaling and iterative prompt optimization. Our results provide insights into improving MLLMs for effective and responsible deployment in multimodal applications.