Rui Zhou
2026
Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts
Haolei Xu | Haiwen Hong | Hongxing Li | Rui Zhou | Yang Zhang | Longtao Huang | Hui Xue | Yongliang Shen | Weiming Lu | Yueting Zhuang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Haolei Xu | Haiwen Hong | Hongxing Li | Rui Zhou | Yang Zhang | Longtao Huang | Hui Xue | Yongliang Shen | Weiming Lu | Yueting Zhuang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Multimodal Mixture-of-Experts (MoE) models have achieved remarkable performance on vision-language tasks. However, we identify a puzzling phenomenon termed Seeing but Not Thinking: models accurately perceive image content yet fail in subsequent reasoning, while correctly solving identical problems presented as pure text. Through systematic analysis, we first verify that cross-modal semantic sharing exists in MoE architectures, ruling out semantic alignment failure as the sole explanation. We then reveal that visual experts and domain experts exhibit layer-wise separation, with image inputs inducing significant routing divergence from text inputs in middle layers where domain experts concentrate. Based on these findings, we propose the Routing Distraction hypothesis: when processing visual inputs, the routing mechanism fails to adequately activate task-relevant reasoning experts. To validate this hypothesis, we design a routing-guided intervention method that enhances domain expert activation. Experiments on three multimodal MoE models across six benchmarks demonstrate consistent improvements, with gains of up to 3.17% on complex visual reasoning tasks. Our analysis further reveals that domain expert identification locates cognitive functions rather than sample-specific solutions, enabling effective transfer across tasks with different information structures.
2023
Not The End of Story: An Evaluation of ChatGPT-Driven Vulnerability Description Mappings
Xin Liu | Yuan Tan | Zhenghang Xiao | Jianwei Zhuge | Rui Zhou
Findings of the Association for Computational Linguistics: ACL 2023
Xin Liu | Yuan Tan | Zhenghang Xiao | Jianwei Zhuge | Rui Zhou
Findings of the Association for Computational Linguistics: ACL 2023
As the number of vulnerabilities increases day by day, security management requires more and more structured data. In addition to textual descriptions of vulnerabilities, security engineers must classify and assess vulnerabilities and clarify their associated techniques. Vulnerability Description Mapping (VDM) refers to mapping vulnerabilities to Common Weakness Enumeration (CWE), Common Attack Pattern Enumeration and Classification, ATT&CK Techniques, and other classifications. Accurate VDM is necessary to reduce the pressure of security management and improve the speed of security emergency response. ChatGPT is the latest state-of-the-art closed-source conversational large language model (LLM), which performs excellently on many tasks. This paper explores the application of closed-source LLMs to real-world security management scenarios by evaluating ChatGPT’s performance on VDM tasks. The results show that although ChatGPT may be close to the level of human experts on some tasks, it still cannot replace the critical role of professional security engineers in vulnerability analysis. In a word, closed-source LLM is not the end of story.