Yi He
2026
Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoning
Fanding Huang | Guanbo Huang | Xiao Fan | Yi He | Xiao Liang | Xiao Chen | Qinting Jiang | Faisal Nadeem Khan | Jingyan Jiang | Zhi Wang
Findings of the Association for Computational Linguistics: ACL 2026
Fanding Huang | Guanbo Huang | Xiao Fan | Yi He | Xiao Liang | Xiao Chen | Qinting Jiang | Faisal Nadeem Khan | Jingyan Jiang | Zhi Wang
Findings of the Association for Computational Linguistics: ACL 2026
Reinforcement Learning with Verifiable Rewards (RLVR) for LLM reasoning is often framed as balancing exploration and exploitation in action space, typically operationalized with token-level proxies (e.g., output entropy or confidence). We argue that this apparent trade-off is largely a measurement artifact: token-level statistics reflect next-token uncertainty rather than how reasoning progresses over multi-token semantic structures. We therefore study exploration and exploitation in the hidden-state space of response trajectories. We use Effective Rank (ER) to quantify representational exploration and introduce its temporal derivatives, Effective Rank Velocity (ERV) and Effective Rank Acceleration (ERA), to characterize exploitative refinement dynamics. Empirically and theoretically, ER and ERV exhibit near-zero correlation in semantic space, suggesting the two capacities can be improved simultaneously. Motivated by this, we propose Velocity-Exploiting Rank Learning (VERL), which shapes the RL advantage with an auxiliary signal derived from ER/ERV and uses the more stable ERA as a meta-control variable to adaptively balance the incentives. Across multiple base models, RL algorithms, and reasoning benchmarks, VERL yields consistent improvements, including large gains on challenging tasks (e.g., 21.4% in Gaokao 2024).
2025
Query-Driven Multimodal GraphRAG: Dynamic Local Knowledge Graph Construction for Online Reasoning
Chenyang Bu | Guojie Chang | Zihao Chen | CunYuan Dang | Zhize Wu | Yi He | Xindong Wu
Findings of the Association for Computational Linguistics: ACL 2025
Chenyang Bu | Guojie Chang | Zihao Chen | CunYuan Dang | Zhize Wu | Yi He | Xindong Wu
Findings of the Association for Computational Linguistics: ACL 2025
An increasing adoption of Large Language Models (LLMs) in complex reasoning tasks necessitates their interpretability and reliability. Recent advances to that end include retrieval-augmented generation (RAG) and knowledge graph-enhanced RAG (GraphRAG), whereas they are constrained by static knowledge bases and ineffective multimodal data integration. In response, we propose a Query-Driven Multimodal GraphRAG framework that dynamically constructs local knowledge graphs tailored to query semantics. Our approach 1) derives graph patterns from query semantics to guide knowledge extraction, 2) employs a multi-path retrieval strategy to pinpoint core knowledge, and 3) supplements missing multimodal information ad hoc. Experimental results on the MultimodalQA and WebQA datasets demonstrate that our framework achieves the state-of-the-art performance among unsupervised competitors, particularly excelling in cross-modal understanding of complex queries.