Beijun Shen
2026
SWE-QA: Can Language Models Answer Repository-level Code Questions?
Weihan Peng | Yuling Shi | Yuhang Wang | Xinyun Zhang | Beijun Shen | Xiaodong Gu
Findings of the Association for Computational Linguistics: ACL 2026
Weihan Peng | Yuling Shi | Yuhang Wang | Xinyun Zhang | Beijun Shen | Xiaodong Gu
Findings of the Association for Computational Linguistics: ACL 2026
Understanding and reasoning about entire soft-ware repositories is an essential capability for intelligent software engineering tools. While existing benchmarks such as CoSQA and CodeQA have advanced the field, they predominantly focus on small, self-contained code snippets. These setups fail to capture the complexity of real-world repositories, where effective understanding and reasoning often require navigating multiple files, understanding software architecture, and grounding answers in long-range code dependencies. In this paper, we present SWE-QA, a repository-level code question answering (QA) benchmark designed to facilitate research on automated QA systems in realistic code environments. SWE-QA involves 720 high-quality question-answer pairs spanning diverse categories, including intention understanding, cross-file reasoning, and multi-hop dependency analysis. To construct SWE-QA, we first crawled 77,100 GitHub issues from 12 popular repositories. Based on an analysis of naturally occurring developer questions extracted from these issues, we developed a two-level taxonomy of repository-level questions and constructed a set of seed questions for each category. For each category, we manually curated and validated questions and collected their corresponding answers. We evaluate six advanced LLMs on SWE-QA under various context augmentation strategies. Experimental results highlight the promise of LLMs.
GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughts
Wenhao Zeng | Xuteng Zhang | Yuling Shi | Chao Hu | Yuting Chen | Beijun Shen | Xiaodong Gu
Findings of the Association for Computational Linguistics: ACL 2026
Wenhao Zeng | Xuteng Zhang | Yuling Shi | Chao Hu | Yuting Chen | Beijun Shen | Xiaodong Gu
Findings of the Association for Computational Linguistics: ACL 2026
Large Reasoning Models (LRMs) achieve remarkable performance by explicitly generating multi-step chains of thought, but this capability incurs substantial inference latency and computational cost. Collaborative inference offers a promising solution by selectively allocating work between lightweight and large models, yet a fundamental challenge remains: determining when a reasoning step requires the capacity of a large model or the efficiency of a small model. Existing routing strategies either rely on local token probabilities or post-hoc verification, introducing significant inference overhead. In this work, we propose a novel perspective on step-wise collaboration: the difficulty of a reasoning step can be inferred from its very first token. Inspired by the “Aha Moment” phenomenon in LRMs, we show that the entropy of the initial token serves as a strong predictor of step difficulty. Building on this insight, we introduce GlimpRouter, a training-free step-wise collaboration framework. GlimpRouter employs a lightweight model to generate only the first token of each reasoning step and routes the step to a larger model only when the initial token entropy exceeds a threshold. Experiments on multiple benchmarks demonstrate that our approach significantly reduces inference latency while preserving accuracy. For instance, GlimpRouter attains a substantial 10.7% improvement in accuracy while reducing inference latency by 25.9% compared to a standalone large model on AIME25. These results suggest a simple yet effective mechanism for reasoning: allocating computation based on a glimpse of thought rather than full-step evaluation.
2025
Transplant Then Regenerate: A New Paradigm for Text Data Augmentation
Guangzhan Wang | Hongyu Zhang | Beijun Shen | Xiaodong Gu
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Guangzhan Wang | Hongyu Zhang | Beijun Shen | Xiaodong Gu
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Data augmentation is a critical technique in deep learning. Traditional methods like Back-translation typically focus on lexical-level rephrasing, which primarily produces variations with the same semantics. While large language models (LLMs) have enhanced text augmentation by their “knowledge emergence” capability, controlling the style and structure of these outputs remains challenging and requires meticulous prompt engineering. In this paper, we propose LMTransplant, a novel text augmentation paradigm leveraging LLMs. The core idea of LMTransplant is transplant-then-regenerate: incorporating seed text into a context expanded by LLM, and asking the LLM to regenerate a variant based on the expanded context. This strategy allows the model to create more diverse and creative content-level variants by fully leveraging the knowledge embedded in LLMs, while preserving the core attributes of the original text. We evaluate LMTransplant across various text-related tasks, demonstrating its superior performance over existing text augmentation methods. Moreover, LMTransplant demonstrates exceptional scalability as the size of augmented data grows.