Chaomeng Chen
2026
X-Router: Decoupling Knowledge and Reasoning for Cost-Effective LLM Inference
Zixuan Wang | Yinze Ding | Zihan Wang | Jinyu Guo | Zhenhong Zhou | Junhao Dong | Chaomeng Chen
Findings of the Association for Computational Linguistics: ACL 2026
Zixuan Wang | Yinze Ding | Zihan Wang | Jinyu Guo | Zhenhong Zhou | Junhao Dong | Chaomeng Chen
Findings of the Association for Computational Linguistics: ACL 2026
Large Language Models (LLMs) are often augmented with Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) prompting, yet static “always-on” use is computationally wasteful. Existing adaptive methods typically optimize a single axis, overlooking that evidence need and reasoning depth are only partially correlated. We present , a dual-axis routing framework that separates retrieval necessity from reasoning necessity under a user-defined cost–quality trade-off. Offline, profiles four pipelines (Direct, RAG, CoT, RAG+CoT) and derives supervision by selecting the utility-maximizing strategy that trades answer quality against token usage and latency. Online, a compact dual-head router, conditioned on cost weights, uses lightweight probes—retrieval-score dispersion (NQC) and single-pass draft negative log-likelihood (NLL)—to decide whether to invoke RAG and/or CoT without sampling or model internals. Across six QA benchmarks, reduces token usage by up to 86% and latency by up to 84% while improving answer quality over strong baselines.