Yujia Fu
2026
FLARE: Fine-Grained Length-Aware Routing for Resource-Efficient Heterogeneous LLM Serving
Yujia Fu | Heming Zhong | Dan Huang | Yutong Lu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yujia Fu | Heming Zhong | Dan Huang | Yutong Lu
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
With the rapid proliferation of large language models (LLMs), model pools have become increasingly heterogeneous in both capability and efficiency. Larger LLMs can improve quality but incur higher latency and cost, while smaller LLMs are the opposite, making per-query model selection crucial in practice. This has spawned LLM routers that dispatch each query to an appropriate model. Existing routers lack fine-grained resource awareness across deployment settings, which degrades efficiency metrics in real-world serving. To this end, We propose FLARE, a length-centric, resource-aware multi-LLM routing framework that uses length-based models to estimate per-query latency and cost. FLARE formulates routing as a discrete multi-objective optimization problem to achieve efficient trade-off. Experiments show that FLARE reduces latency and cost by up to 68% and 75% while maintaining competitive accuracy, and can be easily applied to new datasets and LLMs.
2024
How Do Your Code LLMs perform? Empowering Code Instruction Tuning with Really Good Data
Yejie Wang | Keqing He | Dayuan Fu | Zhuoma GongQue | Heyang Xu | Yanxu Chen | Zhexu Wang | Yujia Fu | Guanting Dong | Muxi Diao | Jingang Wang | Mengdi Zhang | Xunliang Cai | Weiran Xu
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Yejie Wang | Keqing He | Dayuan Fu | Zhuoma GongQue | Heyang Xu | Yanxu Chen | Zhexu Wang | Yujia Fu | Guanting Dong | Muxi Diao | Jingang Wang | Mengdi Zhang | Xunliang Cai | Weiran Xu
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Recently, there has been a growing interest in studying how to construct better code instruction tuning data. However, we observe Code models trained with these datasets exhibit high performance on HumanEval but perform worse on other benchmarks such as LiveCodeBench. Upon further investigation, we find that many datasets suffer from severe data leakage. After cleaning up most of the leaked data, some well-known high-quality datasets perform poorly. This discovery reveals a new challenge: identifying which dataset genuinely qualify as high-quality code instruction data. To address this, we propose an efficient code data pruning strategy for selecting good samples. Our approach is based on three dimensions: instruction complexity, response quality, and instruction diversity. Based on our selected data, we present XCoder, a family of models finetuned from LLaMA3. Our experiments show Xcoder achieves new state-of-the-art performance using fewer training data, which verify the effectiveness of our data strategy. Moreover, we perform a comprehensive analysis on the data composition and find existing code datasets have different characteristics according to their construction methods, which provide new insights for future code LLMs.