@inproceedings{lv-etal-2026-beyond,
    title = "Beyond Query Memorization: Large Language Model Routing with Query Decomposition and Historical Matching",
    author = "Lv, Bo  and
      Sun, Jingbo  and
      Lv, Jianwei  and
      Tang, Chen  and
      Zhang, Shaojie  and
      Liu, Nayu  and
      Yu, Guoxin  and
      Li, Zihao  and
      Zhang, Qichao  and
      Zhao, Dongbin  and
      Luo, Ping  and
      Yu, Yue",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1852/",
    pages = "39876--39892",
    ISBN = "979-8-89176-390-6",
    abstract = "Optimizing the trade-off among predictive performance and computational cost is a central focus in the deployment of Large Language Models (LLMs). Current routing methods primarily rely on direct mapping from queries to models based on surface-level features, making them susceptible to the memorization trap and leading to poor generalizability on out-of-distribution (OOD) data. In this paper, we propose DecoR, a novel routing framework that recasts the routing task as a matching process of sifting similar queries from historical logs, effectively mitigating the memorization trap. To enhance matching accuracy, we introduce a query capability deconstruction method that decouples linguistic surface forms from task-intrinsic requirements, directing matching toward capability dimensions to ground decisions in essential task attributes. Furthermore, we develop CodaSet, a comprehensive benchmark for assessing routing generalization, where experimental results demonstrate that DecoR maintains superior accuracy while substantially lowering inference costs across both in-distribution and OOD settings."
}