@inproceedings{ou-etal-2026-adapshot,
    title = "{A}dap{S}hot: Adaptive Many-Shot In-Context Learning with Semantic-Aware {KV} Cache Reuse",
    author = "Ou, Jie  and
      Guo, Jinyu  and
      Guo, Shiyao  and
      Li, Yuang  and
      Wu, Ruiqi  and
      Wang, Zhaokun  and
      Li, Wenyi  and
      Tian, Wenhong",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1990/",
    pages = "42946--42960",
    ISBN = "979-8-89176-390-6",
    abstract = "Many-Shot In-Context Learning (ICL) has emerged as a promising paradigm, leveraging extensive examples to unlock the reasoning potential of Large Language Models (LLMs). However, existing methods typically rely on a predetermined, fixed number of shots. This static approach often fails to adapt to the varying difficulty of different queries, leading to either insufficient context or interference from noise. Furthermore, the prohibitive computational and memory costs of long contexts severely limit Many-Shot{'}s feasibility. To address the above limitations, we propose AdapShot, which dynamically optimizes shot counts and leverages KV cache reuse for efficient inference. Specifically, we design a probe-based evaluation mechanism that utilizes output entropy to determine the optimal number of shots. To bypass the redundant prefilling computation during both the probing and inference phases, we incorporate a semantics-aware KV cache reuse strategy. Within this reuse strategy, to address positional encoding incompatibilities, we introduce a decoupling and re-encoding method that enables the flexible reordering of cached key-value pairs. Extensive experiments demonstrate that AdapShot achieves an average performance gain of $\sim$10{\%} and a 4.64$\times$ speedup compared to state-of-the-art DBSA."
}