@inproceedings{zhang-etal-2025-selecting,
    title = "Selecting Demonstrations for Many-Shot In-Context Learning via Gradient Matching",
    author = "Zhang, Jianfei  and
      Li, Bei  and
      Bai, Jun  and
      Li, Rumei  and
      Wang, Yanmeng  and
      Lin, Chenghua  and
      Rong, Wenge",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/display_plenaries/2025.findings-acl.608/",
    pages = "11686--11704",
    ISBN = "979-8-89176-256-5",
    abstract = "In-Context Learning (ICL) empowers Large Language Models (LLMs) for rapid task adaptation without Fine-Tuning (FT), but its reliance on demonstration selection remains a critical challenge. While many-shot ICL shows promising performance through scaled demonstrations, the selection method for many-shot demonstrations remains limited to random selection in existing work. Since the conventional instance-level retrieval is not suitable for many-shot scenarios, we hypothesize that the data requirements for in-context learning and fine-tuning are analogous. To this end, we introduce a novel gradient matching approach that selects demonstrations by aligning fine-tuning gradients between the entire training set of the target task and the selected examples, so as to approach the learning effect on the entire training set within the selected examples. Through gradient matching on relatively small models, e.g., Qwen2.5-3B or Llama3-8B, our method consistently outperforms random selection on larger LLMs from 4-shot to 128-shot scenarios across 9 diverse datasets. For instance, it surpasses random selection by 4{\%} on Qwen2.5-72B and Llama3-70B, and by around 2{\%} on 5 closed-source LLMs. This work unlocks more reliable and effective many-shot ICL, paving the way for its broader application."
}