@inproceedings{he-etal-2025-data,
    title = "Data Poisoning for In-context Learning",
    author = "He, Pengfei  and
      Xu, Han  and
      Xing, Yue  and
      Liu, Hui  and
      Yamada, Makoto  and
      Tang, Jiliang",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2025",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/Ingest-2025-COMPUTEL/2025.findings-naacl.91/",
    pages = "1680--1700",
    ISBN = "979-8-89176-195-7",
    abstract = "In-context learning (ICL) has emerged as a capability of large language models (LLMs), enabling them to adapt to new tasks using provided examples. While ICL has demonstrated its strong effectiveness, there is limited understanding of its vulnerability against potential threats. This paper examines ICL`s vulnerability to data poisoning attacks. We introduce ICLPoison, an attacking method specially designed to exploit ICL`s unique learning mechanisms by identifying discrete text perturbations that influence LLM hidden states. We propose three representative attack strategies, evaluated across various models and tasks. Our experiments, including those on GPT-4, show that ICL performance can be significantly compromised by these attacks, highlighting the urgent need for improved defense mechanisms to protect LLMs' integrity and reliability."
}