@inproceedings{afonin-etal-2026-emergent,
    title = "Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned {LLM}s",
    author = "Afonin, Nikita  and
      Andriianov, Nikita  and
      Hovhannisyan, Vahagn  and
      Bageshpura, Nikhil  and
      Liu, Kyle  and
      Zhu, Kevin  and
      Dev, Sunishchal  and
      Panda, Ashwinee  and
      Rogov, Oleg  and
      Tutubalina, Elena  and
      Panchenko, Alexander  and
      Seleznyov, Mikhail",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1770/",
    pages = "38197--38212",
    ISBN = "979-8-89176-390-6",
    abstract = "Recent work has shown that narrow finetuning can produce broadly misaligned LLMs, a phenomenon termed emergent misalignment (EM). While concerning, these findings were limited to finetuning and activation steering, leaving out in-context learning (ICL). We therefore ask: does EM emerge in ICL? We find that it does: across four model families (Gemini, Kimi-K2, Grok, and Qwen), narrow in-context examples cause models to produce misaligned responses to benign, unrelated queries. With 16 in-context examples, EM rates range from 1{\%} to 24{\%} depending on model and domain, appearing with as few as 2 examples. Neither larger model scale nor explicit reasoning provides reliable protection, and larger models are typically even more susceptible. Next, we formulate and test a hypothesis, which explains in-context EM as conflict between safety objectives and context-following behavior. Consistent with this, instructing models to prioritize safety reduces EM while prioritizing context-following increases it. These findings establish ICL as a previously underappreciated vector for emergent misalignment that resists simple scaling-based solutions."
}