@inproceedings{fu-etal-2026-anchoring,
    title = "Anchoring the Cache: Mitigating Contextual Hallucination in {KV}-Compressed Long-Context Summarization",
    author = "Fu, Yu  and
      Luo, Chen  and
      Valvoda, Josef  and
      Zhang, Xin  and
      Lei, Xuejing  and
      Pan, Xiao  and
      Liu, Hui  and
      Dong, Yue",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1542/",
    pages = "33397--33413",
    ISBN = "979-8-89176-390-6",
    abstract = "Key-Value (KV) cache compression techniques have improved the efficiency of long-context summarization in Large Language Models (LLMs), but their impact on model hallucination remains underexplored. In this paper, we present the first systematic study of how KV cache compression affects hallucination in long-context summarization, demonstrating that aggressive compression can increase hallucination scores by up to 3.36{\texttimes} compared to the baseline. To mitigate this issue, we propose HalluKV, a decoding-phase strategy that selectively removes generated KV pairs from retrieval heads responsible for retrieving critical information from source context, thereby anchoring their attention on the preserved source information. Our approach maintains computational efficiency while significantly reducing hallucination across multiple models and datasets, achieving up to 5.48 average point reductions on Llama-3-8B-Instruct, enabling more trustworthy long-context summarization."
}