@inproceedings{li-etal-2025-graphkv,
    title = "{G}raph{KV}: Breaking the Static Selection Paradigm with Graph-Based {KV} Cache Eviction",
    author = "Li, Xuelin  and
      Jin, Xiangqi  and
      Zhang, Linfeng",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/name-variant-enfa-fane/2025.emnlp-main.1112/",
    doi = "10.18653/v1/2025.emnlp-main.1112",
    pages = "21910--21920",
    ISBN = "979-8-89176-332-6",
    abstract = "Efficient Key-Value (KV) cache management is essential for processing long text sequences in large language models (LLMs), where memory constraints often limit performance. Conventional KV eviction strategies, such as top-k selection based on attention scores, depend on static heuristics that fail to capture the evolving implicit dependencies among tokens during inference. To overcome this, we propose GraphKV, a graph-based framework that redefines token selection for KV cache compression. In GraphKV, $\textbf{tokens}$ are modeled as $\textbf{nodes}$ with importance scores, and $\textbf{edges}$ represent their $\textbf{similarity relationships}$. Through a decay-signal-propagation mechanism, token importance is dynamically updated by propagating information across the graph, enabling adaptive retention of the most contextually significant tokens. GraphKV can be seamlessly utilized in existing KV cache eviction methods such as SnapKV and PyramidKV in a plug-and-play manner. Codes are available in the supplementary materials and will be released on Github."
}