@inproceedings{zhang-etal-2025-leank,
    title = "{L}ean{K}: Learnable K Cache Channel Pruning for Efficient Decoding",
    author = "Zhang, Yike  and
      He, Zhiyuan  and
      Jiang, Huiqiang  and
      Zhang, Chengruidong  and
      Yang, Yuqing  and
      Wang, Jianyong  and
      Qiu, Lili",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1584/",
    pages = "31110--31125",
    ISBN = "979-8-89176-332-6",
    abstract = "Large language models (LLMs) enable long-context tasks but face efficiency challenges due to the growing key-value (KV) cache. We propose LeanK, a learning-based method that prunes unimportant key (K) cache channels by leveraging static channel sparsity. LeanK reduces GPU memory and accelerates decoding without sacrificing accuracy. Experiments demonstrate up to 70{\%} K cache and 16{\%}{--}18{\%} V cache memory reduction, and 1.45{\texttimes} decoding speedup. We also provide insights into model channels and attention heads during long-context inference by analyzing the learned importance distribution. Our code is anonymously available at https://anonymous.4open.science/r/LeanK-7A87/README.md."
}