@inproceedings{huang-etal-2026-mask,
    title = "Mask Tokens as Prophet: Fine-Grained Cache Eviction for Efficient d{LLM} Inference",
    author = "Huang, Jianuo  and
      Zhang, Yaojie  and
      Yang, Yicun  and
      Huang, Benhao  and
      Zhang, Linfeng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.170/",
    pages = "3456--3479",
    ISBN = "979-8-89176-395-1",
    abstract = "Diffusion large language models (dLLMs) present a promising alternative to dominant autoregressive models (ARMs) by the ability of parallel decoding at the expense of substantial computation and memory costs. Specifically, the cache mechanism for bidirectional attention in dLLMs demands large memory footprint, restricting their ability to handle long contexts under resource-limited settings. Existing cache eviction strategies are primarily designed for ARMs and fail to account for the role of mask tokens and specific characteristics in dLLMs, resulting in suboptimal performance.To address these challenges, we introduce \textit{MaskKV}, a training-free cache eviction framework tailored to dLLMs, focusing on the effect of mask tokens in dLLMs. \textit{MaskKV} is built on two key innovations: (1) a mask-query guided scoring mechanism that leverages attention weights to identify and evict less critical prompt tokens for each head; (2) an adaptive cache budgeting strategy that improves efficiency by reducing allocation in intermediate layers and concentrating resources on prompt-preferring heads. On LLaDA with \textit{MaskKV}, compressing the KV cache to only 256 pairs (less than 5{\%} of tokens) retains 94{\%} of the full-cache performance on LongBench and achieves up to 31 $\times$ acceleration at 32k prompt length. \textit{Our code will be released on Github.}"
}