@inproceedings{zhang-etal-2026-hermes,
    title = "{HERMES}: {KV} Cache as Hierarchical Memory for Efficient Streaming Video Understanding",
    author = "Zhang, Haowei  and
      Yang, Shudong  and
      Fu, Jinlan  and
      Ng, See-Kiong  and
      Qiu, Xipeng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.381/",
    pages = "8411--8430",
    ISBN = "979-8-89176-390-6",
    abstract = "Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated significant improvement in offline video understanding. However, extending these capabilities to streaming video inputs, remains challenging, as existing models struggle to simultaneously maintain stable understanding performance, real-time responses, and low GPU memory overhead. To address this challenge, we propose HERMES, a novel training-free architecture for real-time and accurate understanding of video streams. Based on a mechanistic attention investigation, we conceptualize KV cache as a hierarchical memory framework that encapsulates video information across multiple granularities. During inference, HERMES reuses a compact KV cache, enabling efficient streaming understanding under resource constraints. Notably, HERMES requires no auxiliary computations upon the arrival of user queries, thereby guaranteeing real-time responses for continuous video stream interactions. HERMES achieves 10$\times$ faster TTFT compared to prior SOTA. Even when reducing video tokens by up to 68{\%} compared with uniform sampling, HERMES achieves superior or comparable accuracy across all benchmarks, with up to 11.4{\%} gains on streaming datasets."
}