@inproceedings{ji-etal-2026-see,
    title = "See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video {LLM}s",
    author = "Ji, Yicheng  and
      Zhang, Jun  and
      Chen, Jinpeng  and
      Wang, Cong  and
      Shou, Lidan  and
      Chen, Gang  and
      Li, Huan",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1087/",
    pages = "23707--23726",
    ISBN = "979-8-89176-390-6",
    abstract = "Video Large Language Models (Video-LLMs) excel in video understanding but suffer from high inference latency due to autoregressive generation. Speculative Decoding (SD) mitigates this by applying a draft-and-verify paradigm, yet existing methods are constrained by rigid exact-match rules, severely limiting the acceleration potential. To bridge this gap, we propose LVSpec, the first training-free loosely SD framework tailored for Video-LLMs. Grounded in the insight that generation is governed by sparse visual-relevant anchors (mandating strictness) amidst abundant visual-irrelevant fillers (permitting loose verification), LVSpec employs a lightweight visual-relevant token identification scheme to accurately pinpoint the former. To further maximize acceptance, we augment this with a position-shift tolerant mechanism that effectively salvages positionally mismatched but semantically equivalent tokens. Experiments demonstrate that LVSpec is high-fidelity and rapid: it preserves $>99.8${\%} of target performance while accelerating Qwen2.5-VL-32B by $2.70 \times$ and LLaVA-OneVision-72B by $2.94 \times$. Notably, it boosts the mean accepted length and speedup ratio by 136{\%} and 35{\%} compared to SOTA training-free SD methods for Video-LLMs. Code is provided in the submitted software."
}