@inproceedings{wang-etal-2026-howtonarrate,
    title = "{H}ow{T}o{N}arrate: A General-Domain Benchmark for Synchronized Video Narration with External Knowledge",
    author = "Wang, Xueyan  and
      Yang, Dingyi  and
      Jin, Qin",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/revision-workflow/2026.acl-long.1815/",
    doi = "10.18653/v1/2026.acl-long.1815",
    pages = "39110--39131",
    ISBN = "979-8-89176-390-6",
    abstract = "We present \textit{\textbf{HowToNarrate}}, the first general-domain benchmark for Synchronized Video Narration. The benchmark contains 3.2K videos across seven domains, segmented into 37.5K clips with aligned narrations and associated external knowledge. Effective narration requires models to \textit{understand visual scenes}, incorporate \textit{relevant knowledge}, and produce \textit{coherent, length-appropriate} descriptions. We systematically benchmark current Multimodal LLMs (MLLMs) on these abilities. Our analysis shows that existing MLLMs overemphasize knowledge retrieval while largely neglecting prior context (receiving less than 10{\%} attention). Moreover, they often conflate narration context with external knowledge, leading to redundancy and incoherence. To mitigate these issues, we propose VideoNarrationAgent, a multi-agent framework that combines context compression, knowledge retrieval, and narration generation. Experiments demonstrate that our method significantly improves MLLM performance. Furthermore, instruction tuning on HowToNarrate enhances both context-awareness and length control, boosting Qwen2.5-VL{'}s score from 25 to 84. We will release all data and code to support future research in synchronized video narration."
}