@inproceedings{pan-etal-2026-mt,
    title = "{MT}-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal {LLM}s in Multi-Turn Dialogues",
    author = "Pan, Yaning  and
      Xie, Qianqian  and
      Zhang, Guohui  and
      Wang, Zekun Moore  and
      Wen, Yongqian  and
      Zhang, Yuanxing  and
      Hu, Haoxuan  and
      Pan, Zhiyu  and
      Huang, Yibing  and
      Gan, Zhidong  and
      Lin, Yonghong  and
      Ping, An  and
      Li, Shihao  and
      Wang, Yanghai  and
      Peng, Tianhao  and
      Liu, Jiaheng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.397/",
    pages = "8105--8126",
    ISBN = "979-8-89176-395-1",
    abstract = "The recent development of Multimodal Large Language Models (MLLMs) has significantly advanced AI{'}s ability to understand visual modalities. However, existing evaluation benchmarks remain limited to single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios. To bridge this gap, we introduce $\textbf{MT-Video-Bench}$, a holistic video understanding benchmark for evaluating MLLMs in multi-turn dialogues. Specifically, our MT-Video-Bench mainly assesses six core competencies that focus on perceptivity and interactivity, encompassing 1,000 meticulously curated multi-turn dialogues from diverse domains. These capabilities are rigorously aligned with real-world applications, such as interactive sports analysis and multi-turn video-based intelligent tutoring. With MT-Video-Bench, we extensively evaluate various state-of-the-art open-source and closed-source MLLMs, revealing their significant performance discrepancies and limitations in handling multi-turn video dialogues. The benchmark will be publicly available to foster future research."
}