Jinqiang Li
2025
Reasoning is All You Need for Video Generalization: A Counterfactual Benchmark with Sub-question Evaluation
Qiji Zhou
|
YiFan Gong
|
Guangsheng Bao
|
Hongjie Qiu
|
Jinqiang Li
|
Xiangrong Zhu
|
Huajian Zhang
|
Yue Zhang
Findings of the Association for Computational Linguistics: ACL 2025
Counterfactual reasoning is crucial for robust video understanding but remains underexplored in existing multimodal benchmarks. In this paper, we introduce **COVER** (**CO**unterfactual **V**id**E**o **R**easoning), a multidimensional multimodal benchmark that systematically evaluates MLLMs across the abstract-concrete and perception-cognition dimensions. Beyond prior multimodal benchmarks, COVER decomposes complex queries into structured sub-questions, enabling fine-grained reasoning analysis. Experiments on commercial and open-source models reveal a strong correlation between sub-question accuracy and counterfactual reasoning performance, highlighting the role of structured inference in video understanding. Furthermore, our results suggest a key insight: enhancing the reasoning capability of models is essential for improving the robustness of video understanding. COVER establishes a new standard for assessing MLLMs’ logical reasoning abilities in dynamic environments. Our work is available at https://github.com/gongyifan-hash/COVER-Benchmark.
Search
Fix author
Co-authors
- Guangsheng Bao 1
- Yifan Gong 1
- Hongjie Qiu 1
- Huajian Zhang 1
- Yue Zhang (张岳, 章岳) 1
- show all...