@inproceedings{wang-etal-2026-meur,
    title = "{MEUR}: A Benchmark for Evaluating Vision-Language Models on Multimodal Event Understanding and Reasoning",
    author = "Wang, Zimu  and
      Wang, Yuqi  and
      Chen, Tong  and
      Zeng, Changyu  and
      Na, Hongbin  and
      Han, Nijia  and
      Xing, Fuyu  and
      Chen, Qi  and
      Wang, Qiufeng  and
      Nguyen, Anh  and
      Wang, Shuihua  and
      Chen, Ling  and
      Su, Jionglong  and
      Zhang, Haiyang  and
      Wang, Wei",
    editor = "Piperidis, Stelios  and
      Bel, N{\'u}ria  and
      van den Heuvel, Henk  and
      Ide, Nancy  and
      Krek, Simon  and
      Toral, Antonio",
    booktitle = "Proceedings of the Fifteenth Language Resources and Evaluation Conference",
    month = may,
    year = "2026",
    address = "Palma de Mallorca, Spain",
    publisher = "ELRA Language Resource Association",
    url = "https://preview.aclanthology.org/ingest-lrec/2026.lrec-1.134/",
    doi = "10.63317/4ftadqtyt374",
    pages = "1696--1709",
    abstract = "Event understanding and reasoning play critical roles in thoroughly evaluating the capabilities of Vision-Language Models (VLMs); however, existing Visual Question Answering (VQA) datasets predominantly focus on entity-centric questions, while event- or action-related questions are limited in scale and suffer from significant shortcut issues. We introduce MEUR, the first Multimodal Event Understanding and Reasoning dataset consisting of 1,200 images and 4,217 questions, necessitating VLMs with a diverse range of multimodal understanding and reasoning capabilities to answer, ranging from basic event recognition to more complex tasks such as counting and comparison. To streamline the annotation process, we propose a novel semi-automated pipeline that combines advanced VLMs with human annotators, achieving high quality and efficiency. We conduct extensive experiments on state-of-the-art non-thinking and thinking VLMs to demonstrate their capabilities and limitations in multimodal event understanding and reasoning. Furthermore, we provide a detailed error analysis that points out promising directions for future research."
}