@inproceedings{ding-etal-2026-decoding,
    title = "Decoding Scientific Experimental Images: The {SPUR} Benchmark for Perception, Understanding, and Reasoning",
    author = "Ding, Junpeng  and
      Tang, Zichen  and
      E, Haihong  and
      Ji, Mengyuan  and
      Liu, Yang  and
      Tian, Haolin  and
      Sun, Haiyang  and
      Sun, Pengqi  and
      Xu, Yang  and
      Liu, Yichen  and
      Gao, Haocheng  and
      Xi, Zijie  and
      Jiang, Ruomeng  and
      Zhao, Peizhi  and
      Li, Rongjin  and
      Li, Yuanze  and
      Liu, Jiacheng  and
      Yang, Zhongjun  and
      Chen, Jintong  and
      Lin, Siying",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.585/",
    pages = "12859--12882",
    ISBN = "979-8-89176-390-6",
    abstract = "We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images. SPUR features three key innovations: (1) Panel-Level Fine-Grained Perception: evaluating the visual perception of multimodal large language models (MLLMs) across three dimensions (numerical, morphological, and information localization) on six fine-grained panel types; (2) Cross-Panel Relation Understanding: utilizing complex images with an average of 14.3 panels per sample to evaluate MLLMs' ability to decipher intricate cross-panel relations; (3) Expert-Level Reasoning: assessment of qualitative and quantitative reasoning across five experimental paradigms to determine if models can infer conclusions from evidence as human experts do. Comprehensive evaluation of 20 MLLMs and four multimodal Chain-of-Thought (MCoT) methods reveals that current models fall significantly short of the expert-level requirements for scientific image interpretation, underscoring a critical bottleneck in AI for Science (AI4S) research."
}