Rongjin Li
2026
Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning
Junpeng Ding | Zichen Tang | Haihong E | Mengyuan Ji | Yang Liu | Haolin Tian | Haiyang Sun | Pengqi Sun | Yang Xu | Yichen Liu | Haocheng Gao | Zijie Xi | Ruomeng Jiang | Peizhi Zhao | Rongjin Li | Yuanze Li | Jiacheng Liu | Zhongjun Yang | Jintong Chen | Siying Lin
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Junpeng Ding | Zichen Tang | Haihong E | Mengyuan Ji | Yang Liu | Haolin Tian | Haiyang Sun | Pengqi Sun | Yang Xu | Yichen Liu | Haocheng Gao | Zijie Xi | Ruomeng Jiang | Peizhi Zhao | Rongjin Li | Yuanze Li | Jiacheng Liu | Zhongjun Yang | Jintong Chen | Siying Lin
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images. SPUR features three key innovations: (1) Panel-Level Fine-Grained Perception: evaluating the visual perception of multimodal large language models (MLLMs) across three dimensions (numerical, morphological, and information localization) on six fine-grained panel types; (2) Cross-Panel Relation Understanding: utilizing complex images with an average of 14.3 panels per sample to evaluate MLLMs’ ability to decipher intricate cross-panel relations; (3) Expert-Level Reasoning: assessment of qualitative and quantitative reasoning across five experimental paradigms to determine if models can infer conclusions from evidence as human experts do. Comprehensive evaluation of 20 MLLMs and four multimodal Chain-of-Thought (MCoT) methods reveals that current models fall significantly short of the expert-level requirements for scientific image interpretation, underscoring a critical bottleneck in AI for Science (AI4S) research.
2025
FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging
Zichen Tang | Haihong E | Ziyan Ma | Haoyang He | Jiacheng Liu | Zhongjun Yang | Zihua Rong | Rongjin Li | Kun Ji | Qing Huang | Xinyang Hu | Yang Liu | Qianhe Zheng
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zichen Tang | Haihong E | Ziyan Ma | Haoyang He | Jiacheng Liu | Zhongjun Yang | Zihua Rong | Rongjin Li | Kun Ji | Qing Huang | Xinyang Hu | Yang Liu | Qianhe Zheng
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
We introduce **FinanceReasoning**, a novel benchmark designed to evaluate the reasoning capabilities of large reasoning models (LRMs) in financial numerical reasoning problems. Compared to existing benchmarks, our work provides three key advancements. (1) **Credibility**: We update 15.6% of the questions from four public datasets, annotating 908 new questions with detailed Python solutions and rigorously refining evaluation standards. This enables an accurate assessment of the reasoning improvements of LRMs. (2) **Comprehensiveness**: FinanceReasoning covers 67.8% of financial concepts and formulas, significantly surpassing existing datasets. Additionally, we construct 3,133 Python-formatted functions, which enhances LRMs’ financial reasoning capabilities through refined knowledge (*e.g.*, 83.2% → 91.6% for GPT-4o). (3) **Challenge**: Models are required to apply multiple financial formulas for precise numerical reasoning on 238 *Hard* problems. The best-performing model (*i.e.*, OpenAI o1 with PoT) achieves 89.1% accuracy, yet LRMs still face challenges in numerical precision. We demonstrate that combining Reasoner and Programmer models can effectively enhance LRMs’ performance (*e.g.*, 83.2% → 87.8% for DeepSeek-R1). Our work paves the way for future research on evaluating and improving LRMs in domain-specific complex reasoning tasks.