Zhongjun Yang
2026
AEGIS: A Holistic Benchmark for Evaluating Forensic Analysis of AI-Generated Academic Images
Bo Zhang | Tzu-Yen Ma | Zichen Tang | Junpeng Ding | Zirui Wang | Yizhuo Zhao | Peilin Gao | Zijie Xi | Zixin Ding | Haiyang Sun | Haocheng Gao | Yuan Liu | Liangjia Wang | Yiling Huang | Yujie Wang | Yuyue Zhang | Ronghui Xi | Yuanze Li | Jiacheng Liu | Zhongjun Yang | Haihong E
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Bo Zhang | Tzu-Yen Ma | Zichen Tang | Junpeng Ding | Zirui Wang | Yizhuo Zhao | Peilin Gao | Zijie Xi | Zixin Ding | Haiyang Sun | Haocheng Gao | Yuan Liu | Liangjia Wang | Yiling Huang | Yujie Wang | Yuyue Zhang | Ronghui Xi | Yuanze Li | Jiacheng Liu | Zhongjun Yang | Haihong E
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
We introduce AEGIS, A holistic benchmark for Evaluating forensic analysis of AI-Generated academic ImageS. Compared to existing benchmarks, AEGIS features three key advances: (1) Domain-Specific Complexity: covering seven academic categories with 39 fine-grained subtypes, exposing intrinsic forensic difficulty, where even GPT-5.1 reaches 48.80% overall performance and expert models achieve only limited localization accuracy (IoU 30.09%); (2) Diverse Forgery Simulations: modeling four prevalent academic forgery strategies across 25 generative models, with 11 yielding average forensic accuracy below 50%, showing that forensics lag behind generative advances; and (3) Multi-Dimensional Forensic Evaluation: jointly assessing detection, reasoning, and localization, revealing complementary strengths between model families, with multimodal large language models (MLLMs) at 84.74% accuracy in textual artifact recognition and expert detectors peaking at 79.54% accuracy in binary authenticity detection. By evaluating 25 leading MLLMs, nine expert models, and one unified multimodal understanding and generation model, AEGIS serves as a diagnostic testbed exposing fundamental limitations in academic image forensics.
Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning
Junpeng Ding | Zichen Tang | Haihong E | Mengyuan Ji | Yang Liu | Haolin Tian | Haiyang Sun | Pengqi Sun | Yang Xu | Yichen Liu | Haocheng Gao | Zijie Xi | Ruomeng Jiang | Peizhi Zhao | Rongjin Li | Yuanze Li | Jiacheng Liu | Zhongjun Yang | Jintong Chen | Siying Lin
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Junpeng Ding | Zichen Tang | Haihong E | Mengyuan Ji | Yang Liu | Haolin Tian | Haiyang Sun | Pengqi Sun | Yang Xu | Yichen Liu | Haocheng Gao | Zijie Xi | Ruomeng Jiang | Peizhi Zhao | Rongjin Li | Yuanze Li | Jiacheng Liu | Zhongjun Yang | Jintong Chen | Siying Lin
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images. SPUR features three key innovations: (1) Panel-Level Fine-Grained Perception: evaluating the visual perception of multimodal large language models (MLLMs) across three dimensions (numerical, morphological, and information localization) on six fine-grained panel types; (2) Cross-Panel Relation Understanding: utilizing complex images with an average of 14.3 panels per sample to evaluate MLLMs’ ability to decipher intricate cross-panel relations; (3) Expert-Level Reasoning: assessment of qualitative and quantitative reasoning across five experimental paradigms to determine if models can infer conclusions from evidence as human experts do. Comprehensive evaluation of 20 MLLMs and four multimodal Chain-of-Thought (MCoT) methods reveals that current models fall significantly short of the expert-level requirements for scientific image interpretation, underscoring a critical bottleneck in AI for Science (AI4S) research.
2025
FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging
Zichen Tang | Haihong E | Ziyan Ma | Haoyang He | Jiacheng Liu | Zhongjun Yang | Zihua Rong | Rongjin Li | Kun Ji | Qing Huang | Xinyang Hu | Yang Liu | Qianhe Zheng
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zichen Tang | Haihong E | Ziyan Ma | Haoyang He | Jiacheng Liu | Zhongjun Yang | Zihua Rong | Rongjin Li | Kun Ji | Qing Huang | Xinyang Hu | Yang Liu | Qianhe Zheng
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
We introduce **FinanceReasoning**, a novel benchmark designed to evaluate the reasoning capabilities of large reasoning models (LRMs) in financial numerical reasoning problems. Compared to existing benchmarks, our work provides three key advancements. (1) **Credibility**: We update 15.6% of the questions from four public datasets, annotating 908 new questions with detailed Python solutions and rigorously refining evaluation standards. This enables an accurate assessment of the reasoning improvements of LRMs. (2) **Comprehensiveness**: FinanceReasoning covers 67.8% of financial concepts and formulas, significantly surpassing existing datasets. Additionally, we construct 3,133 Python-formatted functions, which enhances LRMs’ financial reasoning capabilities through refined knowledge (*e.g.*, 83.2% → 91.6% for GPT-4o). (3) **Challenge**: Models are required to apply multiple financial formulas for precise numerical reasoning on 238 *Hard* problems. The best-performing model (*i.e.*, OpenAI o1 with PoT) achieves 89.1% accuracy, yet LRMs still face challenges in numerical precision. We demonstrate that combining Reasoner and Programmer models can effectively enhance LRMs’ performance (*e.g.*, 83.2% → 87.8% for DeepSeek-R1). Our work paves the way for future research on evaluating and improving LRMs in domain-specific complex reasoning tasks.
Search
Fix author
Co-authors
- Haihong E 3
- Zichen Tang 3
- Junpeng Ding 2
- Haocheng Gao 2
- Rongjin Li 2
- Yuanze Li 2
- Jiacheng Liu 2
- Haiyang Sun 2
- Zijie Xi 2
- Jintong Chen 1
- Zixin Ding 1
- Peilin Gao 1
- Haoyang He 1
- Xinyang Hu 1
- Qing Huang 1
- Yiling Huang 1
- Kun Ji 1
- Mengyuan Ji 1
- Ruomeng Jiang 1
- Siying Lin 1
- Jiacheng Liu 1
- Yang Liu 1
- Yang Liu 1
- Yichen Liu 1
- Yuan Liu 1
- Tzu-Yen Ma 1
- Ziyan Ma 1
- Zihua Rong 1
- Pengqi Sun 1
- Haolin Tian 1
- Liangjia Wang 1
- Yujie Wang 1
- Zirui Wang 1
- Ronghui Xi 1
- Yang Xu 1
- Bo Zhang 1
- Yuyue Zhang 1
- Peizhi Zhao 1
- Yizhuo Zhao 1
- Qianhe Zheng 1
Venues
- ACL3