@inproceedings{qiao-etal-2025-math,
    title = "We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?",
    author = "Qiao, Runqi  and
      Tan, Qiuna  and
      Dong, Guanting  and
      MinhuiWu, MinhuiWu  and
      Sun, Chong  and
      Song, Xiaoshuai  and
      Wang, Jiapeng  and
      GongQue, Zhuoma  and
      Lei, Shanglin  and
      Zhang, YiFan  and
      Wei, Zhe  and
      Zhang, Miaoxuan  and
      Qiao, Runfeng  and
      Zong, Xiao  and
      Xu, Yida  and
      Yang, Peiqing  and
      Bao, Zhimin  and
      Diao, Muxi  and
      Li, Chen  and
      Zhang, Honggang",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.983/",
    pages = "20023--20070",
    ISBN = "979-8-89176-251-0",
    abstract = "Visual mathematical reasoning, as a fundamental visual reasoning ability, has received widespread attention from the Large Multimodal Models (LMMs) community. Existing benchmarks mainly focus more on the end-to-end performance, but neglect the underlying principles of knowledge acquisition and generalization. Instead, we introduce WE-MATH, the first benchmark specifically designed to explore the problem-solving principles. We meticulously collect 6.5K visual math problems and decompose them into 10.9K step-level questions for evaluation, spanning 5 layers of knowledge granularity and 67 hierarchical knowledge concepts. Specifically, we decompose composite problems into sub-problems according to the required knowledge concepts and introduce a novel four-dimensional metric to hierarchically assess inherent issues in LMMs' reasoning process. With WE-MATH, we conduct a thorough evaluation of existing LMMs in visual mathematical reasoning and provide comprehensive analysis and insight for future development. We anticipate that WE-MATH will open new pathways for advancements in visual mathematical reasoning for LMMs. Data and code are available at https://github.com/We-Math/We-Math."
}