@inproceedings{wang-etal-2025-agri,
    title = "Agri-{CM}$^3$: A {C}hinese Massive Multi-modal, Multi-level Benchmark for Agricultural Understanding and Reasoning",
    author = "Wang, Haotian  and
      Guan, Yi  and
      Meng, Fanshu  and
      Zhao, Chao  and
      Yan, Lian  and
      Yang, Yang  and
      Jiang, Jingchi",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.576/",
    pages = "11729--11754",
    ISBN = "979-8-89176-251-0",
    abstract = "Multi-modal Large Language Models (MLLMs) integrating images, text, and speech can provide farmers with accurate diagnoses and treatment of pests and diseases, enhancing agricultural efficiency and sustainability. However, existing benchmarks lack comprehensive evaluations, particularly in multi-level reasoning, making it challenging to identify model limitations. To address this issue, we introduce Agri-CM$^3$, an expert-validated benchmark assessing MLLMs' understanding and reasoning in agricultural management. It includes 3,939 images and 15,901 multi-level multiple-choice questions with detailed explanations. Evaluations of 45 MLLMs reveal significant gaps. Even GPT-4o achieves only 63.64{\%} accuracy, falling short in fine-grained reasoning tasks. Analysis across three reasoning levels and seven compositional abilities highlights key challenges in accuracy and cognitive understanding. Our study provides insights for advancing MLLMs in agricultural management, driving their development and application. Code and data are available at \url{https://github.com/HIT-Kwoo/Agri-CM3}."
}