@inproceedings{gu-etal-2025-see,
    title = "See the World, Discover Knowledge: A {C}hinese Factuality Evaluation for Large Vision Language Models",
    author = "Gu, Jihao  and
      Wang, Yingyao  and
      Bu, Pi  and
      Wang, Chen  and
      Wang, Ziming  and
      Song, Tengtao  and
      Wei, Donglai  and
      Yuan, Jiale  and
      Zhao, Yingxiu  and
      He, Yancheng  and
      Li, Shilong  and
      Liu, Jiaheng  and
      Cao, Meng  and
      Song, Jun  and
      Tan, Yingshui  and
      Li, Xiang  and
      Su, Wenbo  and
      Zhu, Xiaoyong  and
      Zheng, Bo",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.844/",
    pages = "16422--16447",
    ISBN = "979-8-89176-256-5",
    abstract = "The evaluation of factual accuracy in large vision language models (LVLMs) has lagged behind their rapid development, making it challenging to fully reflect these models' knowledge capacity and reliability. In this paper, we introduce the first factuality-based visual question-answering benchmark in Chinese, named \textbf{ChineseSimpleVQA}, aimed at assessing the visual factuality of LVLMs across 8 major topics and 56 subtopics. The key features of this benchmark include a focus on the \textbf{Chinese} language, \textbf{diverse} knowledge types, a \textbf{multi-hop} question construction, \textbf{high-quality} data, \textbf{static} consistency, and \textbf{easy-to-evaluate} through short answers. Moreover, we contribute a rigorous data construction pipeline and decouple the visual factuality into two parts: seeing the world (i.e., object recognition) and discovering knowledge. This decoupling allows us to analyze the capability boundaries and execution mechanisms of LVLMs. Subsequently, we evaluate 34 advanced open-source and closed-source models, revealing critical performance gaps within this field."
}