@inproceedings{hung-etal-2025-msr2,
    title = "{MSR}$^2$: A Benchmark for Multi-Source Retrieval and Reasoning in Visual Question Answering",
    author = "Hung, Kuo-Han  and
      Fang, Hung-Chieh  and
      Huang, Chao-Wei  and
      Chen, Yun-Nung",
    editor = "Shi, Weijia  and
      Yu, Wenhao  and
      Asai, Akari  and
      Jiang, Meng  and
      Durrett, Greg  and
      Hajishirzi, Hannaneh  and
      Zettlemoyer, Luke",
    booktitle = "Proceedings of the 4th International Workshop on Knowledge-Augmented Methods for Natural Language Processing",
    month = may,
    year = "2025",
    address = "Albuquerque, New Mexico, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/moar-dois/2025.knowledgenlp-1.24/",
    doi = "10.18653/v1/2025.knowledgenlp-1.24",
    pages = "259--271",
    ISBN = "979-8-89176-229-9",
    abstract = "This paper introduces MSR$^2$, a benchmark for multi-source retrieval and reasoning in visual question answering. Unlike previous knowledge-based visual question answering datasets, MSR$^2$ focuses on questions involving multiple fine-grained entities, providing a unique opportunity to assess a model{'}s spatial reasoning ability and its capacity to retrieve and aggregate information from various sources for different entities. Through comprehensive evaluation using MSR$^2$, we gain valuable insights into the capabilities and limitations of state-of-the-art large vision-language models (LVLMs).Our findings reveal that even state-of-the-art LVLMs struggle with questions requiring multi-entities and knowledge-intensive reasoning, highlighting important new directions for future research.Additionally, we demonstrate that enhanced visual entity recognition and knowledge retrieval can significantly improve performance on MSR$^2$, pinpointing key areas for advancement."
}