@inproceedings{yang-etal-2026-finmragbench,
    title = "{F}in{MRAGB}ench: A Realistic and Complex Benchmark for Multi-Modal {RAG} in Financial Document Analysis",
    author = "Yang, Shouqing  and
      Zhang, Qi  and
      Yang, Yuhang  and
      Xu, Ruikang  and
      Hou, Yuwei  and
      Jia, Zhulin  and
      Gao, Lirong  and
      Wang, Haobo  and
      Chen, Jinglei  and
      Wang, Jiexiang  and
      Guo, Sheng  and
      Zheng, Bo  and
      Chen, Gang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.187/",
    pages = "3822--3867",
    ISBN = "979-8-89176-395-1",
    abstract = "Retrieval-augmented generation (RAG) has become a widely adopted paradigm for realistic financial analysis over financial documents. However, existing benchmarks fail to capture realistic financial analysis settings that involve cross-document retrieval, multi-page evidence integration, and diverse analytical tasks. To address this gap, we introduce $\textbf{FinMRAGBench}$, a comprehensive multi-modal financial RAG benchmark in which most questions require retrieving evidence scattered across multiple pages and documents, constructed from large-scale real-world annual reports and comprising 887 expert-verified QA pairs spanning five representative financial analysis tasks. Moreover, we introduce $\textbf{FinMRAGAgent}$, an agent trained on high-quality agentic trajectories following the reasoning-and-acting (ReAct) paradigm, capable of dynamic tool invocation and multi-step financial analysis. Our extensive experiments show that current multi-modal RAG systems still struggle with incomplete retrieval and complex financial reasoning. In contrast, FinMRAGAgent achieves the strongest overall performance across all models, demonstrating that our structured reasoning approach significantly enhances multi-modal RAG in realistic financial scenarios. The code and data are available at https://github.com/sqyangit/FinMRAGBench."
}