@inproceedings{chen-etal-2026-rpc,
    title = "{RPC}-Bench: A Fine-grained Benchmark for Research Paper Comprehension",
    author = "Chen, Yelin  and
      Zhang, Fanjin  and
      Sun, Suping  and
      Pang, Yunhe  and
      Wang, Yuanchun  and
      Song, Jian  and
      Li, XiaoYan  and
      Hou, Lei  and
      Zhao, Shu  and
      Tang, Jie  and
      Li, Juanzi",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1277/",
    pages = "27683--27717",
    ISBN = "979-8-89176-390-6",
    abstract = "Understanding research papers remains challenging for foundation models due to specialized scientific discourse and complex figures and tables, yet existing benchmarks offer limited fine-grained evaluation at scale. To address this gap, we introduce RPC-Bench, a large-scale question-answering benchmark built from review{--}rebuttal exchanges of high-quality computer science papers, containing 15K human-verified QA pairs. We design a fine-grained taxonomy aligned with the scientific research flow to assess models' ability to understand and answer why, what, and how questions in scholarly contexts. We also define an elaborate LLM{--}human interaction annotation framework to support large-scale labeling and quality control. Following the LLM-as-a-Judge paradigm, we develop a scalable framework that evaluates models on correctness-completeness and conciseness, with high agreement to human judgment. Experiments reveal that even the strongest models (GPT-5) achieve only 68.2{\%} correctness-completeness, dropping to 37.46{\%} after conciseness adjustment, highlighting substantial gaps in precise academic paper understanding."
}