@inproceedings{zeng-etal-2025-reviewrl,
    title = "{R}eview{RL}: Towards Automated Scientific Review with {RL}",
    author = "Zeng, Sihang  and
      Tian, Kai  and
      Zhang, Kaiyan  and
      Wang, Yuru  and
      Gao, Junqi  and
      Liu, Runze  and
      Yang, Sa  and
      Li, Jingxuan  and
      Long, Xinwei  and
      Ma, Jiaheng  and
      Qi, Biqing  and
      Zhou, Bowen",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.857/",
    pages = "16942--16954",
    ISBN = "979-8-89176-332-6",
    abstract = "Peer review is essential for scientific progress but faces growing challenges due to increasing submission volumes and reviewer fatigue. Existing automated review approaches struggle with factual accuracy, rating consistency, and analytical depth, often generating superficial or generic feedback lacking the insights characteristic of high-quality human reviews. We introduce ReviewRL, a reinforcement learning framework for generating comprehensive and factually grounded scientific paper reviews. Our approach combines: (1) an ArXiv-MCP retrieval-augmented context generation pipeline that incorporates relevant scientific literature, (2) supervised fine-tuning that establishes foundational reviewing capabilities, and (3) a reinforcement learning procedure with a composite reward function that jointly enhances review quality and rating accuracy. Experiments on ICLR 2025 papers demonstrate that ReviewRL significantly outperforms existing methods across both rule-based metrics and model-based quality assessments. ReviewRL establishes a foundational framework for RL-driven automatic critique generation in scientific discovery, demonstrating promising potential for future development in this domain. The implementation of ReviewRL will be released at GitHub."
}