Position Paper: How Should We Responsibly Adopt LLMs in the Peer Review Process?

Juhwan Choi, JungMin Yun, Changhun Kim, YoungBin Kim


Abstract
This position paper presents a novel perspective on the utilization of Large Language Models (LLMs) in the artificial intelligence paper review process. We first critique the current tendency for LLMs to be primarily used for simple review text generation, arguing instead that this approach overlooks more meaningful applications of LLMs that preserve human expertise at the core of evaluation. Instead, we advocate for leveraging LLMs to support key aspects of the review process—specifically, verifying the reproducibility of experimental results, checking the correctness and relevance of citations, and assisting with ethics review flagging. For example, integrating tools based on LLM Agents for code generation from research papers has recently enabled automated assessment of the reproducibility of the paper, thereby improving the transparency and reliability of research. By reorienting LLM usage toward these targeted and assistive roles, we outline a pathway for more effective and responsible integration of LLMs into peer review, ultimately supporting both reviewer efficiency and the integrity of the scientific process.
Anthology ID:
2026.findings-eacl.9
Volume:
Findings of the Association for Computational Linguistics: EACL 2026
Month:
March
Year:
2026
Address:
Rabat, Morocco
Editors:
Vera Demberg, Kentaro Inui, Lluís Marquez
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
151–165
Language:
URL:
https://preview.aclanthology.org/ingest-eacl/2026.findings-eacl.9/
DOI:
Bibkey:
Cite (ACL):
Juhwan Choi, JungMin Yun, Changhun Kim, and YoungBin Kim. 2026. Position Paper: How Should We Responsibly Adopt LLMs in the Peer Review Process?. In Findings of the Association for Computational Linguistics: EACL 2026, pages 151–165, Rabat, Morocco. Association for Computational Linguistics.
Cite (Informal):
Position Paper: How Should We Responsibly Adopt LLMs in the Peer Review Process? (Choi et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-eacl/2026.findings-eacl.9.pdf
Checklist:
 2026.findings-eacl.9.checklist.pdf