@inproceedings{hossain-etal-2025-llms,
    title = "{LLM}s as Meta-Reviewers' Assistants: A Case Study",
    author = "Hossain, Eftekhar  and
      Sinha, Sanjeev Kumar  and
      Bansal, Naman  and
      Knipper, R. Alexander  and
      Sarkar, Souvika  and
      Salvador, John  and
      Mahajan, Yash  and
      Guttikonda, Sri Ram Pavan Kumar  and
      Akter, Mousumi  and
      Hassan, Md. Mahadi  and
      Freestone, Matthew  and
      Jr., Matthew C. Williams  and
      Feng, Dongji  and
      Karmaker, Santu",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.naacl-long.395/",
    pages = "7763--7803",
    ISBN = "979-8-89176-189-6",
    abstract = "One of the most important yet onerous tasks in the academic peer-reviewing process is composing meta-reviews, which involves assimilating diverse opinions from multiple expert peers, formulating one{'}s self-judgment as a senior expert, and then summarizing all these perspectives into a concise holistic overview to make an overall recommendation. This process is time-consuming and can be compromised by human factors like fatigue, inconsistency, missing tiny details, etc. Given the latest major developments in Large Language Models (LLMs), it is very compelling to rigorously study whether LLMs can help meta-reviewers perform this important task better. In this paper, we perform a case study with three popular LLMs, i.e., GPT-3.5, LLaMA2, and PaLM2, to assist meta-reviewers in better comprehending multiple experts' perspectives by generating a controlled multi-perspective-summary (MPS) of their opinions. To achieve this, we prompt three LLMs with different types/levels of prompts based on the recently proposed TELeR taxonomy. Finally, we perform a detailed qualitative study of the MPSs generated by the LLMs and report our findings."
}