@inproceedings{jang-etal-2025-llm,
    title = "{LLM} Agents at the Roundtable: A Multi-Perspective and Dialectical Reasoning Framework for Essay Scoring",
    author = "Jang, Jinhee  and
      Moon, Ayoung  and
      Jung, Minkyoung  and
      Kim, YoungBin  and
      Lee, Seung Jin",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/name-variant-enfa-fane/2025.findings-emnlp.1072/",
    doi = "10.18653/v1/2025.findings-emnlp.1072",
    pages = "19674--19687",
    ISBN = "979-8-89176-335-7",
    abstract = "The emergence of large language models (LLMs) has brought a new paradigm to automated essay scoring (AES), a long-standing and practical application of natural language processing in education. However, achieving human-level multi-perspective understanding and judgment remains a challenge. In this work, we propose Roundtable Essay Scoring (RES), a multi-agent evaluation framework designed to perform precise and human-aligned scoring under a zero-shot setting. RES constructs evaluator agents based on LLMs, each tailored to a specific prompt and topic context. Each agent independently generates a trait-based rubric and conducts a multi-perspective evaluation. Then, by simulating a roundtable-style discussion, RES consolidates individual evaluations through a dialectical reasoning process to produce a final holistic score that more closely aligns with human evaluation. By enabling collaboration and consensus among agents with diverse evaluation perspectives, RES outperforms prior zero-shot AES approaches. Experiments on the ASAP dataset using ChatGPT and Claude show that RES achieves up to a 34.86{\%} improvement in average QWK over straightforward prompting (Vanilla) methods."
}