@inproceedings{samuel-etal-2025-personagym,
    title = "{P}ersona{G}ym: Evaluating Persona Agents and {LLM}s",
    author = "Samuel, Vinay  and
      Zou, Henry Peng  and
      Zhou, Yue  and
      Chaudhari, Shreyas  and
      Kalyan, Ashwin  and
      Rajpurohit, Tanmay  and
      Deshpande, Ameet  and
      Narasimhan, Karthik R  and
      Murahari, Vishvak",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.368/",
    doi = "10.18653/v1/2025.findings-emnlp.368",
    pages = "6999--7022",
    ISBN = "979-8-89176-335-7",
    abstract = "Persona agents, which are LLM agents conditioned to act according to an assigned persona, enable contextually rich and user-aligned interactions across domains like education and healthcare.However, evaluating how faithfully these agents adhere to their personas remains a significant challenge, particularly in free-form settings that demand consistency across diverse, persona-relevant environments.We introduce PersonaGym, the first dynamic evaluation framework for persona agents, and PersonaScore, a human-aligned automatic metric grounded in decision theory that enables comprehensive large-scale evaluation. Our evaluation of 10 leading LLMs across 200 personas and 10,000 questions reveals significant advancement opportunities.For example, GPT-4.1 had the exact same PersonaScore as LLaMA-3-8b despite being a more recent and advanced closed-source model. Importantly, increased model size and complexity do not necessarily enhance persona agent capabilities, underscoring the need for algorithmic and architectural innovation toward faithful, performant persona agents."
}