@inproceedings{pan-etal-2026-psycheval,
    title = "{P}sych{E}val: A Multi-Session and Multi-Therapy Benchmark for High-Realism {AI} Psychological Counselor",
    author = "Pan, Qianjun  and
      Wang, Junyi  and
      Zhou, Jie  and
      Yang, Yutao  and
      Li, Junsong  and
      Xu, Kaiyin  and
      Zhou, Yougen  and
      Li, Yihan  and
      Zhao, JingYuan  and
      Chen, Qin  and
      Zhou, Ningning  and
      Chen, Kai  and
      He, Liang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1115/",
    pages = "22192--22212",
    ISBN = "979-8-89176-395-1",
    abstract = "To develop a reliable AI for psychological assessment, we introduce PsychEval, a multi-session, multi-therapy, and highly realistic benchmark designed to address three key challenges:**1) Can we train a highly realistic AI counselor?** Realistic counseling is a longitudinal task requiring sustained memory and dynamic goal tracking. We propose a multi-session benchmark (spanning 6-10 sessions across three distinct stages) that demands critical capabilities such as memory continuity, adaptive reasoning, and longitudinal planning. The dataset is annotated with extensive professional skills, comprising over 677 meta-skills and 4577 atomic skills. **2) How to train a multi-therapy AI counselor?** While existing models often focus on a single therapy, complex cases frequently require flexible strategies among various therapies. We construct a diverse dataset covering five therapeutic modalities alongside an integrative therapy with a unified three-stage clinical framework across six core psychological topics. **3) How to systematically evaluate an AI counselor?** We establish a holistic evaluation framework with 18 therapy-specific and therapy-shared metrics across Client-Level and Counselor-Level dimensions. To We also construct over 2,000 diverse client profiles. Extensive experimental analysis fully validates the superior quality and clinical fidelity of our dataset.Our datasets and evaluation framework are anonymously available at this repository."
}