Yutao Yang
2026
PsychEval: A Multi-Session and Multi-Therapy Benchmark for High-Realism AI Psychological Counselor
Qianjun Pan | Junyi Wang | Jie Zhou | Yutao Yang | Junsong Li | Kaiyin Xu | Yougen Zhou | Yihan Li | JingYuan Zhao | Qin Chen | Ningning Zhou | Kai Chen | Liang He
Findings of the Association for Computational Linguistics: ACL 2026
Qianjun Pan | Junyi Wang | Jie Zhou | Yutao Yang | Junsong Li | Kaiyin Xu | Yougen Zhou | Yihan Li | JingYuan Zhao | Qin Chen | Ningning Zhou | Kai Chen | Liang He
Findings of the Association for Computational Linguistics: ACL 2026
To develop a reliable AI for psychological assessment, we introduce PsychEval, a multi-session, multi-therapy, and highly realistic benchmark designed to address three key challenges:**1) Can we train a highly realistic AI counselor?** Realistic counseling is a longitudinal task requiring sustained memory and dynamic goal tracking. We propose a multi-session benchmark (spanning 6-10 sessions across three distinct stages) that demands critical capabilities such as memory continuity, adaptive reasoning, and longitudinal planning. The dataset is annotated with extensive professional skills, comprising over 677 meta-skills and 4577 atomic skills. **2) How to train a multi-therapy AI counselor?** While existing models often focus on a single therapy, complex cases frequently require flexible strategies among various therapies. We construct a diverse dataset covering five therapeutic modalities alongside an integrative therapy with a unified three-stage clinical framework across six core psychological topics. **3) How to systematically evaluate an AI counselor?** We establish a holistic evaluation framework with 18 therapy-specific and therapy-shared metrics across Client-Level and Counselor-Level dimensions. To We also construct over 2,000 diverse client profiles. Extensive experimental analysis fully validates the superior quality and clinical fidelity of our dataset.Our datasets and evaluation framework are anonymously available at this repository.
OSCR-Attack: One-Shot Character Level Attacks through Self-Optimizing Continuous Relaxation
Lingyi Kong | Zhuo Liu | Zhanghao Hu | Qilong Qiu | Yutao Yang | Jingjing Xue | Zheng Wang | Lin Gui | Feiping Nie
Findings of the Association for Computational Linguistics: ACL 2026
Lingyi Kong | Zhuo Liu | Zhanghao Hu | Qilong Qiu | Yutao Yang | Jingjing Xue | Zheng Wang | Lin Gui | Feiping Nie
Findings of the Association for Computational Linguistics: ACL 2026
Adversarial attacks have attracted growing attention across domains, including natural language processing (NLP). Character-level adversarial attacks preserve semantics, but they have received less attention because the discrete operations they use are costly and inefficient. Challenging these beliefs, we introduce two adaptively learnable matrices that transform discrete choices into continuous representations, enabling automatic one-shot multi-position, multi-character insertion. To optimize the two learnable matrices, we propose OSCR-Attack, an end-to-end framework based on gradient-based optimization, with a conflict resolution strategy that maps the optimized continuous distributions back into discrete insertion operations. Extensive experiments on three benchmarks with three open-source large language models (LLMs) show that OSCR-Attack improves attack success rate (ASR) by up to 21.45% points and accelerates the attack by up to 3.66 times compared to recent baselines.