Yihan Li
2026
PsychEval: A Multi-Session and Multi-Therapy Benchmark for High-Realism AI Psychological Counselor
Qianjun Pan | Junyi Wang | Jie Zhou | Yutao Yang | Junsong Li | Kaiyin Xu | Yougen Zhou | Yihan Li | JingYuan Zhao | Qin Chen | Ningning Zhou | Kai Chen | Liang He
Findings of the Association for Computational Linguistics: ACL 2026
Qianjun Pan | Junyi Wang | Jie Zhou | Yutao Yang | Junsong Li | Kaiyin Xu | Yougen Zhou | Yihan Li | JingYuan Zhao | Qin Chen | Ningning Zhou | Kai Chen | Liang He
Findings of the Association for Computational Linguistics: ACL 2026
To develop a reliable AI for psychological assessment, we introduce PsychEval, a multi-session, multi-therapy, and highly realistic benchmark designed to address three key challenges:**1) Can we train a highly realistic AI counselor?** Realistic counseling is a longitudinal task requiring sustained memory and dynamic goal tracking. We propose a multi-session benchmark (spanning 6-10 sessions across three distinct stages) that demands critical capabilities such as memory continuity, adaptive reasoning, and longitudinal planning. The dataset is annotated with extensive professional skills, comprising over 677 meta-skills and 4577 atomic skills. **2) How to train a multi-therapy AI counselor?** While existing models often focus on a single therapy, complex cases frequently require flexible strategies among various therapies. We construct a diverse dataset covering five therapeutic modalities alongside an integrative therapy with a unified three-stage clinical framework across six core psychological topics. **3) How to systematically evaluate an AI counselor?** We establish a holistic evaluation framework with 18 therapy-specific and therapy-shared metrics across Client-Level and Counselor-Level dimensions. To We also construct over 2,000 diverse client profiles. Extensive experimental analysis fully validates the superior quality and clinical fidelity of our dataset.Our datasets and evaluation framework are anonymously available at this repository.
2025
A Middle Path for On-Premises LLM Deployment: Preserving Privacy Without Sacrificing Model Confidentiality
Hanbo Huang | Yihan Li | Bowen Jiang | Bo Jiang | Lin Liu | Zhuotao Liu | Ruoyu Sun | Shiyu Liang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Hanbo Huang | Yihan Li | Bowen Jiang | Bo Jiang | Lin Liu | Zhuotao Liu | Ruoyu Sun | Shiyu Liang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Privacy-sensitive users require deploying large language models (LLMs) within their own infrastructure (on-premises) to safeguard private data and enable customization. However, vulnerabilities in local environments can lead to unauthorized access and potential model theft. To address this, prior research on small models has explored securing only the output layer within hardware-secured devices to balance model confidentiality and customization. Yet this approach fails to protect LLMs effectively. In this paper, we discover that (1) query-based distillation attacks targeting the secured top layer can produce a functionally equivalent replica of the victim model; (2) securing the same number of layers, bottom layers before a transition layer provide stronger protection against distillation attacks than top layers, with comparable effects on customization performance; and (3) the number of secured layers creates a trade-off between protection and customization flexibility. Based on these insights, we propose SOLID, a novel deployment framework that secures a few bottom layers in a secure environment and introduces an efficient metric to optimize the trade-off by determining the ideal number of hidden layers. Extensive experiments on five models (1.3B to 70B parameters) demonstrate that SOLID outperforms baselines, achieving a better balance between protection and downstream customization.