Hong Shen


2025

pdf bib
REALM: A Dataset of Real-World LLM Use Cases
Jingwen Cheng | Kshitish Ghate | Wenyue Hua | William Yang Wang | Hong Shen | Fei Fang
Findings of the Association for Computational Linguistics: ACL 2025

Large Language Models (LLMs), such as the GPT series, have driven significant industrial applications, leading to economic and societal transformations. However, a comprehensive understanding of their real-world applications remains limited.To address this, we introduce **REALM**, a dataset of over 94,000 LLM use cases collected from Reddit and news articles. **REALM** captures two key dimensions: the diverse applications of LLMs and the demographics of their users. It categorizes LLM applications and explores how users’ occupations relate to the types of applications they use.By integrating real-world data, **REALM** offers insights into LLM adoption across different domains, providing a foundation for future research on their evolving societal roles. An interactive dashboard ([https://realm-e7682.web.app/](https://realm-e7682.web.app/)) is provided for easy exploration of the dataset.

2024

pdf bib
PATIENT-πœ“: Using Large Language Models to Simulate Patients for Training Mental Health Professionals
Ruiyi Wang | Stephanie Milani | Jamie C. Chiu | Jiayin Zhi | Shaun M. Eack | Travis Labrum | Samuel M Murphy | Nev Jones | Kate V Hardy | Hong Shen | Fei Fang | Zhiyu Chen
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing

Mental illness remains one of the most critical public health issues. Despite its importance, many mental health professionals highlight a disconnect between their training and actual real-world patient practice. To help bridge this gap, we propose PATIENT-πœ“, a novel patient simulation framework for cognitive behavior therapy (CBT) training. To build PATIENT-πœ“, we construct diverse patient cognitive models based on CBT principles and use large language models (LLMs) programmed with these cognitive models to act as a simulated therapy patient. We propose an interactive training scheme, PATIENT-πœ“-TRAINER, for mental health trainees to practice a key skill in CBT – formulating the cognitive model of the patient – through role-playing a therapy session with PATIENT-πœ“. To evaluate PATIENT-πœ“, we conducted a comprehensive user study of 13 mental health trainees and 20 experts. The results demonstrate that practice using PATIENT-πœ“-TRAINER enhances the perceived skill acquisition and confidence of the trainees beyond existing forms of training such as textbooks, videos, and role-play with non-patients. Based on the experts’ perceptions, PATIENT-πœ“ is perceived to be closer to real patient interactions than GPT-4, and PATIENT-πœ“-TRAINER holds strong promise to improve trainee competencies. Our code and data are released at https://github.com/ruiyiw/patient-psi.