Simulating Diverse Patient Populations Using Patient Vignettes and Large Language Models

Daniel Reichenpfader, Kerstin Denecke

Abstract
Ensuring equitable access to digital therapeutics (DTx) is essential to avoid healthcare inequalities in an era of increasing digitization. This requires DTx to be tested with users from diverse populations, which is often not realistic due to time and resource constraints. In this paper, we propose the use of large language models (LLMs) to simulate diverse patients. Specifically, we manually create a patient vignette that characterizes a specific population group. Variations of this vignette are used for role-prompting a commercial LLM, GPT-4, instructing the LLM to take on the role described in the patient vignette and act accordingly. We investigate if the LLM stays in its given role. To do this, we simulate a medical anamnesis interview with the role-prompted LLM and analyze its responses for compliance, coherence, correctness, containment, and clarification. Our results show that GPT-4 generates compliant, coherent and clinically valid responses, including information that is not explicitly stated in the provided patient vignette.
Anthology ID:
2024.cl4health-1.3
Volume:
Proceedings of the First Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC-COLING 2024
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Dina Demner-Fushman, Sophia Ananiadou, Paul Thompson, Brian Ondov
Venues:
CL4Health | WS
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
20–25
Language:
URL:
https://aclanthology.org/2024.cl4health-1.3
DOI:
Bibkey:
Cite (ACL):
Daniel Reichenpfader and Kerstin Denecke. 2024. Simulating Diverse Patient Populations Using Patient Vignettes and Large Language Models. In Proceedings of the First Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC-COLING 2024, pages 20–25, Torino, Italia. ELRA and ICCL.
Cite (Informal):
Simulating Diverse Patient Populations Using Patient Vignettes and Large Language Models (Reichenpfader & Denecke, CL4Health-WS 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/2024.cl4health-1.3.pdf
PDF Search