Lost in Translation: Cross-Cultural Bias in LLM-Assisted Medical Symptom Interpretation
Yuting Tian, Salar Khaleghzadegan, Benjamin Huh, Yash Raj, Gena Heng
Abstract
Large language models (LLMs) are increasingly used to convert patient language into clinical-style summaries, yet patient symptom descriptions may vary across linguistic, cultural, and cross-linguistic contexts. In this pilot study, we operationalize this variation using four expression styles: direct English, indirect English, culturally mediated English, and Chinese-original patient language. We propose a compact red-teaming framework for testing whether LLM-based symptom interpretation changes when the same underlying concern is expressed in different linguistic and cultural forms. Our pilot dataset contains eight symptom scenarios, each expressed in four styles, yielding 32 vignettes before prompt variation. We evaluate GPT-5 mini as a pilot case-study model under generic and culture-aware prompts, repeating the full evaluation three times to produce 192 model outputs. Reference labels and a stratified subset of model output annotations were reviewed for face validity by an independent reviewer with clinical training.The model usually preserves broad symptom categories, but subtle failure modes emerge. Culture-aware prompting reduces severity downgrades from 14.6% to 9.4% and ambiguity-flagging failures from 28.1% to 13.5%, but does not reduce interpretation inconsistency or clinical category shift, both of which remain at 6.2%. Indirect English shows the highest severity-downgrade and flagging-failure rates, while Chinese-original expressions are often interpreted with the correct broad category but are not consistently flagged as ambiguous. These findings suggest that medical LLM evaluation should assess cultural robustness, severity framing, ambiguity preservation, and human-review escalation in addition to factual accuracy.- Anthology ID:
- 2026.stereacult-1.2
- Volume:
- Proceedings of the 1st Workshop on Stereotypes Across Cultures in Language Technologies (StereACuLT 2026)
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, United States
- Editors:
- Weicheng Ma, Soroush Vosoughi, Nabeel Gillani, Rolando Coto-Solano
- Venues:
- StereACuLT | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 13–19
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.stereacult-1.2/
- DOI:
- Cite (ACL):
- Yuting Tian, Salar Khaleghzadegan, Benjamin Huh, Yash Raj, and Gena Heng. 2026. Lost in Translation: Cross-Cultural Bias in LLM-Assisted Medical Symptom Interpretation. In Proceedings of the 1st Workshop on Stereotypes Across Cultures in Language Technologies (StereACuLT 2026), pages 13–19, San Diego, California, United States. Association for Computational Linguistics.
- Cite (Informal):
- Lost in Translation: Cross-Cultural Bias in LLM-Assisted Medical Symptom Interpretation (Tian et al., StereACuLT 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.stereacult-1.2.pdf