Large Language Models for Mental Health: A Multilingual Evaluation
Nishat Raihan, Sadiya Sayara Chowdhury Puspo, Ana-Maria Bucur, Stevie Chancellor, Marcos Zampieri
Abstract
Large Language Models (LLMs) have remarkable capabilities across NLP tasks. However, their performance in multilingual contexts, especially within the mental health domain, has not been thoroughly explored. In this paper, we evaluate proprietary and open-source LLMs on eight mental health datasets in various languages, as well as their machine-translated (MT) counterparts. We compare LLM performance in zero-shot, few-shot, and fine-tuned settings against conventional NLP baselines that do not employ LLMs. In addition, we assess translation quality across language families and typologies to understand its influence on LLM performance. Proprietary LLMs and fine-tuned open-source LLMs achieve competitive F1 scores on several datasets, often surpassing state-of-the-art results. However, performance on MT data is generally lower, and the extent of this decline varies by language and typology. This variation highlights both the strengths of LLMs in handling mental health tasks in languages other than English and their limitations when translation quality introduces structural or lexical mismatches.- Anthology ID:
- 2026.loreslm-1.29
- Volume:
- Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026)
- Month:
- March
- Year:
- 2026
- Address:
- Rabat, Morocco
- Editors:
- Hansi Hettiarachchi, Tharindu Ranasinghe, Alistair Plum, Paul Rayson, Ruslan Mitkov, Mohamed Gaber, Damith Premasiri, Fiona Anting Tan, Lasitha Uyangodage
- Venue:
- LoResLM
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 335–346
- Language:
- URL:
- https://preview.aclanthology.org/manual-author-scripts/2026.loreslm-1.29/
- DOI:
- Cite (ACL):
- Nishat Raihan, Sadiya Sayara Chowdhury Puspo, Ana-Maria Bucur, Stevie Chancellor, and Marcos Zampieri. 2026. Large Language Models for Mental Health: A Multilingual Evaluation. In Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026), pages 335–346, Rabat, Morocco. Association for Computational Linguistics.
- Cite (Informal):
- Large Language Models for Mental Health: A Multilingual Evaluation (Raihan et al., LoResLM 2026)
- PDF:
- https://preview.aclanthology.org/manual-author-scripts/2026.loreslm-1.29.pdf