Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili

Ruolei Zhang, Teddy Njuguna, Yue Feng


Abstract
Large language models are increasingly deployed in multilingual contexts, yet safety alignment and bias evaluation remain overwhelmingly English-centric. We investigate whether social biases generalise across languages by submitting 4,900 symmetric English–Swahili prompt pairs to GPT-5.2 and Gemini 2.5 Flash across nine demographic bias axes, yielding 19,600 completions evaluated for stereotype prevalence, sentiment, refusal behaviour, and cross-lingual semantic similarity. Our findings show that bias transforms rather than transfers: stereotype rates shifted by up to 12 percentage points on specific axes, Gemini’s neutral-sentiment rate doubled in Swahili, and GPT-5.2 refused 169 prompts in English and zero in Swahili, indicating safety mechanisms functionally anchored to English-language tokens. Over 55% of prompt pairs produced semantically dissimilar completions across both models. These reinforce the idea that English-only bias audits do not produce adequate coverage for multilingual deployment.
Anthology ID:
2026.mellm-1.17
Volume:
Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026)
Month:
July
Year:
2026
Address:
San Diego, United States
Editors:
Kaiyu Huang, Fengran Mo, Pinzhen Chen, Meng Jiang
Venues:
MeLLM | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
181–190
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.mellm-1.17/
DOI:
Bibkey:
Cite (ACL):
Ruolei Zhang, Teddy Njuguna, and Yue Feng. 2026. Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili. In Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM 2026), pages 181–190, San Diego, United States. Association for Computational Linguistics.
Cite (Informal):
Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili (Zhang et al., MeLLM 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.mellm-1.17.pdf