Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE

Beatrice Savoldi, Giuseppe Attanasio, Eleonora Cupin, Eleni Gkovedarou, Janiça Hackenbuchner, Anne Lauscher, Matteo Negri, Andrea Piergentili, Manjinder Thind, Luisa Bentivogli


Abstract
Avoiding the propagation of undue (binary) gender inferences and default masculine language remains a key challenge towards inclusive multilingual technologies, particularly when translating into languages with extensive gendered morphology. Gender-neutral translation (GNT) represents a linguistic strategy towards fairer communication across languages. However, research on GNT is limited to a few resources and language pairs. To address this gap, we introduce mGeNTE, an expert-curated resource, and use it to conduct the first systematic multilingual evaluation of inclusive translation with state-of-the-art instruction-following language models (LMs). Experiments on en-es/de/it/el reveal that while models can recognize when neutrality is appropriate, they cannot consistently produce neutral translations, limiting their usability. To probe this behavior, we enrich our evaluation with interpretability analyses that identify task-relevant features and offer initial insights into the internal dynamics of LM-based GNT.
Anthology ID:
2025.emnlp-main.692
Volume:
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2025
Address:
Suzhou, China
Editors:
Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, Violet Peng
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
13709–13731
Language:
URL:
https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.692/
DOI:
Bibkey:
Cite (ACL):
Beatrice Savoldi, Giuseppe Attanasio, Eleonora Cupin, Eleni Gkovedarou, Janiça Hackenbuchner, Anne Lauscher, Matteo Negri, Andrea Piergentili, Manjinder Thind, and Luisa Bentivogli. 2025. Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 13709–13731, Suzhou, China. Association for Computational Linguistics.
Cite (Informal):
Mind the Inclusivity Gap: Multilingual Gender-Neutral Translation Evaluation with mGeNTE (Savoldi et al., EMNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.692.pdf
Checklist:
 2025.emnlp-main.692.checklist.pdf