Building Bridges: A Dataset for Evaluating Gender-Fair Machine Translation into German

Manuel Lardelli, Giuseppe Attanasio, Anne Lauscher


Abstract
The translation of gender-neutral person-referring terms (e.g.,the students) is often non-trivial.Translating from English into German poses an interesting case—in German, person-referring nouns are usually gender-specific, and if the gender of the referent(s) is unknown or diverse, the generic masculine (die Studenten (m.)) is commonly used. This solution, however, reduces the visibility of other genders, such as women and non-binary people. To counteract gender discrimination, a societal movement towards using gender-fair language exists (e.g., by adopting neosystems). However, gender-fair German is currently barely supported in machine translation (MT), requiring post-editing or manual translations. We address this research gap by studying gender-fair language in English-to-German MT. Concretely, we enrich a community-created gender-fair language dictionary and sample multi-sentence test instances from encyclopedic text and parliamentary speeches.Using these novel resources, we conduct the first benchmark study involving two commercial systems and six neural MT models for translating words in isolation and natural contexts across two domains. Our findings show that most systems produce mainly masculine forms and rarely gender-neutral variants, highlighting the need for future research. We release code and data at https://github.com/g8a9/building-bridges-gender-fair-german-mt.
Anthology ID:
2024.findings-acl.448
Volume:
Findings of the Association for Computational Linguistics: ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7542–7550
Language:
URL:
https://aclanthology.org/2024.findings-acl.448
DOI:
10.18653/v1/2024.findings-acl.448
Bibkey:
Cite (ACL):
Manuel Lardelli, Giuseppe Attanasio, and Anne Lauscher. 2024. Building Bridges: A Dataset for Evaluating Gender-Fair Machine Translation into German. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7542–7550, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Building Bridges: A Dataset for Evaluating Gender-Fair Machine Translation into German (Lardelli et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2024.findings-acl.448.pdf