Overview of the Shared Task on Machine Translation Gender Bias Evaluation with Multilingual Holistic Bias
Marta Costa-jussà, Pierre Andrews, Christine Basta, Juan Ciro, Agnieszka Falenska, Seraphina Goldfarb-Tarrant, Rafael Mosquera, Debora Nozza, Eduardo Sánchez
Abstract
We describe the details of the Shared Task of the 5th ACL Workshop on Gender Bias in Natural Language Processing (GeBNLP 2024). The task uses dataset to investigate the quality of Machine Translation systems on a particular case of gender robustness. We report baseline results as well as the results of the first participants. The shared task will be permanently available in the Dynabench platform.- Anthology ID:
- 2024.gebnlp-1.26
- Volume:
- Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Agnieszka Faleńska, Christine Basta, Marta Costa-jussà, Seraphina Goldfarb-Tarrant, Debora Nozza
- Venues:
- GeBNLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 399–404
- Language:
- URL:
- https://aclanthology.org/2024.gebnlp-1.26
- DOI:
- Cite (ACL):
- Marta Costa-jussà, Pierre Andrews, Christine Basta, Juan Ciro, Agnieszka Falenska, Seraphina Goldfarb-Tarrant, Rafael Mosquera, Debora Nozza, and Eduardo Sánchez. 2024. Overview of the Shared Task on Machine Translation Gender Bias Evaluation with Multilingual Holistic Bias. In Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 399–404, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Overview of the Shared Task on Machine Translation Gender Bias Evaluation with Multilingual Holistic Bias (Costa-jussà et al., GeBNLP-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.gebnlp-1.26.pdf