Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation

Shahar Levy, Koren Lazar, Gabriel Stanovsky


Abstract
Recent works have found evidence of gender bias in models of machine translation and coreference resolution using mostly synthetic diagnostic datasets. While these quantify bias in a controlled experiment, they often do so on a small scale and consist mostly of artificial, out-of-distribution sentences. In this work, we find grammatical patterns indicating stereotypical and non-stereotypical gender-role assignments (e.g., female nurses versus male dancers) in corpora from three domains, resulting in a first large-scale gender bias dataset of 108K diverse real-world English sentences. We manually verify the quality of our corpus and use it to evaluate gender bias in various coreference resolution and machine translation models. We find that all tested models tend to over-rely on gender stereotypes when presented with natural inputs, which may be especially harmful when deployed in commercial systems. Finally, we show that our dataset lends itself to finetuning a coreference resolution model, finding it mitigates bias on a held out set. Our dataset and models are publicly available at github.com/SLAB-NLP/BUG. We hope they will spur future research into gender bias evaluation mitigation techniques in realistic settings.
Anthology ID:
2021.findings-emnlp.211
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2021
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
2470–2480
Language:
URL:
https://aclanthology.org/2021.findings-emnlp.211
DOI:
10.18653/v1/2021.findings-emnlp.211
Bibkey:
Cite (ACL):
Shahar Levy, Koren Lazar, and Gabriel Stanovsky. 2021. Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2470–2480, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation (Levy et al., Findings 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2021.findings-emnlp.211.pdf
Video:
 https://preview.aclanthology.org/ingestion-script-update/2021.findings-emnlp.211.mp4
Code
 slab-nlp/bug
Data
BUGGAP Coreference DatasetWinoBias