Roadblocks in Gender Bias Measurement for Diachronic Corpora
Saied Alshahrani, Esma Wali, Abdullah R Alshamsan, Yan Chen, Jeanna Matthews
Abstract
The use of word embeddings is an important NLP technique for extracting meaningful conclusions from corpora of human text. One important question that has been raised about word embeddings is the degree of gender bias learned from corpora. Bolukbasi et al. (2016) proposed an important technique for quantifying gender bias in word embeddings that, at its heart, is lexically based and relies on sets of highly gendered word pairs (e.g., mother/father and madam/sir) and a list of professions words (e.g., doctor and nurse). In this paper, we document problems that arise with this method to quantify gender bias in diachronic corpora. Focusing on Arabic and Chinese corpora, in particular, we document clear changes in profession words used over time and, somewhat surprisingly, even changes in the simpler gendered defining set word pairs. We further document complications in languages such as Arabic, where many words are highly polysemous/homonymous, especially female professions words.- Anthology ID:
- 2022.lchange-1.15
- Volume:
- Proceedings of the 3rd Workshop on Computational Approaches to Historical Language Change
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Nina Tahmasebi, Syrielle Montariol, Andrey Kutuzov, Simon Hengchen, Haim Dubossarsky, Lars Borin
- Venue:
- LChange
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 140–148
- Language:
- URL:
- https://aclanthology.org/2022.lchange-1.15
- DOI:
- 10.18653/v1/2022.lchange-1.15
- Cite (ACL):
- Saied Alshahrani, Esma Wali, Abdullah R Alshamsan, Yan Chen, and Jeanna Matthews. 2022. Roadblocks in Gender Bias Measurement for Diachronic Corpora. In Proceedings of the 3rd Workshop on Computational Approaches to Historical Language Change, pages 140–148, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Roadblocks in Gender Bias Measurement for Diachronic Corpora (Alshahrani et al., LChange 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2022.lchange-1.15.pdf
- Code
- clarkson-accountability-transparency/gbiasroadblocks