Abstract
We propose a bias-aware methodology to engage with power relations in natural language processing (NLP) research. NLP research rarely engages with bias in social contexts, limiting its ability to mitigate bias. While researchers have recommended actions, technical methods, and documentation practices, no methodology exists to integrate critical reflections on bias with technical NLP methods. In this paper, after an extensive and interdisciplinary literature review, we contribute a bias-aware methodology for NLP research. We also contribute a definition of biased text, a discussion of the implications of biased NLP systems, and a case study demonstrating how we are executing the bias-aware methodology in research on archival metadata descriptions.- Anthology ID:
- 2020.gebnlp-1.10
- Volume:
- Proceedings of the Second Workshop on Gender Bias in Natural Language Processing
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona, Spain (Online)
- Venue:
- GeBNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 107–124
- Language:
- URL:
- https://aclanthology.org/2020.gebnlp-1.10
- DOI:
- Cite (ACL):
- Lucy Havens, Melissa Terras, Benjamin Bach, and Beatrice Alex. 2020. Situated Data, Situated Systems: A Methodology to Engage with Power Relations in Natural Language Processing Research. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pages 107–124, Barcelona, Spain (Online). Association for Computational Linguistics.
- Cite (Informal):
- Situated Data, Situated Systems: A Methodology to Engage with Power Relations in Natural Language Processing Research (Havens et al., GeBNLP 2020)
- PDF:
- https://preview.aclanthology.org/nodalida-main-page/2020.gebnlp-1.10.pdf
- Data
- GAP Coreference Dataset