Abstract
We analyze the Stanford Natural Language Inference (SNLI) corpus in an investigation of bias and stereotyping in NLP data. The SNLI human-elicitation protocol makes it prone to amplifying bias and stereotypical associations, which we demonstrate statistically (using pointwise mutual information) and with qualitative examples.- Anthology ID:
- W17-1609
- Volume:
- Proceedings of the First ACL Workshop on Ethics in Natural Language Processing
- Month:
- April
- Year:
- 2017
- Address:
- Valencia, Spain
- Editors:
- Dirk Hovy, Shannon Spruit, Margaret Mitchell, Emily M. Bender, Michael Strube, Hanna Wallach
- Venue:
- EthNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 74–79
- Language:
- URL:
- https://aclanthology.org/W17-1609
- DOI:
- 10.18653/v1/W17-1609
- Cite (ACL):
- Rachel Rudinger, Chandler May, and Benjamin Van Durme. 2017. Social Bias in Elicited Natural Language Inferences. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pages 74–79, Valencia, Spain. Association for Computational Linguistics.
- Cite (Informal):
- Social Bias in Elicited Natural Language Inferences (Rudinger et al., EthNLP 2017)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/W17-1609.pdf
- Code
- cjmay/snli-ethics
- Data
- SNLI