Abstract
Due to their similarity-based learning objectives, pretrained sentence encoders often internalize stereotypical assumptions that reflect the social biases that exist within their training corpora. In this paper, we describe several kinds of stereotypes concerning different communities that are present in popular sentence representation models, including pretrained next sentence prediction and contrastive sentence representation models. We compare such models to textual entailment models that learn language logic for a variety of downstream language understanding tasks. By comparing strong pretrained models based on text similarity with textual entailment learning, we conclude that the explicit logic learning with textual entailment can significantly reduce bias and improve the recognition of social communities, without an explicit de-biasing process.- Anthology ID:
- 2023.eacl-main.89
- Volume:
- Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Andreas Vlachos, Isabelle Augenstein
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1243–1254
- Language:
- URL:
- https://aclanthology.org/2023.eacl-main.89
- DOI:
- 10.18653/v1/2023.eacl-main.89
- Cite (ACL):
- Hongyin Luo and James Glass. 2023. Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoning. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1243–1254, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoning (Luo & Glass, EACL 2023)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2023.eacl-main.89.pdf