FairBelief - Assessing Harmful Beliefs in Language Models
Mattia Setzu, Marta Marchiori Manerba, Pasquale Minervini, Debora Nozza
Abstract
Language Models (LMs) have been shown to inherit undesired biases that might hurt minorities and underrepresented groups if such systems were integrated into real-world applications without careful fairness auditing.This paper proposes FairBelief, an analytical approach to capture and assess beliefs, i.e., propositions that an LM may embed with different degrees of confidence and that covertly influence its predictions. With FairBelief, we leverage prompting to study the behavior of several state-of-the-art LMs across different previously neglected axes, such as model scale and likelihood, assessing predictions on a fairness dataset specifically designed to quantify LMs’ outputs’ hurtfulness.Finally, we conclude with an in-depth qualitative assessment of the beliefs emitted by the models.We apply FairBelief to English LMs, revealing that, although these architectures enable high performances on diverse natural language processing tasks, they show hurtful beliefs about specific genders. Interestingly, training procedure and dataset, model scale, and architecture induce beliefs of different degrees of hurtfulness.- Anthology ID:
- 2024.trustnlp-1.3
- Volume:
- Proceedings of the 4th Workshop on Trustworthy Natural Language Processing (TrustNLP 2024)
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Kai-Wei Chang, Anaelia Ovalle, Jieyu Zhao, Yang Trista Cao, Ninareh Mehrabi, Aram Galstyan, Jwala Dhamala, Anoop Kumar, Rahul Gupta
- Venues:
- TrustNLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 27–39
- Language:
- URL:
- https://aclanthology.org/2024.trustnlp-1.3
- DOI:
- Cite (ACL):
- Mattia Setzu, Marta Marchiori Manerba, Pasquale Minervini, and Debora Nozza. 2024. FairBelief - Assessing Harmful Beliefs in Language Models. In Proceedings of the 4th Workshop on Trustworthy Natural Language Processing (TrustNLP 2024), pages 27–39, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- FairBelief - Assessing Harmful Beliefs in Language Models (Setzu et al., TrustNLP-WS 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.trustnlp-1.3.pdf