“Flex Tape Can’t Fix That”: Bias and Misinformation in Edited Language Models
Karina H Halevy, Anna Sotnikova, Badr AlKhamissi, Syrielle Montariol, Antoine Bosselut
Abstract
Weight-based model editing methods update the parametric knowledge of language models post-training. However, these methods can unintentionally alter unrelated parametric knowledge representations, potentially increasing the risk of harm. In this work, we investigate how weight editing methods unexpectedly amplify model biases after edits. We introduce a novel benchmark dataset, Seesaw-CF, for measuring bias amplification of model editing methods for demographic traits such as race, geographic origin, and gender. We use Seesaw-CF to examine the impact of model editing on bias in five large language models. Our results demonstrate that edited models exhibit, to various degrees, more biased behavior for certain demographic groups than before they were edited, specifically becoming less confident in properties for Asian and African subjects. Additionally, editing facts about place of birth, country of citizenship, or gender has particularly negative effects on the model’s knowledge about unrelated properties, such as field of work, a pattern observed across multiple models.- Anthology ID:
- 2024.emnlp-main.494
- Volume:
- Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 8690–8707
- Language:
- URL:
- https://preview.aclanthology.org/remove-affiliations/2024.emnlp-main.494/
- DOI:
- 10.18653/v1/2024.emnlp-main.494
- Cite (ACL):
- Karina H Halevy, Anna Sotnikova, Badr AlKhamissi, Syrielle Montariol, and Antoine Bosselut. 2024. “Flex Tape Can’t Fix That”: Bias and Misinformation in Edited Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8690–8707, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- “Flex Tape Can’t Fix That”: Bias and Misinformation in Edited Language Models (Halevy et al., EMNLP 2024)
- PDF:
- https://preview.aclanthology.org/remove-affiliations/2024.emnlp-main.494.pdf