Adversarial Concept Erasure in Kernel Space
Shauli Ravfogel, Francisco Vargas, Yoav Goldberg, Ryan Cotterell
Abstract
The representation space of neural models for textual data emerges in an unsupervised manner during training. Understanding how human-interpretable concepts, such as gender, are encoded in these representations would improve the ability of users to control the content of these representations and analyze the working of the models that rely on them. One prominent approach to the control problem is the identification and removal of linear concept subspaces – subspaces in the representation space that correspond to a given concept. While those are tractable and interpretable, neural network do not necessarily represent concepts in linear subspaces. We propose a kernelization of the recently-proposed linear concept-removal objective, and show that it is effective in guarding against the ability of certain nonlinear adversaries to recover the concept. Interestingly, our findings suggest that the division between linear and nonlinear models is overly simplistic: when considering the concept of binary gender and its neutralization, we do not find a single kernel space that exclusively contains all the concept-related information. It is therefore challenging to protect against all nonlinear adversaries at once.- Anthology ID:
- 2022.emnlp-main.405
- Volume:
- Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Editors:
- Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 6034–6055
- Language:
- URL:
- https://aclanthology.org/2022.emnlp-main.405
- DOI:
- 10.18653/v1/2022.emnlp-main.405
- Cite (ACL):
- Shauli Ravfogel, Francisco Vargas, Yoav Goldberg, and Ryan Cotterell. 2022. Adversarial Concept Erasure in Kernel Space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6034–6055, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- Adversarial Concept Erasure in Kernel Space (Ravfogel et al., EMNLP 2022)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2022.emnlp-main.405.pdf