DUSK: Do Not Unlearn Shared Knowledge
Wonje Jeung, Sangyeon Yoon, Hyesoo Hong, Soeun Kim, Seungju Han, Youngjae Yu, Albert No
Abstract
Machine unlearning aims to remove “forget” data while preserving knowledge from the “retain” data, yet a fundamental question arises when the two share content. By definition, an unlearned model should be indistinguishable from a model retrained solely on the retain set, which implies that shared knowledge must remain while only forget-specific content is removed. To evaluate this requirement, we introduce DUSK, the first benchmark for unlearning under realistic knowledge overlap. DUSK constructs documents containing both shared and unique knowledge and defines seven metrics to test whether methods erase forget-specific expressions without discarding shared facts. Evaluating nine recent approaches, we find that although surface text is often removed, current methods struggle to distinguish shared from unique knowledge, either erasing information that should be retained or failing to fully forget target content. DUSK provides a controlled, reproducible testbed for diagnosing these failures and guiding precise unlearning algorithms.- Anthology ID:
- 2026.findings-acl.2085
- Volume:
- Findings of the Association for Computational Linguistics: ACL 2026
- Month:
- July
- Year:
- 2026
- Address:
- San Diego, California, United States
- Editors:
- Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 42013–42031
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl/2026.findings-acl.2085/
- DOI:
- Cite (ACL):
- Wonje Jeung, Sangyeon Yoon, Hyesoo Hong, Soeun Kim, Seungju Han, Youngjae Yu, and Albert No. 2026. DUSK: Do Not Unlearn Shared Knowledge. In Findings of the Association for Computational Linguistics: ACL 2026, pages 42013–42031, San Diego, California, United States. Association for Computational Linguistics.
- Cite (Informal):
- DUSK: Do Not Unlearn Shared Knowledge (Jeung et al., Findings 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl/2026.findings-acl.2085.pdf