DUSK: Do Not Unlearn Shared Knowledge

Wonje Jeung, Sangyeon Yoon, Hyesoo Hong, Soeun Kim, Seungju Han, Youngjae Yu, Albert No


Abstract
Machine unlearning aims to remove “forget” data while preserving knowledge from the “retain” data, yet a fundamental question arises when the two share content. By definition, an unlearned model should be indistinguishable from a model retrained solely on the retain set, which implies that shared knowledge must remain while only forget-specific content is removed. To evaluate this requirement, we introduce DUSK, the first benchmark for unlearning under realistic knowledge overlap. DUSK constructs documents containing both shared and unique knowledge and defines seven metrics to test whether methods erase forget-specific expressions without discarding shared facts. Evaluating nine recent approaches, we find that although surface text is often removed, current methods struggle to distinguish shared from unique knowledge, either erasing information that should be retained or failing to fully forget target content. DUSK provides a controlled, reproducible testbed for diagnosing these failures and guiding precise unlearning algorithms.
Anthology ID:
2026.findings-acl.2085
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
42013–42031
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.2085/
DOI:
Bibkey:
Cite (ACL):
Wonje Jeung, Sangyeon Yoon, Hyesoo Hong, Soeun Kim, Seungju Han, Youngjae Yu, and Albert No. 2026. DUSK: Do Not Unlearn Shared Knowledge. In Findings of the Association for Computational Linguistics: ACL 2026, pages 42013–42031, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
DUSK: Do Not Unlearn Shared Knowledge (Jeung et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.2085.pdf
Checklist:
 2026.findings-acl.2085.checklist.pdf