NLPART at SemEval-2025 Task 4: Forgetting is harder than Learning

Hoorieh Sabzevari, Milad Molazadeh Oskuee, Tohid Abedini, Ghazal Zamaninejad, Sara Baruni, Zahra Amirmahani, Amirmohammad Salehoof


Abstract
Unlearning is a critical capability for ensuring privacy, security, and compliance in AI systems, enabling models to forget specific data while retaining overall performance. In this work, we participated in Task 4 of SemEval 2025, which focused on unlearning across three sub-tasks: (1) long-form synthetic creative documents, (2) short-form synthetic biographies containing personally identifiable information, and (3) real documents sampled from the target model’s training dataset. We conducted four experiments, employing Supervised Fine-Tuning (SFT) and Negative Preference Optimization (NPO). Despite achieving good performance on the retain set—data that the model was supposed to remember—our findings demonstrate that these techniques did not perform well on the forget set, where unlearning was required.
Anthology ID:
2025.semeval-1.304
Volume:
Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Sara Rosenthal, Aiala Rosá, Debanjan Ghosh, Marcos Zampieri
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2336–2341
Language:
URL:
https://preview.aclanthology.org/corrections-2025-08/2025.semeval-1.304/
DOI:
Bibkey:
Cite (ACL):
Hoorieh Sabzevari, Milad Molazadeh Oskuee, Tohid Abedini, Ghazal Zamaninejad, Sara Baruni, Zahra Amirmahani, and Amirmohammad Salehoof. 2025. NLPART at SemEval-2025 Task 4: Forgetting is harder than Learning. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 2336–2341, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
NLPART at SemEval-2025 Task 4: Forgetting is harder than Learning (Sabzevari et al., SemEval 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/corrections-2025-08/2025.semeval-1.304.pdf