@inproceedings{jin-etal-2025-unlearning,
    title = "Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate",
    author = "Jin, Xiaomeng  and
      Bu, Zhiqi  and
      Vinzamuri, Bhanukiran  and
      Ramakrishna, Anil  and
      Chang, Kai-Wei  and
      Cevher, Volkan  and
      Hong, Mingyi",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/Author-page-Marten-During-lu/2025.naacl-long.563/",
    pages = "11278--11294",
    ISBN = "979-8-89176-189-6",
    abstract = "Machine unlearning has been used to remove unwanted knowledge acquired by large language models (LLMs). In this paper, we examine machine unlearning from an optimization perspective, framing it as a regularized multi-task optimization problem, where one task optimizes a forgetting objective and another optimizes the model performance. In particular, we introduce a normalized gradient difference algorithm, enabling us to have better control over the trade-off between the objectives, while integrating a new, automatic learning rate scheduler. We provide a theoretical analysis and empirically demonstrate the superior performance of among state-of-the-art unlearning methods on the TOFU and MUSE datasets while exhibiting stable training."
}