@inproceedings{savelli-etal-2025-malto-semeval,
    title = "{MALTO} at {S}em{E}val-2025 Task 4: Dual Teachers for Unlearning Sensitive Content in {LLM}s",
    author = "Savelli, Claudio  and
      Munis, Evren  and
      Bayat, Erfan  and
      Grieco, Andrea  and
      Giobergia, Flavio",
    editor = "Rosenthal, Sara  and
      Ros{\'a}, Aiala  and
      Ghosh, Debanjan  and
      Zampieri, Marcos",
    booktitle = "Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/transition-to-people-yaml/2025.semeval-1.229/",
    pages = "1747--1752",
    ISBN = "979-8-89176-273-2",
    abstract = "Large language models (LLMs) may retain and reproduce sensitive information learned during training, posing significant privacy and ethical concerns. Once detected, this personal information should be deleted from the model. A naive answer could be to retrain these models from scratch when needed. However, this solution is unfeasible given the immense computational, economic, and environmental costs required to train these models. For this reason, Machine Unlearning (MU) has risen in recent years as an emerging field of research to efficiently delete specific information from a model{'}s knowledge. This paper presents our solution to the ``Unlearning sensitive content from Large Language Models'' shared task at SemEval-2025, which challenges researchers to develop effective LLM MU techniques. We adopt a Dual-Teacher framework that leverages a Competent and an Incompetent Teacher to erase unwanted information while selectively preserving model utility. Our approach adapts established computer vision unlearning methods to the sequential nature of language models through KL divergence minimization over next-token prediction probabilities. Our experimental results demonstrate that our method outperforms the state-of-the-art techniques."
}