@inproceedings{chen-etal-2025-soft,
    title = "Soft Token Attacks Cannot Reliably Audit Unlearning in Large Language Models",
    author = "Chen, Haokun  and
      Szyller, Sebastian  and
      Xu, Weilin  and
      Himayat, Nageen",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.117/",
    doi = "10.18653/v1/2025.findings-emnlp.117",
    pages = "2183--2192",
    ISBN = "979-8-89176-335-7",
    abstract = "Large language models (LLMs) are trained using massive datasets.However, these datasets often contain undesirable content, e.g., harmful texts, personal information, and copyrighted material.To address this, \textit{machine unlearning} aims to remove information from trained models.Recent work has shown that soft token attacks () can successfully extract unlearned information from LLMs.In this work, we show that s can be an inadequate tool for auditing unlearning.Using common unlearning benchmarks, i.e., \textit{Who Is Harry Potter?} and \textit{TOFU}, we demonstrate that, in a \textit{strong auditor} setting, such attacks can elicit any information from the LLM, regardless of (1) the deployed unlearning algorithm, and (2) whether the queried content was originally present in the training corpus.Also, we show that with just a few soft tokens ($1-10$) can elicit random strings over 400-characters long.Thus showing that s must be used carefully to effectively audit unlearning.Example code can be found at https://github.com/IntelLabs/LLMart/tree/main/examples/unlearning"
}