@inproceedings{wang-etal-2026-erasing,
    title = "Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Models",
    author = "Wang, Huazheng  and
      Jing, Yongcheng  and
      Sun, Haifeng  and
      Wang, Yingjie  and
      Wang, Jingyu  and
      Liao, Jianxin  and
      Tao, Dacheng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.88/",
    pages = "1962--1994",
    ISBN = "979-8-89176-390-6",
    abstract = "In this paper, we investigate knowledge forgetting in large language models with a focus on its generalisation{---}ensuring that models forget not only specific training samples but also related implicit knowledge. To this end, we begin by identifying a broader unlearning scope that includes both target data and logically associated samples, including rephrased, subject-replaced, relation-reversed, and one-hop reasoned data. We then conduct a rigorous evaluation of 15 state-of-the-art methods across three datasets, revealing that unlearned models still recall paraphrased answers and retain target facts in their intermediate layers. This motivates us to take a preliminary step toward more generalised implicit knowledge forgetting by proposing PERMU{---}a novel probability perturbation-based unlearning paradigm. PERMU simulates adversarial unlearning samples to eliminate fact-related tokens from the logit distribution, collectively reducing the probabilities of all answer-associated tokens. Experiments are conducted on a diverse range of datasets, including TOFU, Harry Potter, ZsRE, WMDP, and MUSE, using models ranging from 1.3B to 13B in scale. The results demonstrate that PERMU delivers up to a 50.40{\%} improvement in unlearning vanilla target data while maintaining a 40.73{\%} boost in forgetting implicit knowledge. Our code can be found in the supplementary material."
}