@inproceedings{feng-etal-2025-geoedit,
    title = "{G}eo{E}dit: Geometric Knowledge Editing for Large Language Models",
    author = "Feng, Yujie  and
      Zhan, Li-Ming  and
      Lu, Zexin  and
      Xu, Yongxin  and
      Chu, Xu  and
      Wang, Yasha  and
      Cao, Jiannong  and
      Yu, Philip S.  and
      Wu, Xiao-Ming",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.676/",
    pages = "13401--13416",
    ISBN = "979-8-89176-332-6",
    abstract = "Regular updates are essential for maintaining up-to-date knowledge in large language models (LLMs). However, existing training-based model editing methods often struggle to effectively incorporate new knowledge while preserving unrelated general knowledge. To address this challenge, we propose a novel framework called Geometric Knowledge Editing (GeoEdit). GeoEdit utilizes the geometric relationships of parameter updates from fine-tuning to differentiate between neurons associated with new knowledge updates and those related to general knowledge perturbations. By employing a direction-aware knowledge identification method, we avoid updating neurons with directions approximately orthogonal to existing knowledge, thus preserving the model{'}s generalization ability. For the remaining neurons, we integrate both old and new knowledge for aligned directions and apply a ``forget-then-learn'' editing strategy for opposite directions. Additionally, we introduce an importance-guided task vector fusion technique that filters out redundant information and provides adaptive neuron-level weighting, further enhancing model editing performance. Extensive experiments on two publicly available datasets demonstrate the superiority of GeoEdit over existing state-of-the-art methods."
}