@inproceedings{yang-etal-2025-wanda,
    title = "Wanda++: Pruning Large Language Models via Regional Gradients",
    author = {Yang, Yifan  and
      Zhen, Kai  and
      Ganesh, Bhavana  and
      Galstyan, Aram  and
      Huybrechts, Goeric  and
      M{\"u}ller, Markus  and
      K{\"u}bler, Jonas M.  and
      Swaminathan, Rupak Vignesh  and
      Mouchtaris, Athanasios  and
      Bodapati, Sravan Babu  and
      Susanj, Nathan  and
      Zhang, Zheng  and
      FitzGerald, Jack  and
      Kumar, Abhishek},
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/mtsummit-25-ingestion/2025.findings-acl.224/",
    doi = "10.18653/v1/2025.findings-acl.224",
    pages = "4321--4333",
    ISBN = "979-8-89176-256-5",
    abstract = "Large Language Models (LLMs) pruning seeks to remove unimportant weights for inference speedup with minimal accuracy impact. However, existing methods often suffer from accuracy degradation without full-model sparsity-aware fine-tuning. This paper presents Wanda++, a novel pruning framework that outperforms the state-of-the-art methods by utilizing decoder-block-level \textbf{regional} gradients. Specifically, Wanda++ improves the pruning score with regional gradients for the first time and proposes an efficient regional optimization method to minimize pruning-induced output discrepancies between the dense and sparse decoder output. Notably, Wanda++ improves perplexity by up to 32{\%} over Wanda in the language modeling task and generalizes effectively to downstream tasks. Moreover, despite updating weights with regional optimization, Wanda++ remains orthogonal to sparsity-aware fine-tuning, further reducing perplexity with LoRA in great extend. Our approach is lightweight, pruning a 7B LLaMA model in under 10 minutes on a single H100 GPU."
}