@inproceedings{zhao-etal-2025-fistapruner,
    title = "{FISTAP}runer: Layer-wise Post-training Pruning for Large Language Models",
    author = "Zhao, Pengxiang  and
      Hu, Hanyu  and
      Li, Ping  and
      Zheng, Yi  and
      Wang, Zhefeng  and
      Yuan, Xiaoming",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1498/",
    pages = "29457--29475",
    ISBN = "979-8-89176-332-6",
    abstract = "Pruning is a critical strategy for compressing trained large language models (LLMs), aiming at substantial memory conservation and computational acceleration without compromising performance. However, existing pruning methods typically necessitate inefficient retraining for billion-scale LLMs or rely on heuristically designed metrics to determine pruning masks, leading to performance degradation. This paper presents, for the first time, a LASSO-like convex optimization model crafted to induce sparsity in LLMs. By leveraging FISTA, we introduce FISTAPruner, a novel method that includes a cumulative error elimination mechanism within decoder layers and supports parallel pruning for unstructured pruning. Additionally, we extend this method to 2:4 semi-structured pruning. We comprehensively evaluate FISTAPruner on models such as OPT, LLaMA, and Qwen variants with 125M to 70B parameters under unstructured and 2:4 semi-structured sparsity, showcasing superior performance over existing methods across various language benchmarks. Notably, it can remove 50{\%} of the model parameters for LLaMA-3-70B while retaining 98.6{\%} and 95.6{\%} of the zero-shot task performance under these two sparsity patterns, respectively."
}