@inproceedings{zhou-etal-2025-rankadaptor,
    title = "{R}ank{A}daptor: Hierarchical Rank Allocation for Efficient Fine-Tuning Pruned {LLM}s via Performance Model",
    author = "Zhou, Changhai  and
      Han, Shijie  and
      Yang, Lining  and
      Zhou, Yuhua  and
      Cheng, Xu  and
      Wang, Yibin  and
      Li, Hongguang",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Findings of the Association for Computational Linguistics: NAACL 2025",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.findings-naacl.321/",
    pages = "5781--5795",
    ISBN = "979-8-89176-195-7",
    abstract = "The efficient compression of large language models (LLMs) has become increasingly popular. However, recovering the performance of compressed LLMs remains a major challenge. The current practice in LLM compression entails the implementation of structural pruning, complemented by a recovery phase that leverages the Low-Rank Adaptation (LoRA) algorithm. Structural pruning{'}s uneven modification of model architecture, coupled with standard LoRA{'}s fixed configuration allocation across layers in an online pipeline, leads to suboptimal performance in various downstream tasks for pruned models. To address this challenge, we introduce RankAdaptor, a hierarchical rank allocation method that enables efficient fine-tuning of pruned LLMs according to layerwise specific recovery requirements. We employ a performance model that conducts offline meta-learning and online incremental learning to explore optimal rank values for each layer. Comprehensive experiments on popular benchmarks show that RankAdaptor consistently outperforms state-of-the-art methods across a variety of pruning settings and LLM architectures, with improvements ranging from 0.7{\%} to 5.5{\%}."
}