@inproceedings{lu-etal-2025-flrc,
    title = "{FLRC}: Fine-grained Low-Rank Compressor for Efficient {LLM} Inference",
    author = "Lu, Yu-Chen  and
      Chen, Chong-Yan  and
      Chang, Chi-Chih  and
      Hu, Yu-Fang  and
      Wu, Kai-Chiang",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.emnlp-main.755/",
    doi = "10.18653/v1/2025.emnlp-main.755",
    pages = "14956--14966",
    ISBN = "979-8-89176-332-6",
    abstract = "Although large language models (LLM) have achieved remarkable performance, their enormous parameter counts hinder deployment on resource-constrained hardware. Low-rank compression can reduce both memory usage and computational demand, but applying a uniform compression ratio across all layers often leads to significant performance degradation, and previous methods perform poorly during decoding. To address these issues, we propose the Fine-grained Low-Rank Compressor (FLRC), which efficiently determines an optimal rank allocation for each layer, and incorporates progressive low-rank decoding to maintain text generation quality. Comprehensive experiments on diverse benchmarks demonstrate the superiority of FLRC, achieving up to a 17{\%} improvement in ROUGE-L on summarization tasks compared to state-of-the-art low-rank compression methods, establishing a more robust and efficient framework to improve LLM inference."
}