@inproceedings{li-etal-2026-qrelo,
    title = "{D}-{QRELO}: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation",
    author = "Li, Junlin  and
      Song, Shuangyong  and
      DU, Guodong  and
      Wong, Ngai  and
      Liu, Xuebo  and
      Li, Yongxiang  and
      Zhang, Min  and
      Li, Jing  and
      Li, Xuelong",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1081/",
    pages = "21507--21518",
    ISBN = "979-8-89176-395-1",
    abstract = "Supervised Fine-Tuning (SFT) accelerates task-specific large language models (LLMs) development, but the resulting proliferation of fine-tuned models incurs substantial memory overhead. Delta compression addresses this by retaining a single pre-trained LLM with multiple compressed delta weights. However, existing methods fail on models fine-tuned with large-scale datasets. We find that larger SFT data scale amplifies delta parameter magnitude, singular values, and entropy, exacerbating compression errors. To tackle this, we propose D-QRELO ( $\textbf{D}$elta Compression via $\textbf{Q}$uantization and $\textbf{R}$sidual $\textbf{Lo}$w-Rank), a novel training- and data-free delta compression method. It combines coarse-grained one-bit quantization to capture the dominant structure of the delta, followed by compensated residual low-rank approximation to recover fine-grained details from the smaller residual error. Experiments on various LLMs spanning dense and MoE architectures across multiple domains under this challenging setting demonstrate that D-QRELO outperforms existing methods. Moreover, we establish key design principles for delta compression through extensive empirical analysis, demonstrating how task difficulty, architecture, and layer positioning create predictable patterns that can guide optimal compression strategies in production systems."
}