@inproceedings{liu-etal-2025-ecolora,
    title = "{E}co{L}o{RA}: Communication-Efficient Federated Fine-Tuning of Large Language Models",
    author = "Liu, Han  and
      Wen, Ruoyao  and
      Nair, Srijith  and
      Liu, Jia  and
      Lou, Wenjing  and
      Zhang, Chongjie  and
      Yeoh, William  and
      Vorobeychik, Yevgeniy  and
      Zhang, Ning",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1046/",
    pages = "20743--20757",
    ISBN = "979-8-89176-332-6",
    abstract = "To address data locality and privacy restrictions, Federated Learning (FL) has recently been adopted to fine-tune large language models (LLMs), enabling improved performance on various downstream tasks without requiring aggregated data. However, the repeated exchange of model updates in FL can result in prohibitively high communication costs, hindering the distributed learning process. To address this challenge, we propose EcoLoRA, a novel communication-efficient federated fine-tuning framework for LLMs. Leveraging the modular structure, we propose a round-robin segment sharing scheme, where each client uploads only a complementary LoRA segment per round to reduce network bandwidth. It is further combined with adaptive sparsification methods tailored to LoRA{'}s training dynamics and lossless encoding techniques. We conduct extensive evaluations on both question-answering and value-alignment tasks across multiple datasets and models. The results show that EcoLoRA significantly reduces communication overhead without compromising performance. For instance, it reduces communication time by up to 79{\%} and total training time by up to 65{\%}."
}