@inproceedings{jiao-etal-2025-synergistic,
    title = "Synergistic Weak-Strong Collaboration by Aligning Preferences",
    author = "Jiao, Yizhu  and
      Zhang, Xuchao  and
      Wang, Zhaoyang  and
      Ma, Yubo  and
      Deng, Zhun  and
      Wang, Rujia  and
      Bansal, Chetan  and
      Rajmohan, Saravan  and
      Han, Jiawei  and
      Yao, Huaxiu",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.995/",
    pages = "20355--20371",
    ISBN = "979-8-89176-251-0",
    abstract = "Current Large Language Models excel in general reasoning yet struggle with specialized tasks requiring proprietary or domain-specific knowledge. Fine-tuning large models for every niche application is often infeasible due to black-box constraints and high computational overhead. To address this, we propose a collaborative framework that pairs a specialized weak model with a general strong model. The weak model, tailored to specific domains, produces initial drafts and background information, while the strong model leverages its advanced reasoning to refine these drafts, extending LLMs' capabilities to critical yet specialized tasks. To optimize this collaboration, we introduce a collaborative feedback to fine-tunes the weak model, which quantifies the influence of the weak model{'}s contributions in the collaboration procedure and establishes preference pairs to guide preference tuning of the weak model. We validate our framework through experiments on three domains. We find that the collaboration significantly outperforms each model alone by leveraging complementary strengths. Moreover, aligning the weak model with the collaborative preference further enhances overall performance."
}