@inproceedings{lyu-etal-2025-streamlining,
    title = "Streamlining the Collaborative Chain of Models into A Single Forward Pass in Generation-Based Tasks",
    author = "Lyu, Yuanjie  and
      Zhang, Chao  and
      Chen, Yuhao  and
      Chen, Yong  and
      Xu, Tong",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.findings-acl.330/",
    pages = "6390--6404",
    ISBN = "979-8-89176-256-5",
    abstract = "In Retrieval-Augmented Generation (RAG) and agent-based frameworks, the ``Chain of Models'' approach is widely used, where multiple specialized models work sequentially on distinct sub-tasks. This approach is effective but increases resource demands as each model must be deployed separately. Recent advancements attempt to address this by applying prompt tuning, which allows a shared base model to adapt to multiple tasks with minimal parameter changes. However, a key challenge remains: intermediate outputs, passed between models as plain text, require recomputation of hidden states (i.e., Key and Value (KV) states in Transformers) during inference. In this paper, we introduce FTHSS, a novel prompt-tuning method that enables models to share KV hidden states, eliminating redundant forward passes and reducing KV cache storage. By modifying input and attention masks during training, FTHSS allows models to effectively utilize KV hidden states from prior models in both single- and multi-round scenarios. Empirical results on four tasks show that FTHSS matches the performance of traditional model chains while improving inference efficiency."
}