@inproceedings{hu-etal-2025-stitchllm,
    title = "{S}titch{LLM}: Serving {LLM}s, One Block at a Time",
    author = "Hu, Bodun  and
      Li, Shuozhe  and
      Agarwal, Saurabh  and
      Lee, Myungjin  and
      Jajoo, Akshay  and
      Li, Jiamin  and
      Xu, Le  and
      Kim, Geon-Woo  and
      Kim, Donghyun  and
      Xu, Hong  and
      Zhang, Amy  and
      Akella, Aditya",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1305/",
    pages = "26887--26903",
    ISBN = "979-8-89176-251-0",
    abstract = "The rapid evolution of large language models (LLMs) has revolutionized natural language processing (NLP) tasks such as text generation, translation, and comprehension. However, the increasing computational demands and inference costs of these models present significant challenges. This study investigates the dynamic and efficient utilization of pre-trained weights from open-sourced LLMs of varying parameter sizes to achieve an optimal balance between computational efficiency and task performance. Drawing inspiration from the dual-process theory of human cognition, we introduce StitchLLM: a dynamic model routing framework that employs a powerful bottom model to process all queries, and uses a lightweight routing mechanism to allocate computational resources appropriately. Our novel framework optimizes efficiency and maintains performance, leveraging a trainable stitching layer for seamless integration of decoder layers across different LLMs. Experimental results demonstrate that StitchLLM improves system throughput while minimizing performance degradation, offering a flexible solution for deploying LLMs in resource-constrained settings."
}