@inproceedings{ding-etal-2025-consultant,
    title = "Consultant Decoding: Yet Another Synergistic Mechanism",
    author = "Ding, Chuanghao  and
      Wang, Jiaping  and
      Yang, Ziqing  and
      Wang, Xiaoliang  and
      Lin, Dahua  and
      Cam-Tu, Nguyen  and
      Tan, Fei",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.findings-acl.797/",
    pages = "15438--15452",
    ISBN = "979-8-89176-256-5",
    abstract = "The synergistic mechanism based on Speculative Decoding (SD) has garnered considerable attention as a simple yet effective approach for accelerating the inference of large language models (LLMs). Nonetheless, the high rejection rates require repeated LLMs calls to validate draft tokens, undermining the overall efficiency gain of SD.In this work, we revisit existing verification mechanisms and propose a novel synergetic mechanism Consultant Decoding (CD). CD achieves up to a 2.5-fold increase in inference speed compared to the target model, while maintaining comparable generation quality ({\textasciitilde}100{\%} of the target model{'}s performance). Interestingly, this is achieved by combining models whose parameter sizes differ by two orders of magnitude.In addition, CD reduces the call frequency of the large target model to below 10{\%}, particularly in more demanding tasks.CD{'}s performance was even found to surpass that of the large target model, which theoretically represents the upper bound for speculative decoding."
}