@inproceedings{li-etal-2025-amphista,
    title = "Amphista: Bi-directional Multi-head Decoding for Accelerating {LLM} Inference",
    author = "Li, Zeping  and
      Yang, Xinlong  and
      Gao, Ziheng  and
      Liu, Ji  and
      Li, Guanchen  and
      Liu, Zhuang  and
      Li, Dong  and
      Peng, Jinzhang  and
      Tian, Lu  and
      Barsoum, Emad",
    editor = "Chiruzzo, Luis  and
      Ritter, Alan  and
      Wang, Lu",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.naacl-long.450/",
    pages = "8925--8938",
    ISBN = "979-8-89176-189-6",
    abstract = "Large Language Models (LLMs) inherently use autoregressive decoding, which lacks parallelism in inference and results in significantly slow inference speed. While methods such as Medusa constructs parallelized heads, they lack adequate information interaction across different prediction positions. To overcome this limitation, we introduce Amphista, an enhanced speculative decoding framework that builds upon Medusa. Specifically, Amphista models an *Auto-embedding Block* capable of parallel inference, incorporating bi-directional attention to enable interaction between different drafting heads. Additionally, Amphista integrates *Staged Adaptation Layers*, which ensure a seamless transition of semantic information from the target model{'}s autoregressive inference to the drafting heads' non-autoregressive inference, effectively achieving paradigm shift and feature fusion. Experimental results on Vicuna models using MT-Bench and Spec-Bench demonstrate that Amphista achieves substantial acceleration while maintaining generation quality. On MT-Bench, Amphista delivers up to **2.75{\texttimes}** speedup over vanilla autoregressive decoding and **1.40{\texttimes}** over Medusa on Vicuna 33B in wall-clock time."
}