@inproceedings{kim-etal-2025-fractalllm,
    title = "{F}ractal{LLM}: Lossless Self-Speculative Decoding with Layer Embedded Self-Compression",
    author = "Kim, Juhyeong  and
      Yu, Sangyeon  and
      Kim, Gyunyeop  and
      Kang, Sangwoo",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.1286/",
    doi = "10.18653/v1/2025.findings-emnlp.1286",
    pages = "23666--23673",
    ISBN = "979-8-89176-335-7",
    abstract = "Autoregressive decoding in large language models (LLMs) necessitates a full forward pass for each generated token, significantly increasing inference latency. To address this limitation, we propose Fractal-LLM, a lossless self-speculative decoding method that embeds a compressed model within selected decoder layers of the original model. Specifically, our approach generates multiple draft tokens in parallel by injecting compressed layers into selected decoder layers. These draft tokens are subsequently verified through a single forward pass of the original model, ensuring the final outputs exactly match those produced by the original model. Experimental results across diverse benchmarks{---}including GSM8K, XSUM, CNN/DailyMail, and HumanEval{---}demonstrate that our method achieves substantial inference speed-ups (up to 2.47{\texttimes}) compared to standard autoregressive decoding, without requiring any additional training."
}