@inproceedings{akhondzadeh-etal-2025-kurtail,
    title = "{K}ur{T}ail : Kurtosis-based {LLM} Quantization",
    author = "Akhondzadeh, Mohammad Sadegh  and
      Bojchevski, Aleksandar  and
      Eleftheriou, Evangelos  and
      Dazzi, Martino",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.943/",
    doi = "10.18653/v1/2025.findings-emnlp.943",
    pages = "17404--17419",
    ISBN = "979-8-89176-335-7",
    abstract = "One challenge of quantizing a large language model (LLM) is the presence of outliers. Outliers often make uniform quantization schemes less effective, particularly in extreme cases such as 4-bit quantization. We introduce KurTail, a new post-training quantization (PTQ) scheme that leverages Kurtosis-based rotation to mitigate outliers in the activations of LLMs. Our method optimizes Kurtosis as a measure of tailedness. This approach enables the quantization of weights, activations, and the KV cache in 4 bits. We utilize layer-wise optimization, ensuring memory efficiency. KurTail outperforms existing quantization methods, offering a 13.3{\%} boost in MMLU accuracy and a 15.5{\%} boost in Wiki perplexity compared to QuaRot. It also outperforms SpinQuant with a 2.6{\%} MMLU gain and reduces perplexity by 2.9{\%}, all while reducing the training cost. For comparison, learning the rotation using SpinQuant for Llama3-70B requires at least four NVIDIA H100 80GB GPUs, whereas our method requires only a single GPU, making it more accessible."
}