@inproceedings{xie-etal-2025-automated,
    title = "Automated Fine-Grained Mixture-of-Experts Quantization",
    author = "Xie, Zhanhao  and
      Ma, Yuexiao  and
      Zheng, Xiawu  and
      Chao, Fei  and
      Sui, Wanchen  and
      Li, Yong  and
      Li, Shen  and
      Ji, Rongrong",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.1386/",
    pages = "27024--27037",
    ISBN = "979-8-89176-256-5",
    abstract = "The Mixture of Experts (MoE) architecture enables efficient model scaling through conditional computation, where only subset of parameters are activated per input. However, this distributed architecture poses unprecedented challenges for model compression, as conventional quantization methods optimized for dense networks prove inadequate. This paper introduces a specialized quantization framework for MoE architectures, motivated by our discovery that weight matrices across expert networks exhibit distinctive channel-wise outlier distributions, necessitating a more nuanced compression approach. Through theoretical analysis incorporating Fisher Information matrices and condition number characteristics, we establish a fundamental relationship between layer functionality and quantization sensitivity, demonstrating that down-projection layers inherently demand higher precision compared to up-projection layers. Leveraging these insights, we develop an automated channel-wise quantization framework that dynamically determines optimal bit-width allocations while maintaining minimal computational overhead through efficient statistical approximations. When evaluated on the Mixtral-8x7b-v0.1 architecture, our methodology demonstrates a 3.96{\%} improvement over existing state-of-the-art approaches across natural language understanding benchmarks, while achieving superior compression ratios."
}