@inproceedings{huber-etal-2026-cosmoes,
    title = "{C}o{SM}o{E}s: Compact Sparse Mixture of Experts",
    author = "Huber, Patrick  and
      Shrivastava, Akshat  and
      Chang, Ernie  and
      Sankar, Chinnadhurai  and
      Aly, Ahmed A  and
      Sagar, Adithya",
    editor = "Yan, Qianqi  and
      Montariol, Syrielle  and
      Fan, Yue  and
      Gu, Jing  and
      Pan, Jiayi  and
      Li, Manling  and
      Kordjamshidi, Parisa  and
      Suhr, Alane  and
      Wang, Xin Eric",
    booktitle = "Proceedings of the 4th Workshop on Advances in Language and Vision Research ({ALVR})",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.alvr-main.4/",
    pages = "46--56",
    ISBN = "979-8-89176-398-2",
    abstract = "Sparse Mixture of Expert (MoE) models are widely used foundation architectures at large scale, yet remain under-explored at smaller sizes. In this work, we introduce Compact Sparse Mixture of Experts (CoSMoEs) for on-device inference, addressing three key challenges: Quality, Memory, and Latency. On the quality front, we conduct a fair evaluation (removing confounding factors) and show that MoE architectures outperform dense models at on-device scale. We further propose weight-decomposed experts, which improve MoE performance beyond the standard formulation. On the memory and latency front, we address the prohibitively large parameter count of MoE models by improving expert offloading efficiency through a novel training-time loss, reducing inference latency for on-device deployment"
}