@inproceedings{luo-etal-2026-lightmoe,
    title = "{L}ight{M}o{E}: Task-Aware Expert Availability Management for Memory-Efficient {M}o{E}-{LLM} Inference",
    author = "Luo, Puhan  and
      Yao, Yunhao  and
      Wang, Junyang  and
      Zhang, Junyang  and
      Li, Xiangyang",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1062/",
    pages = "21124--21134",
    ISBN = "979-8-89176-395-1",
    abstract = "Mixture-of-Experts (MoE) models offer a promising path for scaling model capacity, yet their massive memory footprint poses significant challenges for deployment on resource-constrained edge devices. Existing solutions, such as static pruning or dynamic offloading, often struggle to balance model accuracy with inference latency due to irreversible information loss or prohibitive I/O overhead. In this paper, we propose LightMoE, a novel framework for memory-efficient MoE inference that exploits the inherent functional redundancy and temporal locality of expert activation. LightMoE employs a frequency-aware expert initialization strategy to retain a compact core of resident experts and introduces a similarity-based redirection mechanism to compensate for missing experts without incurring I/O costs. Furthermore, it incorporates a lightweight runtime manager that performs coarse-grained, task-level expert replacement to adapt to shifting data distributions. Empirical evaluations on representative edge platforms demonstrate that LightMoE achieves a superior accuracy-efficiency trade-off, improving average accuracy by 4.3{\%} over static pruning and 2.4{\%} over dynamic swapping methods, while maintaining inference latency comparable to strictly pruned models."
}