@inproceedings{lu-etal-2025-demystifying,
    title = "Demystifying Small Language Models for Edge Deployment",
    author = "Lu, Zhenyan  and
      Li, Xiang  and
      Cai, Dongqi  and
      Yi, Rongjie  and
      Liu, Fangming  and
      Liu, Wei  and
      Luan, Jian  and
      Zhang, Xiwen  and
      Lane, Nicholas D.  and
      Xu, Mengwei",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.718/",
    pages = "14747--14764",
    ISBN = "979-8-89176-251-0",
    abstract = "Small language models (SLMs) have emerged as a promising solution for deploying resource-constrained devices, such as smartphones and Web of Things. This work presents the first comprehensive study of over 60 SLMs such as Microsoft Phi and Google Gemma that are publicly accessible. Our findings show that state-of-the-art SLMs outperform 7B models in general tasks, proving their practical viability. However, SLMs' in-context learning capabilities remain limited, and their efficiency has significant optimization potential. We identify key SLM optimization opportunities, including dynamic task-specific routing, model-hardware co-design, and vocabulary/KV cache compression. Overall, we expect the work to reveal an all-sided landscape of SLMs, benefiting the research community across algorithm, model, system, and hardware levels."
}