@inproceedings{nguyen-etal-2025-survey,
    title = "A Survey on Small Language Models",
    author = "Nguyen, Chien Van  and
      Shen, Xuan  and
      Aponte, Ryan  and
      Xia, Yu  and
      Basu, Samyadeep  and
      Hu, Zhengmian  and
      Chen, Jian  and
      Parmar, Mihir  and
      Kunapuli, Sasidhar  and
      Barrow3, Joe  and
      Wu, Junda  and
      Singh, Ashish  and
      Wang, Yu  and
      Gu, Jiuxiang  and
      K. Ahmed, Nesreen  and
      Lipka, Nedim  and
      Zhang, Ruiyi  and
      Chen, Xiang  and
      Yu, Tong  and
      Kim, Sungchul  and
      Deilamsalehy, Hanieh  and
      Park, Namyong  and
      Rimer, Michael  and
      Zhang, Zhehao  and
      Yang, Huanrui  and
      Mathur, Puneet  and
      Wu, Gang  and
      Dernoncourt, Franck  and
      Rossi, Ryan  and
      Nguyen, Thien Huu",
    editor = "Angelova, Galia  and
      Kunilovskaya, Maria  and
      Escribe, Marie  and
      Mitkov, Ruslan",
    booktitle = "Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI Era",
    month = sep,
    year = "2025",
    address = "Varna, Bulgaria",
    publisher = "INCOMA Ltd., Shoumen, Bulgaria",
    url = "https://preview.aclanthology.org/corrections-2026-01/2025.ranlp-1.93/",
    pages = "807--821",
    abstract = "Small Language Models (SLMs) have become increasingly important due to their efficiency and performance to perform various language tasks with minimal computational resources, making them ideal for various settings including on-device, mobile, edge devices, among many others. In this article, we present a comprehensive survey on SLMs, focusing on their architectures, training techniques, and model compression techniques. We propose a novel taxonomy for categorizing the methods used to optimize SLMs, including model compression, pruning, and quantization techniques. We summarize the benchmark datasets that are useful for benchmarking SLMs along with the evaluation metrics commonly used. Additionally, we highlight key open challenges that remain to be addressed. Our survey aims to serve as a valuable resource for researchers and practitioners interested in developing and deploying small yet efficient language models."
}