@inproceedings{behdin-etal-2025-scaling,
    title = "Scaling Down, Serving Fast: Compressing and Deploying Efficient {LLM}s for Recommendation Systems",
    author = "Behdin, Kayhan  and
      Fatahibaarzi, Ata  and
      Song, Qingquan  and
      Dai, Yun  and
      Gupta, Aman  and
      Wang, Zhipeng  and
      Sang, Hejian  and
      Tang, Shao  and
      Dexter, Gregory  and
      Zhu, Sirou  and
      Zhu, Siyu  and
      Dharamsi, Tejas  and
      Kothapalli, Vignesh  and
      Fu, Zhoutong  and
      Cao, Yihan  and
      Hsu, Pin-Lun  and
      Borisyuk, Fedor  and
      Pillai, Natesh S.  and
      Simon, Luke  and
      Mazumder, Rahul",
    editor = "Potdar, Saloni  and
      Rojas-Barahona, Lina  and
      Montella, Sebastien",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track",
    month = nov,
    year = "2025",
    address = "Suzhou (China)",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-industry.119/",
    pages = "1687--1702",
    ISBN = "979-8-89176-333-3",
    abstract = "Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications, from search and recommendation systems to generative tasks. Although scaling laws indicate that larger models generally yield better generalization and performance, their substantial computational requirements often render them impractical for many real-world scenarios at scale. In this paper, we present a comprehensive set of insights for training and deploying small language models (SLMs) that deliver high performance for a variety of industry use cases. We focus on two key techniques: (1) knowledge distillation and (2) model compression via structured pruning and quantization. These approaches enable SLMs to retain much of the quality of their larger counterparts while significantly reducing training/serving costs and latency. We detail the impact of these techniques on a variety of use cases in a large professional social network platform and share deployment lessons, including hardware optimization strategies that improve speed and throughput for both predictive and reasoning-based applications in Recommendation Systems."
}