@inproceedings{tian-etal-2025-espnet,
    title = "{ESP}net-{S}peech{LM}: An Open Speech Language Model Toolkit",
    author = "Tian, Jinchuan  and
      Shi, Jiatong  and
      Chen, William  and
      Arora, Siddhant  and
      Masuyama, Yoshiki  and
      Maekaku, Takashi  and
      Wu, Yihan  and
      Peng, Junyi  and
      Bharadwaj, Shikhar  and
      Zhao, Yiwen  and
      Cornell, Samuele  and
      Peng, Yifan  and
      Yue, Xiang  and
      Yang, Chao-Han Huck  and
      Neubig, Graham  and
      Watanabe, Shinji",
    editor = "Dziri, Nouha  and
      Ren, Sean (Xiang)  and
      Diao, Shizhe",
    booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (System Demonstrations)",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/landing_page/2025.naacl-demo.12/",
    pages = "116--124",
    ISBN = "979-8-89176-191-9",
    abstract = "We present ESPnet-SpeechLM, an open toolkit designed to democratize the development of speech language models (SpeechLMs) and voice-driven agentic applications. The toolkit standardizes speech processing tasks by framing them as universal sequential modeling problems, encompassing a cohesive workflow of data preprocessing, pre-training, inference, and task evaluation. With ESPnet-SpeechLM, users can easily define task templates and configure key settings, enabling seamless and streamlined SpeechLM development. The toolkit ensures flexibility, efficiency, and scalability by offering highly configurable modules for every stage of the workflow. To illustrate its capabilities, we provide multiple use cases demonstrating how competitive SpeechLMs can be constructed with ESPnet-SpeechLM, including a 1.7B-parameter model pre-trained on both text and speech tasks, across diverse benchmarks. The toolkit and its recipes are fully transparent and reproducible at: https://github.com/espnet/espnet/tree/speechlm."
}