OpusDistillery: A Configurable End-to-End Pipeline for Systematic Multilingual Distillation of Open NMT Models
Ona de Gibert, Tommi Nieminen, Yves Scherrer, Jörg Tiedemann
Abstract
In this work, we introduce OpusDistillery, a novel framework to streamline the Knowledge Distillation (KD) process of multilingual NMT models. OpusDistillery’s main features are the integration of openly available teacher models from OPUS-MT and Hugging Face, comprehensive multilingual support and robust GPU utilization tracking. We describe the tool in detail and discuss the individual contributions of its pipeline components, demonstrating its flexibility for different use cases. OpusDistillery is open-source and released under a permissive license, aiming to facilitate further research and development in the field of multilingual KD for any sequence-to-sequence task. Our code is available at https://github.com/Helsinki-NLP/OpusDistillery.- Anthology ID:
- 2025.nodalida-1.20
- Volume:
- Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)
- Month:
- march
- Year:
- 2025
- Address:
- Tallinn, Estonia
- Editors:
- Richard Johansson, Sara Stymne
- Venue:
- NoDaLiDa
- SIG:
- Publisher:
- University of Tartu Library
- Note:
- Pages:
- 201–208
- Language:
- URL:
- https://preview.aclanthology.org/fix-sig-urls/2025.nodalida-1.20/
- DOI:
- Cite (ACL):
- Ona de Gibert, Tommi Nieminen, Yves Scherrer, and Jörg Tiedemann. 2025. OpusDistillery: A Configurable End-to-End Pipeline for Systematic Multilingual Distillation of Open NMT Models. In Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025), pages 201–208, Tallinn, Estonia. University of Tartu Library.
- Cite (Informal):
- OpusDistillery: A Configurable End-to-End Pipeline for Systematic Multilingual Distillation of Open NMT Models (Gibert et al., NoDaLiDa 2025)
- PDF:
- https://preview.aclanthology.org/fix-sig-urls/2025.nodalida-1.20.pdf