OpenSeq2Seq: Extensible Toolkit for Distributed and Mixed Precision Training of Sequence-to-Sequence Models
Oleksii Kuchaiev, Boris Ginsburg, Igor Gitman, Vitaly Lavrukhin, Carl Case, Paulius Micikevicius
Abstract
We present OpenSeq2Seq – an open-source toolkit for training sequence-to-sequence models. The main goal of our toolkit is to allow researchers to most effectively explore different sequence-to-sequence architectures. The efficiency is achieved by fully supporting distributed and mixed-precision training. OpenSeq2Seq provides building blocks for training encoder-decoder models for neural machine translation and automatic speech recognition. We plan to extend it with other modalities in the future.- Anthology ID:
- W18-2507
- Volume:
- Proceedings of Workshop for NLP Open Source Software (NLP-OSS)
- Month:
- July
- Year:
- 2018
- Address:
- Melbourne, Australia
- Editors:
- Eunjeong L. Park, Masato Hagiwara, Dmitrijs Milajevs, Liling Tan
- Venue:
- NLPOSS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 41–46
- Language:
- URL:
- https://aclanthology.org/W18-2507
- DOI:
- 10.18653/v1/W18-2507
- Cite (ACL):
- Oleksii Kuchaiev, Boris Ginsburg, Igor Gitman, Vitaly Lavrukhin, Carl Case, and Paulius Micikevicius. 2018. OpenSeq2Seq: Extensible Toolkit for Distributed and Mixed Precision Training of Sequence-to-Sequence Models. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 41–46, Melbourne, Australia. Association for Computational Linguistics.
- Cite (Informal):
- OpenSeq2Seq: Extensible Toolkit for Distributed and Mixed Precision Training of Sequence-to-Sequence Models (Kuchaiev et al., NLPOSS 2018)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/W18-2507.pdf