Revisiting Locality Sensitive Hashing for Vocabulary Selection in Fast Neural Machine Translation
Hieu Hoang, Marcin Junczys-dowmunt, Roman Grundkiewicz, Huda Khayrallah
Abstract
Neural machine translation models often contain large target vocabularies. The calculation of logits, softmax and beam search is computationally costly over so many classes. We investigate the use of locality sensitive hashing (LSH) to reduce the number of vocabulary items that must be evaluated and explore the relationship between the hashing algorithm, translation speed and quality. Compared to prior work, our LSH-based solution does not require additional augmentation via word-frequency lists or alignments. We propose a training procedure that produces models, which, when combined with our LSH inference algorithm increase translation speed by up to 87% over the baseline, while maintaining translation quality as measured by BLEU. Apart from just using BLEU, we focus on minimizing search errors compared to the full softmax, a much harsher quality criterion.- Anthology ID:
- 2022.wmt-1.79
- Volume:
- Proceedings of the Seventh Conference on Machine Translation (WMT)
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates (Hybrid)
- Editors:
- Philipp Koehn, Loïc Barrault, Ondřej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Tom Kocmi, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Marco Turchi, Marcos Zampieri
- Venue:
- WMT
- SIG:
- SIGMT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 855–869
- Language:
- URL:
- https://aclanthology.org/2022.wmt-1.79
- DOI:
- Cite (ACL):
- Hieu Hoang, Marcin Junczys-dowmunt, Roman Grundkiewicz, and Huda Khayrallah. 2022. Revisiting Locality Sensitive Hashing for Vocabulary Selection in Fast Neural Machine Translation. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 855–869, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
- Cite (Informal):
- Revisiting Locality Sensitive Hashing for Vocabulary Selection in Fast Neural Machine Translation (Hoang et al., WMT 2022)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2022.wmt-1.79.pdf