Abstract
Most work in NLP makes the assumption that it is desirable to develop solutions in the native language in question. There is consequently a strong trend towards building native language models even for low-resource languages. This paper questions this development, and explores the idea of simply translating the data into English, thereby enabling the use of pretrained, and large-scale, English language models. We demonstrate empirically that a large English language model coupled with modern machine translation outperforms native language models in most Scandinavian languages. The exception to this is Finnish, which we assume is due to inferior translation quality. Our results suggest that machine translation is a mature technology, which raises a serious counter-argument for training native language models for low-resource languages. This paper therefore strives to make a provocative but important point. As English language models are improving at an unprecedented pace, which in turn improves machine translation, it is from an empirical and environmental stand-point more effective to translate data from low-resource languages into English, than to build language models for such languages.- Anthology ID:
- 2021.nodalida-main.42
- Volume:
- Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa)
- Month:
- May 31--2 June
- Year:
- 2021
- Address:
- Reykjavik, Iceland (Online)
- Editors:
- Simon Dobnik, Lilja Øvrelid
- Venue:
- NoDaLiDa
- SIG:
- Publisher:
- Linköping University Electronic Press, Sweden
- Note:
- Pages:
- 385–390
- Language:
- URL:
- https://aclanthology.org/2021.nodalida-main.42
- DOI:
- Cite (ACL):
- Tim Isbister, Fredrik Carlsson, and Magnus Sahlgren. 2021. Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 385–390, Reykjavik, Iceland (Online). Linköping University Electronic Press, Sweden.
- Cite (Informal):
- Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead? (Isbister et al., NoDaLiDa 2021)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2021.nodalida-main.42.pdf
- Code
- timpal0l/ScandiSent