Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations
Aditi Chaudhary, Chunting Zhou, Lori Levin, Graham Neubig, David R. Mortensen, Jaime Carbonell
Abstract
Much work in Natural Language Processing (NLP) has been for resource-rich languages, making generalization to new, less-resourced languages challenging. We present two approaches for improving generalization to low-resourced languages by adapting continuous word representations using linguistically motivated subword units: phonemes, morphemes and graphemes. Our method requires neither parallel corpora nor bilingual dictionaries and provides a significant gain in performance over previous methods relying on these resources. We demonstrate the effectiveness of our approaches on Named Entity Recognition for four languages, namely Uyghur, Turkish, Bengali and Hindi, of which Uyghur and Bengali are low resource languages, and also perform experiments on Machine Translation. Exploiting subwords with transfer learning gives us a boost of +15.2 NER F1 for Uyghur and +9.7 F1 for Bengali. We also show improvements in the monolingual setting where we achieve (avg.) +3 F1 and (avg.) +1.35 BLEU.- Anthology ID:
- D18-1366
- Volume:
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Month:
- October-November
- Year:
- 2018
- Address:
- Brussels, Belgium
- Editors:
- Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
- Venue:
- EMNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3285–3295
- Language:
- URL:
- https://aclanthology.org/D18-1366
- DOI:
- 10.18653/v1/D18-1366
- Cite (ACL):
- Aditi Chaudhary, Chunting Zhou, Lori Levin, Graham Neubig, David R. Mortensen, and Jaime Carbonell. 2018. Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3285–3295, Brussels, Belgium. Association for Computational Linguistics.
- Cite (Informal):
- Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations (Chaudhary et al., EMNLP 2018)
- PDF:
- https://preview.aclanthology.org/naacl24-info/D18-1366.pdf
- Code
- Aditi138/Embeddings