Addressing Low-Resource Scenarios with Character-aware Embeddings
Abstract
Most modern approaches to computing word embeddings assume the availability of text corpora with billions of words. In this paper, we explore a setup where only corpora with millions of words are available, and many words in any new text are out of vocabulary. This setup is both of practical interests – modeling the situation for specific domains and low-resource languages – and of psycholinguistic interest, since it corresponds much more closely to the actual experiences and challenges of human language learning and use. We compare standard skip-gram word embeddings with character-based embeddings on word relatedness prediction. Skip-grams excel on large corpora, while character-based embeddings do well on small corpora generally and rare and complex words specifically. The models can be combined easily.- Anthology ID:
- W18-1204
- Volume:
- Proceedings of the Second Workshop on Subword/Character LEvel Models
- Month:
- June
- Year:
- 2018
- Address:
- New Orleans
- Editors:
- Manaal Faruqui, Hinrich Schütze, Isabel Trancoso, Yulia Tsvetkov, Yadollah Yaghoobzadeh
- Venue:
- SCLeM
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 32–37
- Language:
- URL:
- https://aclanthology.org/W18-1204
- DOI:
- 10.18653/v1/W18-1204
- Cite (ACL):
- Sean Papay, Sebastian Padó, and Ngoc Thang Vu. 2018. Addressing Low-Resource Scenarios with Character-aware Embeddings. In Proceedings of the Second Workshop on Subword/Character LEvel Models, pages 32–37, New Orleans. Association for Computational Linguistics.
- Cite (Informal):
- Addressing Low-Resource Scenarios with Character-aware Embeddings (Papay et al., SCLeM 2018)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/W18-1204.pdf