Abstract
Embedding of rare and out-of-vocabulary (OOV) words is an important open NLP problem. A popular solution is to train a character-level neural network to reproduce the embeddings from a standard word embedding model. The trained network is then used to assign vectors to any input string, including OOV and rare words. We enhance this approach and introduce an algorithm that iteratively refines and improves both word- and character-level models. We demonstrate that our method outperforms the existing algorithms on 5 word similarity data sets, and that it can be successfully applied to job title normalization, an important problem in the e-recruitment domain that suffers from the OOV problem.- Anthology ID:
- 2020.coling-main.104
- Volume:
- Proceedings of the 28th International Conference on Computational Linguistics
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona, Spain (Online)
- Editors:
- Donia Scott, Nuria Bel, Chengqing Zong
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 1204–1213
- Language:
- URL:
- https://aclanthology.org/2020.coling-main.104
- DOI:
- 10.18653/v1/2020.coling-main.104
- Cite (ACL):
- Phong Ha, Shanshan Zhang, Nemanja Djuric, and Slobodan Vucetic. 2020. Improving Word Embeddings through Iterative Refinement of Word- and Character-level Models. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1204–1213, Barcelona, Spain (Online). International Committee on Computational Linguistics.
- Cite (Informal):
- Improving Word Embeddings through Iterative Refinement of Word- and Character-level Models (Ha et al., COLING 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2020.coling-main.104.pdf