Evaluating bilingual word embeddings on the long tail
Fabienne Braune, Viktor Hangya, Tobias Eder, Alexander Fraser
Abstract
Bilingual word embeddings are useful for bilingual lexicon induction, the task of mining translations of given words. Many studies have shown that bilingual word embeddings perform well for bilingual lexicon induction but they focused on frequent words in general domains. For many applications, bilingual lexicon induction of rare and domain-specific words is of critical importance. Therefore, we design a new task to evaluate bilingual word embeddings on rare words in different domains. We show that state-of-the-art approaches fail on this task and present simple new techniques to improve bilingual word embeddings for mining rare words. We release new gold standard datasets and code to stimulate research on this task.- Anthology ID:
- N18-2030
- Volume:
- Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)
- Month:
- June
- Year:
- 2018
- Address:
- New Orleans, Louisiana
- Venue:
- NAACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 188–193
- Language:
- URL:
- https://aclanthology.org/N18-2030
- DOI:
- 10.18653/v1/N18-2030
- Cite (ACL):
- Fabienne Braune, Viktor Hangya, Tobias Eder, and Alexander Fraser. 2018. Evaluating bilingual word embeddings on the long tail. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 188–193, New Orleans, Louisiana. Association for Computational Linguistics.
- Cite (Informal):
- Evaluating bilingual word embeddings on the long tail (Braune et al., NAACL 2018)
- PDF:
- https://preview.aclanthology.org/remove-xml-comments/N18-2030.pdf
- Code
- braunefe/BWEeval