Improving Distributional Similarity with Lessons Learned from Word Embeddings

Omer Levy, Yoav Goldberg, Ido Dagan


Abstract
Recent trends suggest that neural-network-inspired word embedding models outperform traditional count-based distributional models on word similarity and analogy detection tasks. We reveal that much of the performance gains of word embeddings are due to certain system design choices and hyperparameter optimizations, rather than the embedding algorithms themselves. Furthermore, we show that these modifications can be transferred to traditional distributional models, yielding similar gains. In contrast to prior reports, we observe mostly local or insignificant performance differences between the methods, with no global advantage to any single approach over the others.
Anthology ID:
Q15-1016
Volume:
Transactions of the Association for Computational Linguistics, Volume 3
Month:
Year:
2015
Address:
Cambridge, MA
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
211–225
Language:
URL:
https://aclanthology.org/Q15-1016
DOI:
10.1162/tacl_a_00134
Bibkey:
Cite (ACL):
Omer Levy, Yoav Goldberg, and Ido Dagan. 2015. Improving Distributional Similarity with Lessons Learned from Word Embeddings. Transactions of the Association for Computational Linguistics, 3:211–225.
Cite (Informal):
Improving Distributional Similarity with Lessons Learned from Word Embeddings (Levy et al., TACL 2015)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/Q15-1016.pdf