Streaming word similarity mining on the cheap

Olof Görnerup, Daniel Gillblad


Abstract
Accurately and efficiently estimating word similarities from text is fundamental in natural language processing. In this paper, we propose a fast and lightweight method for estimating similarities from streams by explicitly counting second-order co-occurrences. The method rests on the observation that words that are highly correlated with respect to such counts are also highly similar with respect to first-order co-occurrences. Using buffers of co-occurred words per word to count second-order co-occurrences, we can then estimate similarities in a single pass over data without having to do prohibitively expensive similarity calculations. We demonstrate that this approach is scalable, converges rapidly, behaves robustly under parameter changes, and that it captures word similarities on par with those given by state-of-the-art word embeddings.
Anthology ID:
D18-1172
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1425–1434
Language:
URL:
https://aclanthology.org/D18-1172
DOI:
10.18653/v1/D18-1172
Bibkey:
Cite (ACL):
Olof Görnerup and Daniel Gillblad. 2018. Streaming word similarity mining on the cheap. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1425–1434, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
Streaming word similarity mining on the cheap (Görnerup & Gillblad, EMNLP 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/D18-1172.pdf