Abstract
Accurately and efficiently estimating word similarities from text is fundamental in natural language processing. In this paper, we propose a fast and lightweight method for estimating similarities from streams by explicitly counting second-order co-occurrences. The method rests on the observation that words that are highly correlated with respect to such counts are also highly similar with respect to first-order co-occurrences. Using buffers of co-occurred words per word to count second-order co-occurrences, we can then estimate similarities in a single pass over data without having to do prohibitively expensive similarity calculations. We demonstrate that this approach is scalable, converges rapidly, behaves robustly under parameter changes, and that it captures word similarities on par with those given by state-of-the-art word embeddings.- Anthology ID:
- D18-1172
- Volume:
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Month:
- October-November
- Year:
- 2018
- Address:
- Brussels, Belgium
- Editors:
- Ellen Riloff, David Chiang, Julia Hockenmaier, Jun’ichi Tsujii
- Venue:
- EMNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1425–1434
- Language:
- URL:
- https://aclanthology.org/D18-1172
- DOI:
- 10.18653/v1/D18-1172
- Cite (ACL):
- Olof Görnerup and Daniel Gillblad. 2018. Streaming word similarity mining on the cheap. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1425–1434, Brussels, Belgium. Association for Computational Linguistics.
- Cite (Informal):
- Streaming word similarity mining on the cheap (Görnerup & Gillblad, EMNLP 2018)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/D18-1172.pdf