Abstract
We present disambiguated skip-gram: a neural-probabilistic model for learning multi-sense distributed representations of words. Disambiguated skip-gram jointly estimates a skip-gram-like context word prediction model and a word sense disambiguation model. Unlike previous probabilistic models for learning multi-sense word embeddings, disambiguated skip-gram is end-to-end differentiable and can be interpreted as a simple feed-forward neural network. We also introduce an effective pruning strategy for the embeddings learned by disambiguated skip-gram. This allows us to control the granularity of representations learned by our model. In experimental evaluation disambiguated skip-gram improves state-of-the are results in several word sense induction benchmarks.- Anthology ID:
- D18-1174
- Volume:
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Month:
- October-November
- Year:
- 2018
- Address:
- Brussels, Belgium
- Venue:
- EMNLP
- SIG:
- SIGDAT
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1445–1454
- Language:
- URL:
- https://aclanthology.org/D18-1174
- DOI:
- 10.18653/v1/D18-1174
- Cite (ACL):
- Karol Grzegorczyk and Marcin Kurdziel. 2018. Disambiguated skip-gram model. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1445–1454, Brussels, Belgium. Association for Computational Linguistics.
- Cite (Informal):
- Disambiguated skip-gram model (Grzegorczyk & Kurdziel, EMNLP 2018)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/D18-1174.pdf