Abstract
Most word embedding algorithms such as word2vec or fastText construct two sort of vectors: for words and for contexts. Naive use of vectors of only one sort leads to poor results. We suggest using indefinite inner product in skip-gram negative sampling algorithm. This allows us to use only one sort of vectors without loss of quality. Our “context-free” cf algorithm performs on par with SGNS on word similarity datasets- Anthology ID:
- W19-4329
- Volume:
- Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
- Month:
- August
- Year:
- 2019
- Address:
- Florence, Italy
- Venue:
- RepL4NLP
- SIG:
- SIGREP
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 244–249
- Language:
- URL:
- https://aclanthology.org/W19-4329
- DOI:
- 10.18653/v1/W19-4329
- Cite (ACL):
- Alexey Zobnin and Evgenia Elistratova. 2019. Learning Word Embeddings without Context Vectors. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 244–249, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Learning Word Embeddings without Context Vectors (Zobnin & Elistratova, RepL4NLP 2019)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/W19-4329.pdf