Abstract
We present a novel retrofitting model that can leverage relational knowledge available in a knowledge resource to improve word embeddings. The knowledge is captured in terms of relation inequality constraints that compare similarity of related and unrelated entities in the context of an anchor entity. These constraints are used as training data to learn a non-linear transformation function that maps original word vectors to a vector space respecting these constraints. The transformation function is learned in a similarity metric learning setting using Triplet network architecture. We applied our model to synonymy, antonymy and hypernymy relations in WordNet and observed large gains in performance over original distributional models as well as other retrofitting approaches on word similarity task and significant overall improvement on lexical entailment detection task.- Anthology ID:
- 2020.coling-main.111
- Volume:
- Proceedings of the 28th International Conference on Computational Linguistics
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona, Spain (Online)
- Editors:
- Donia Scott, Nuria Bel, Chengqing Zong
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 1292–1298
- Language:
- URL:
- https://aclanthology.org/2020.coling-main.111
- DOI:
- 10.18653/v1/2020.coling-main.111
- Cite (ACL):
- Sapan Shah, Sreedhar Reddy, and Pushpak Bhattacharyya. 2020. A Retrofitting Model for Incorporating Semantic Relations into Word Embeddings. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1292–1298, Barcelona, Spain (Online). International Committee on Computational Linguistics.
- Cite (Informal):
- A Retrofitting Model for Incorporating Semantic Relations into Word Embeddings (Shah et al., COLING 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2020.coling-main.111.pdf