Training Word Sense Embeddings With Lexicon-based Regularization

Luis Nieto-Piña, Richard Johansson


Abstract
We propose to improve word sense embeddings by enriching an automatic corpus-based method with lexicographic data. Information from a lexicon is introduced into the learning algorithm’s objective function through a regularizer. The incorporation of lexicographic data yields embeddings that are able to reflect expert-defined word senses, while retaining the robustness, high quality, and coverage of automatic corpus-based methods. These properties are observed in a manual inspection of the semantic clusters that different degrees of regularizer strength create in the vector space. Moreover, we evaluate the sense embeddings in two downstream applications: word sense disambiguation and semantic frame prediction, where they outperform simpler approaches. Our results show that a corpus-based model balanced with lexicographic data learns better representations and improve their performance in downstream tasks.
Anthology ID:
I17-1029
Volume:
Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
November
Year:
2017
Address:
Taipei, Taiwan
Venue:
IJCNLP
SIG:
Publisher:
Asian Federation of Natural Language Processing
Note:
Pages:
284–294
Language:
URL:
https://aclanthology.org/I17-1029
DOI:
Bibkey:
Cite (ACL):
Luis Nieto-Piña and Richard Johansson. 2017. Training Word Sense Embeddings With Lexicon-based Regularization. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 284–294, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Cite (Informal):
Training Word Sense Embeddings With Lexicon-based Regularization (Nieto-Piña & Johansson, IJCNLP 2017)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/I17-1029.pdf