Using Multi-Sense Vector Embeddings for Reverse Dictionaries

Michael A. Hedderich, Andrew Yates, Dietrich Klakow, Gerard de Melo

Abstract
Popular word embedding methods such as word2vec and GloVe assign a single vector representation to each word, even if a word has multiple distinct meanings. Multi-sense embeddings instead provide different vectors for each sense of a word. However, they typically cannot serve as a drop-in replacement for conventional single-sense embeddings, because the correct sense vector needs to be selected for each word. In this work, we study the effect of multi-sense embeddings on the task of reverse dictionaries. We propose a technique to easily integrate them into an existing neural network architecture using an attention mechanism. Our experiments demonstrate that large improvements can be obtained when employing multi-sense embeddings both in the input sequence as well as for the target representation. An analysis of the sense distributions and of the learned attention is provided as well.
Anthology ID:
W19-0421
Volume:
Proceedings of the 13th International Conference on Computational Semantics - Long Papers
Month:
May
Year:
2019
Address:
Gothenburg, Sweden
Editors:
Simon Dobnik, Stergios Chatzikyriakidis, Vera Demberg
Venue:
IWCS
SIG:
SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
247–258
Language:
URL:
https://aclanthology.org/W19-0421
DOI:
10.18653/v1/W19-0421
Bibkey:
Cite (ACL):
Michael A. Hedderich, Andrew Yates, Dietrich Klakow, and Gerard de Melo. 2019. Using Multi-Sense Vector Embeddings for Reverse Dictionaries. In Proceedings of the 13th International Conference on Computational Semantics - Long Papers, pages 247–258, Gothenburg, Sweden. Association for Computational Linguistics.
Cite (Informal):
Using Multi-Sense Vector Embeddings for Reverse Dictionaries (Hedderich et al., IWCS 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/W19-0421.pdf
Code
 uds-lsv/Multi-Sense-Embeddings-Reverse-Dictionaries
PDF Search Code