Words are Vectors, Dependencies are Matrices: Learning Word Embeddings from Dependency Graphs

Paula Czarnowska, Guy Emerson, Ann Copestake


Abstract
Distributional Semantic Models (DSMs) construct vector representations of word meanings based on their contexts. Typically, the contexts of a word are defined as its closest neighbours, but they can also be retrieved from its syntactic dependency relations. In this work, we propose a new dependency-based DSM. The novelty of our model lies in associating an independent meaning representation, a matrix, with each dependency-label. This allows it to capture specifics of the relations between words and contexts, leading to good performance on both intrinsic and extrinsic evaluation tasks. In addition to that, our model has an inherent ability to represent dependency chains as products of matrices which provides a straightforward way of handling further contexts of a word.
Anthology ID:
W19-0408
Volume:
Proceedings of the 13th International Conference on Computational Semantics - Long Papers
Month:
May
Year:
2019
Address:
Gothenburg, Sweden
Venues:
IWCS | WS
SIG:
SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
91–102
Language:
URL:
https://aclanthology.org/W19-0408
DOI:
10.18653/v1/W19-0408
Bibkey:
Cite (ACL):
Paula Czarnowska, Guy Emerson, and Ann Copestake. 2019. Words are Vectors, Dependencies are Matrices: Learning Word Embeddings from Dependency Graphs. In Proceedings of the 13th International Conference on Computational Semantics - Long Papers, pages 91–102, Gothenburg, Sweden. Association for Computational Linguistics.
Cite (Informal):
Words are Vectors, Dependencies are Matrices: Learning Word Embeddings from Dependency Graphs (Czarnowska et al., 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/W19-0408.pdf