A Comparison of Context-sensitive Models for Lexical Substitution

Aina Garí Soler, Anne Cocos, Marianna Apidianaki, Chris Callison-Burch

Abstract
Word embedding representations provide good estimates of word meaning and give state-of-the art performance in semantic tasks. Embedding approaches differ as to whether and how they account for the context surrounding a word. We present a comparison of different word and context representations on the task of proposing substitutes for a target word in context (lexical substitution). We also experiment with tuning contextualized word embeddings on a dataset of sense-specific instances for each target word. We show that powerful contextualized word representations, which give high performance in several semantics-related tasks, deal less well with the subtle in-context similarity relationships needed for substitution. This is better handled by models trained with this objective in mind, where the inter-dependence between word and context representations is explicitly modeled during training.
Anthology ID:
W19-0423
Volume:
Proceedings of the 13th International Conference on Computational Semantics - Long Papers
Month:
May
Year:
2019
Address:
Gothenburg, Sweden
Editors:
Simon Dobnik, Stergios Chatzikyriakidis, Vera Demberg
Venue:
IWCS
SIG:
SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
271–282
Language:
URL:
https://preview.aclanthology.org/icon-24-ingestion/W19-0423/
DOI:
10.18653/v1/W19-0423
Bibkey:
Cite (ACL):
Aina Garí Soler, Anne Cocos, Marianna Apidianaki, and Chris Callison-Burch. 2019. A Comparison of Context-sensitive Models for Lexical Substitution. In Proceedings of the 13th International Conference on Computational Semantics - Long Papers, pages 271–282, Gothenburg, Sweden. Association for Computational Linguistics.
Cite (Informal):
A Comparison of Context-sensitive Models for Lexical Substitution (Garí Soler et al., IWCS 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/icon-24-ingestion/W19-0423.pdf
PDF Search Fix data