The Importance of Context in the Evaluation of Word Embeddings: The Effects of Antonymy and Polysemy
Abstract
Word embeddings are widely used for diverse applications in natural language processing. Despite extensive research, it is unclear when they succeed or fail to capture human judgements of semantic relatedness and similarity. In this study, we examine a range of models and experimental datasets, showing that while current embeddings perform reasonably well overall, they are unable to account for human judgements of antonyms and polysemy. We suggest that word embeddings perform poorly in representing polysemy and antonymy because they do not consider the context in which humans make word similarity judgements. In support of this, we further show that incorporating additional context into transformer embeddings using general corpora and lexical dictionaries significantly improves the fit with human judgments. Our results provide insight into two key inadequacies of word embeddings, and highlight the importance of incorporating word context into representations of word meaning when accounting for context-free human similarity judgments.- Anthology ID:
- 2023.iwcs-1.17
- Volume:
- Proceedings of the 15th International Conference on Computational Semantics
- Month:
- June
- Year:
- 2023
- Address:
- Nancy, France
- Editors:
- Maxime Amblard, Ellen Breitholtz
- Venue:
- IWCS
- SIG:
- SIGSEM
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 155–172
- Language:
- URL:
- https://aclanthology.org/2023.iwcs-1.17
- DOI:
- Cite (ACL):
- James Fodor, Simon De Deyne, and Shinsuke Suzuki. 2023. The Importance of Context in the Evaluation of Word Embeddings: The Effects of Antonymy and Polysemy. In Proceedings of the 15th International Conference on Computational Semantics, pages 155–172, Nancy, France. Association for Computational Linguistics.
- Cite (Informal):
- The Importance of Context in the Evaluation of Word Embeddings: The Effects of Antonymy and Polysemy (Fodor et al., IWCS 2023)
- PDF:
- https://preview.aclanthology.org/revert-3132-ingestion-checklist/2023.iwcs-1.17.pdf