Abstract
The similarity of representations is crucial for WSD. However, a lot of information is encoded in the contextualized representations, and it is not clear which sentence context features drive this similarity and whether these features are significant to WSD. In this study, we address these questions. First, we identify the sentence context features that are responsible for the similarity of the contextualized representations of different occurrences of words. For this purpose, we conduct an explainability experiment and identify the sentence context features that lead to the formation of the clusters in word sense clustering with CWEs. Then, we provide a qualitative evaluation for assessing the significance of these features to WSD. Our results show that features that lack significance to WSD determine the similarity of the representations even when different senses of a word occur in highly diverse contexts and sentence context provides clear clues for different senses.- Anthology ID:
- 2024.unimplicit-1.4
- Volume:
- Proceedings of the Third Workshop on Understanding Implicit and Underspecified Language
- Month:
- March
- Year:
- 2024
- Address:
- Malta
- Editors:
- Valentina Pyatkin, Daniel Fried, Elias Stengel-Eskin, Alisa Liu, Sandro Pezzelle
- Venues:
- unimplicit | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 42–53
- Language:
- URL:
- https://aclanthology.org/2024.unimplicit-1.4
- DOI:
- Cite (ACL):
- Deniz Ekin Yavas. 2024. Assessing the Significance of Encoded Information in Contextualized Representations to Word Sense Disambiguation. In Proceedings of the Third Workshop on Understanding Implicit and Underspecified Language, pages 42–53, Malta. Association for Computational Linguistics.
- Cite (Informal):
- Assessing the Significance of Encoded Information in Contextualized Representations to Word Sense Disambiguation (Yavas, unimplicit-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.unimplicit-1.4.pdf