Abstract
Aspect, a linguistic category describing how actions and events unfold over time, is traditionally characterized by three semantic properties: stativity, durativity and telicity. In this study, we investigate whether and to what extent these properties are encoded in the verb token embeddings of the contextualized spaces of two English language models – BERT and GPT-2. First, we propose an experiment using semantic projections to examine whether the values of the vector dimensions of annotated verbs for stativity, durativity and telicity reflect human linguistic distinctions. Second, we use distributional similarity to replicate the notorious Imperfective Paradox described by Dowty (1977), and assess whether the embedding models are sensitive to capture contextual nuances of the verb telicity. Our results show that both models encode the semantic distinctions for the aspect properties of stativity and telicity in most of their layers, while durativity is the most challenging feature. As for the Imperfective Paradox, only the embedding similarities computed with the vectors from the early layers of the BERT model align with the expected pattern.- Anthology ID:
- 2024.starsem-1.7
- Volume:
- Proceedings of the 13th Joint Conference on Lexical and Computational Semantics (*SEM 2024)
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Danushka Bollegala, Vered Shwartz
- Venue:
- *SEM
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 80–92
- Language:
- URL:
- https://aclanthology.org/2024.starsem-1.7
- DOI:
- Cite (ACL):
- Yuxi Li, Emmanuele Chersoni, and Yu-Yin Hsu. 2024. Investigating Aspect Features in Contextualized Embeddings with Semantic Scales and Distributional Similarity. In Proceedings of the 13th Joint Conference on Lexical and Computational Semantics (*SEM 2024), pages 80–92, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- Investigating Aspect Features in Contextualized Embeddings with Semantic Scales and Distributional Similarity (Li et al., *SEM 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.starsem-1.7.pdf