Abstract
Unsupervised learning of word representations involves capturing the contextual information surrounding word occurrences, which can be grounded in the observation that word form is largely disconnected from word meaning. While there are fewer reasons to believe that the same holds for sentences, learning through context has been carried over to learning representations of word sequences. However, this work pays minimal to no attention to the role of context in inferring sentence representations. In this article, we present a dialog act tag probing task designed to explicitly compare content-, and context-oriented sentence representations inferred on utterances of telephone conversations (SwDA). Our results suggest that there is no clear benefit of context-based sentence representations over content-based sentence representations. However, there is a very clear benefit of increasing the dimensionality of the sentence vectors in nearly all approaches.- Anthology ID:
- 2023.findings-emnlp.588
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 8784–8792
- Language:
- URL:
- https://aclanthology.org/2023.findings-emnlp.588
- DOI:
- 10.18653/v1/2023.findings-emnlp.588
- Cite (ACL):
- Rastislav Hronsky and Emmanuel Keuleers. 2023. Role of Context in Unsupervised Sentence Representation Learning: the Case of Dialog Act Modeling. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8784–8792, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Role of Context in Unsupervised Sentence Representation Learning: the Case of Dialog Act Modeling (Hronsky & Keuleers, Findings 2023)
- PDF:
- https://preview.aclanthology.org/aacl-23-doi-ingestion/2023.findings-emnlp.588.pdf