Scalable and Interpretable Semantic Change Detection

Syrielle Montariol, Matej Martinc, Lidia Pivovarova


Abstract
Several cluster-based methods for semantic change detection with contextual embeddings emerged recently. They allow a fine-grained analysis of word use change by aggregating embeddings into clusters that reflect the different usages of the word. However, these methods are unscalable in terms of memory consumption and computation time. Therefore, they require a limited set of target words to be picked in advance. This drastically limits the usability of these methods in open exploratory tasks, where each word from the vocabulary can be considered as a potential target. We propose a novel scalable method for word usage-change detection that offers large gains in processing time and significant memory savings while offering the same interpretability and better performance than unscalable methods. We demonstrate the applicability of the proposed method by analysing a large corpus of news articles about COVID-19.
Anthology ID:
2021.naacl-main.369
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4642–4652
Language:
URL:
https://aclanthology.org/2021.naacl-main.369
DOI:
10.18653/v1/2021.naacl-main.369
Bibkey:
Cite (ACL):
Syrielle Montariol, Matej Martinc, and Lidia Pivovarova. 2021. Scalable and Interpretable Semantic Change Detection. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4642–4652, Online. Association for Computational Linguistics.
Cite (Informal):
Scalable and Interpretable Semantic Change Detection (Montariol et al., NAACL 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/2021.naacl-main.369.pdf
Code
 matejmartinc/scalable_semantic_shift