A larger-scale evaluation resource of terms and their shift direction for diachronic lexical semantics
Astrid van Aggelen, Antske Fokkens, Laura Hollink, Jacco van Ossenbruggen
Abstract
Determining how words have changed their meaning is an important topic in Natural Language Processing. However, evaluations of methods to characterise such change have been limited to small, handcrafted resources. We introduce an English evaluation set which is larger, more varied, and more realistic than seen to date, with terms derived from a historical thesaurus. Moreover, the dataset is unique in that it represents change as a shift from the term of interest to a WordNet synset. Using the synset lemmas, we can use this set to evaluate (standard) methods that detect change between word pairs, as well as (adapted) methods that detect the change between a term and a sense overall. We show that performance on the new data set is much lower than earlier reported findings, setting a new standard.- Anthology ID:
- W19-6105
- Volume:
- Proceedings of the 22nd Nordic Conference on Computational Linguistics
- Month:
- September–October
- Year:
- 2019
- Address:
- Turku, Finland
- Editors:
- Mareike Hartmann, Barbara Plank
- Venue:
- NoDaLiDa
- SIG:
- Publisher:
- Linköping University Electronic Press
- Note:
- Pages:
- 44–54
- Language:
- URL:
- https://aclanthology.org/W19-6105
- DOI:
- Cite (ACL):
- Astrid van Aggelen, Antske Fokkens, Laura Hollink, and Jacco van Ossenbruggen. 2019. A larger-scale evaluation resource of terms and their shift direction for diachronic lexical semantics. In Proceedings of the 22nd Nordic Conference on Computational Linguistics, pages 44–54, Turku, Finland. Linköping University Electronic Press.
- Cite (Informal):
- A larger-scale evaluation resource of terms and their shift direction for diachronic lexical semantics (van Aggelen et al., NoDaLiDa 2019)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/W19-6105.pdf