Abstract
Detecting neologisms is essential in real-time natural language processing applications. Not only can it enable to follow the lexical evolution of languages, but it is also essential for updating linguistic resources and parsers. In this paper, neology detection is considered as a classification task where a system has to assess whether a given lexical item is an actual neologism or not. We propose a combination of an unsupervised data mining technique and a supervised machine learning approach. It is inspired by current researches in stylometry and on token-level and character-level patterns. We train and evaluate our system on a manually designed reference dataset in French and Russian. We show that this approach is able to largely outperform state-of-the-art neology detection systems. Furthermore, character-level patterns exhibit good properties for multilingual extensions of the system.- Anthology ID:
- W17-4103
- Volume:
- Proceedings of the First Workshop on Subword and Character Level Models in NLP
- Month:
- September
- Year:
- 2017
- Address:
- Copenhagen, Denmark
- Editors:
- Manaal Faruqui, Hinrich Schuetze, Isabel Trancoso, Yadollah Yaghoobzadeh
- Venue:
- SCLeM
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 25–30
- Language:
- URL:
- https://aclanthology.org/W17-4103
- DOI:
- 10.18653/v1/W17-4103
- Cite (ACL):
- Gaël Lejeune and Emmanuel Cartier. 2017. Character Based Pattern Mining for Neology Detection. In Proceedings of the First Workshop on Subword and Character Level Models in NLP, pages 25–30, Copenhagen, Denmark. Association for Computational Linguistics.
- Cite (Informal):
- Character Based Pattern Mining for Neology Detection (Lejeune & Cartier, SCLeM 2017)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/W17-4103.pdf