Abstract
Measuring the semantic similarity of different texts has many important applications in Digital Humanities research such as information retrieval, document clustering and text summarization. The performance of different methods depends on the length of the text, the domain and the language. This study focuses on experimenting with some of the current approaches to Finnish, which is a morphologically rich language. At the same time, we propose a simple method, TFW2V, which shows high efficiency in handling both long text documents and limited amounts of data. Furthermore, we design an objective evaluation method which can be used as a framework for benchmarking text similarity approaches.- Anthology ID:
- 2021.nlp4dh-1.19
- Volume:
- Proceedings of the Workshop on Natural Language Processing for Digital Humanities
- Month:
- December
- Year:
- 2021
- Address:
- NIT Silchar, India
- Editors:
- Mika Hämäläinen, Khalid Alnajjar, Niko Partanen, Jack Rueter
- Venue:
- NLP4DH
- SIG:
- Publisher:
- NLP Association of India (NLPAI)
- Note:
- Pages:
- 163–172
- Language:
- URL:
- https://aclanthology.org/2021.nlp4dh-1.19
- DOI:
- Cite (ACL):
- Quan Duong, Mika Hämäläinen, and Khalid Alnajjar. 2021. TFW2V: An Enhanced Document Similarity Method for the Morphologically Rich Finnish Language. In Proceedings of the Workshop on Natural Language Processing for Digital Humanities, pages 163–172, NIT Silchar, India. NLP Association of India (NLPAI).
- Cite (Informal):
- TFW2V: An Enhanced Document Similarity Method for the Morphologically Rich Finnish Language (Duong et al., NLP4DH 2021)
- PDF:
- https://preview.aclanthology.org/revert-3132-ingestion-checklist/2021.nlp4dh-1.19.pdf
- Code
- ruathudo/tfw2v