Abstract
We present an optimized implementation of the weighted sequence alignment algorithm (a.k.a. weighted edit distance) in a scenario where the items to align are numeric vectors and the substitution weights are determined by their cosine similarity. The optimization relies on using vector and matrix operations provided by numeric computation libraries (including GPU acceleration) instead of loops. The resulting algorithm provides an efficient way of aligning large sets of texts represented as sequences of continuous-space numeric vectors (embeddings). The optimization made it possible to compute alignment-based similarity for all pairs of texts in a large corpus of Finnic oral folk poetry for the purpose of studying intertextuality in the oral tradition.- Anthology ID:
- 2022.nlp4dh-1.13
- Volume:
- Proceedings of the 2nd International Workshop on Natural Language Processing for Digital Humanities
- Month:
- November
- Year:
- 2022
- Address:
- Taipei, Taiwan
- Venue:
- NLP4DH
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 96–100
- Language:
- URL:
- https://aclanthology.org/2022.nlp4dh-1.13
- DOI:
- Cite (ACL):
- Maciej Janicki. 2022. Optimizing the weighted sequence alignment algorithm for large-scale text similarity computation. In Proceedings of the 2nd International Workshop on Natural Language Processing for Digital Humanities, pages 96–100, Taipei, Taiwan. Association for Computational Linguistics.
- Cite (Informal):
- Optimizing the weighted sequence alignment algorithm for large-scale text similarity computation (Janicki, NLP4DH 2022)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2022.nlp4dh-1.13.pdf