Contextual Bangla Neural Stemmer: Finding Contextualized Root Word Representations for Bangla Words
Md Fahim, Amin Ahsan Ali, M Ashraful Amin, Akmmahbubur Rahman
Abstract
Stemmers are commonly used in NLP to reduce words to their root form. However, this process may discard important information and yield incorrect root forms, affecting the accuracy of NLP tasks. To address these limitations, we propose a Contextual Bangla Neural Stemmer for Bangla language to enhance word representations. Our method involves splitting words into characters within the Neural Stemming Block, obtaining vector representations for both stem words and unknown vocabulary words. A loss function aligns these representations with Word2Vec representations, followed by contextual word representations from a Universal Transformer encoder. Mean Pooling generates sentence-level representations that are aligned with BanglaBERT’s representations using a MLP layer. The proposed model also tries to build good representations for out-of-vocabulary (OOV) words. Experiments with our model on five Bangla datasets shows around 5% average improvement over the vanilla approach. Notably, our method avoids BERT retraining, focusing on root word detection and addressing OOV and sub-word issues. By incorporating our approach into a large corpus-based Language Model, we expect further improvements in aspects like explainability.- Anthology ID:
- 2023.banglalp-1.11
- Volume:
- Proceedings of the First Workshop on Bangla Language Processing (BLP-2023)
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Firoj Alam, Sudipta Kar, Shammur Absar Chowdhury, Farig Sadeque, Ruhul Amin
- Venue:
- BanglaLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 94–103
- Language:
- URL:
- https://aclanthology.org/2023.banglalp-1.11
- DOI:
- 10.18653/v1/2023.banglalp-1.11
- Cite (ACL):
- Md Fahim, Amin Ahsan Ali, M Ashraful Amin, and Akmmahbubur Rahman. 2023. Contextual Bangla Neural Stemmer: Finding Contextualized Root Word Representations for Bangla Words. In Proceedings of the First Workshop on Bangla Language Processing (BLP-2023), pages 94–103, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Contextual Bangla Neural Stemmer: Finding Contextualized Root Word Representations for Bangla Words (Fahim et al., BanglaLP 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.banglalp-1.11.pdf