A Hybrid Transformer–Sequential Model for Depression Detection in Bangla–English Code-Mixed Text

Md Siddikul Imam Kawser, Jidan Al Abrar, Mehebub Bin Kabir, Md. Rayhan Chowdhury, Md Ataullah Bahari


Abstract
Depression detection from social media text is critical for early mental health intervention, yet existing NLP systems underperform in low-resource, code-mixed settings. Bangla-English code-mixing, common across South Asian online communities, poses unique challenges due to irregular grammar, transliteration, and scarce labeled data. To address this gap, we introduce DepressiveText, a 7,019-sample dataset of Bangla-English social media posts annotated for depressive signals, with strong inter-annotator agreement (𝜅 = 0.84). We further propose a hybrid architecture that combines BanglishBERT embeddings with an LSTM classifier, enabling the model to capture both contextual and sequential cues. Comparative experiments with traditional ML, deep learning, and multilingual transformer baselines demonstrate that our approach achieves the highest performance, with an accuracy of 0.8889. We also employ LIME to enhance interpretability by identifying key lexical triggers. Our findings underscore the effectiveness of hybrid transformer–sequence models for low-resource code-mixed NLP and highlight their potential in real-world mental health applications.
Anthology ID:
2025.banglalp-1.8
Volume:
Proceedings of the Second Workshop on Bangla Language Processing (BLP-2025)
Month:
December
Year:
2025
Address:
Mumbai, India
Editors:
Firoj Alam, Sudipta Kar, Shammur Absar Chowdhury, Naeemul Hassan, Enamul Hoque Prince, Mohiuddin Tasnim, Md Rashad Al Hasan Rony, Md Tahmid Rahman Rahman
Venues:
BanglaLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
107–112
Language:
URL:
https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.banglalp-1.8/
DOI:
Bibkey:
Cite (ACL):
Md Siddikul Imam Kawser, Jidan Al Abrar, Mehebub Bin Kabir, Md. Rayhan Chowdhury, and Md Ataullah Bahari. 2025. A Hybrid Transformer–Sequential Model for Depression Detection in Bangla–English Code-Mixed Text. In Proceedings of the Second Workshop on Bangla Language Processing (BLP-2025), pages 107–112, Mumbai, India. Association for Computational Linguistics.
Cite (Informal):
A Hybrid Transformer–Sequential Model for Depression Detection in Bangla–English Code-Mixed Text (Kawser et al., BanglaLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-ijcnlp-aacl/2025.banglalp-1.8.pdf