Hope_Speech_Alchemists@DravidianLangTech 2026: TF-IDF SVM and XLM-RoBERTa with Focal Loss for Hope Speech Detection in Tulu
Joel Johnson, Meclin A Francis, Jyoti Kumari, Malavika Sreekumar, Vinay Babu Ulli
Abstract
This paper describes our system submitted to the shared task on Hope Speech Detection in Tulu at DravidianLangTech@ACL 2026 hope-speech-dravidianlangtech-acl-2026. The task comprises two sub-tasks: coarse-grained classification into four categories Task 1 and fine-grained classification into five categories Task 2. We compare a traditional TF-IDF + LinearSVC baseline against XLM-RoBERTa fine-tuned with minority-class oversampling and Focal Loss. Our experiments reveal an interesting trade-off: while the transformer approach achieves the best validation Macro-F1 of 0.57 on the coarse-grained task, the TF-IDF baseline outperforms it on the smaller fine-grained task, highlighting the data scarcity threshold below which large pre-trained models struggle to generalise. On the official test set, our system achieves a Macro-F1 of 0.55 on Task 1 and 0.40 on Task 2. The code is publicly available at: https://github.com/meclin2345/Hope_Speech_Alchemists- Anthology ID:
- 2026.dravidianlangtech-1.39
- Volume:
- Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages
- Month:
- July
- Year:
- 2026
- Address:
- Underline (Virtual)
- Editors:
- Bharathi Raja Chakravarthi, Ruba Priyadharshini, Anand Kumar Madasamy, Sajeetha Thavareesan, Saranya Rajiakodi, Subalalitha Navaneethakrishnan, Dhivya Chinnappa, Balasubramanian Palani, Malliga Subramanian, Kogilavani Shanmugavadivel, Ratnavel Rajalakshmi
- Venues:
- DravidianLangTech | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 263–267
- Language:
- URL:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.dravidianlangtech-1.39/
- DOI:
- Cite (ACL):
- Joel Johnson, Meclin A Francis, Jyoti Kumari, Malavika Sreekumar, and Vinay Babu Ulli. 2026. Hope_Speech_Alchemists@DravidianLangTech 2026: TF-IDF SVM and XLM-RoBERTa with Focal Loss for Hope Speech Detection in Tulu. In Proceedings of the Sixth Workshop on Speech, Vision, and Language Technologies for Dravidian Languages, pages 263–267, Underline (Virtual). Association for Computational Linguistics.
- Cite (Informal):
- Hope_Speech_Alchemists@DravidianLangTech 2026: TF-IDF SVM and XLM-RoBERTa with Focal Loss for Hope Speech Detection in Tulu (Johnson et al., DravidianLangTech 2026)
- PDF:
- https://preview.aclanthology.org/ingest-acl-workshops/2026.dravidianlangtech-1.39.pdf