Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study

Faeze Ghorbanpour, Daryna Dementieva, Alexandar Fraser


Abstract
Despite growing interest in automated hate speech detection, most existing approaches overlook the linguistic diversity of online content. Multilingual instruction-tuned large language models such as LLaMA, Aya, Qwen, and BloomZ offer promising capabilities across languages, but their effectiveness in identifying hate speech through zero-shot and few-shot prompting remains underexplored. This work evaluates LLM prompting-based detection across eight non-English languages, utilizing several prompting techniques and comparing them to fine-tuned encoder models. We show that while zero-shot and few-shot prompting lag behind fine-tuned encoder models on most of the real-world evaluation sets, they achieve better generalization on functional tests for hate speech detection. Our study also reveals that prompt design plays a critical role, with each language often requiring customized prompting techniques to maximize performance.
Anthology ID:
2025.woah-1.39
Volume:
Proceedings of the The 9th Workshop on Online Abuse and Harms (WOAH)
Month:
August
Year:
2025
Address:
Vienna, Austria
Editors:
Agostina Calabrese, Christine de Kock, Debora Nozza, Flor Miriam Plaza-del-Arco, Zeerak Talat, Francielle Vargas
Venues:
WOAH | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
413–425
Language:
URL:
https://preview.aclanthology.org/landing_page/2025.woah-1.39/
DOI:
Bibkey:
Cite (ACL):
Faeze Ghorbanpour, Daryna Dementieva, and Alexandar Fraser. 2025. Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study. In Proceedings of the The 9th Workshop on Online Abuse and Harms (WOAH), pages 413–425, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study (Ghorbanpour et al., WOAH 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2025.woah-1.39.pdf
Supplementarymaterial:
 2025.woah-1.39.SupplementaryMaterial.zip