Abstract
ChatGPT, developed by OpenAI, has made a significant impact on the world, mainly on how people interact with technology. In this study, we evaluate ChatGPT’s ability to detect hate speech in Turkish tweets and measure its strength using zero- and few-shot paradigms and compare the results to the supervised fine-tuning BERT model. On evaluations with the SIU2023-NST dataset, ChatGPT achieved 65.81% accuracy in detecting hate speech for the few-shot setting, while BERT with supervised fine-tuning achieved 82.22% accuracy. This results supports previous findings that show that, despite its much smaller size, BERT is more suitable for natural language classifications tasks such as hate speech detection.- Anthology ID:
- 2024.case-1.6
- Volume:
- Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024)
- Month:
- March
- Year:
- 2024
- Address:
- St. Julians, Malta
- Editors:
- Ali Hürriyetoğlu, Hristo Tanev, Surendrabikram Thapa, Gökçe Uludoğan
- Venues:
- CASE | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 54–59
- Language:
- URL:
- https://aclanthology.org/2024.case-1.6
- DOI:
- Cite (ACL):
- Somaiyeh Dehghan and Berrin Yanikoglu. 2024. Evaluating ChatGPT’s Ability to Detect Hate Speech in Turkish Tweets. In Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024), pages 54–59, St. Julians, Malta. Association for Computational Linguistics.
- Cite (Informal):
- Evaluating ChatGPT’s Ability to Detect Hate Speech in Turkish Tweets (Dehghan & Yanikoglu, CASE-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.case-1.6.pdf