sthruggle at SemEval-2019 Task 5: An Ensemble Approach to Hate Speech Detection

Aria Nourbakhsh, Frida Vermeer, Gijs Wiltvank, Rob van der Goot

Abstract
In this paper, we present our approach to detection of hate speech against women and immigrants in tweets for our participation in the SemEval-2019 Task 5. We trained an SVM and an RF classifier using character bi- and trigram features and a BiLSTM pre-initialized with external word embeddings. We combined the predictions of the SVM, RF and BiLSTM in two different ensemble models. The first was a majority vote of the binary values, and the second used the average of the confidence scores. For development, we got the highest accuracy (75%) by the final ensemble model with majority voting. For testing, all models scored substantially lower and the scores between the classifiers varied more. We believe that these large differences between the higher accuracies in the development phase and the lower accuracies we obtained in the testing phase have partly to do with differences between the training, development and testing data.
Anthology ID:
S19-2086
Volume:
Proceedings of the 13th International Workshop on Semantic Evaluation
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota, USA
Editors:
Jonathan May, Ekaterina Shutova, Aurelie Herbelot, Xiaodan Zhu, Marianna Apidianaki, Saif M. Mohammad
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
484–488
Language:
URL:
https://preview.aclanthology.org/build-pipeline-with-new-library/S19-2086/
DOI:
10.18653/v1/S19-2086
Bibkey:
Cite (ACL):
Aria Nourbakhsh, Frida Vermeer, Gijs Wiltvank, and Rob van der Goot. 2019. sthruggle at SemEval-2019 Task 5: An Ensemble Approach to Hate Speech Detection. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 484–488, Minneapolis, Minnesota, USA. Association for Computational Linguistics.
Cite (Informal):
sthruggle at SemEval-2019 Task 5: An Ensemble Approach to Hate Speech Detection (Nourbakhsh et al., SemEval 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/build-pipeline-with-new-library/S19-2086.pdf
PDF Search