Teacher and Student Models of Offensive Language in Social Media

Tharindu Ranasinghe, Marcos Zampieri


Abstract
State-of-the-art approaches to identifying offensive language online make use of large pre-trained transformer models. However, the inference time, disk, and memory requirements of these transformer models present challenges for their wide usage in the real world. Even the distilled transformer models remain prohibitively large for many usage scenarios. To cope with these challenges, in this paper, we propose transferring knowledge from transformer models to much smaller neural models to make predictions at the token- and at the post-level. We show that this approach leads to lightweight offensive language identification models that perform on par with large transformers but with 100 times fewer parameters and much less memory usage
Anthology ID:
2023.findings-acl.241
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3910–3922
Language:
URL:
https://aclanthology.org/2023.findings-acl.241
DOI:
Bibkey:
Cite (ACL):
Tharindu Ranasinghe and Marcos Zampieri. 2023. Teacher and Student Models of Offensive Language in Social Media. In Findings of the Association for Computational Linguistics: ACL 2023, pages 3910–3922, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Teacher and Student Models of Offensive Language in Social Media (Ranasinghe & Zampieri, Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/paclic-22-ingestion/2023.findings-acl.241.pdf