Towards a Unified Framework for Adaptable Problematic Content Detection via Continual Learning
Ali Omrani, Alireza Salkhordeh Ziabari, Preni Golazizian, Jeffrey Sorensen, Morteza Dehghani
Abstract
Detecting problematic content, such as hate speech, is a multifaceted and ever-changing task, influenced by social dynamics, user populations, diversity of sources, and evolving language. There has been significant efforts, both in academia and in industry, to develop annotated resources that capture various aspects of problematic content. Due to researchers’ diverse objectives, these annotations are often inconsistent and hence, reports of progress on the detection of problematic content are fragmented. This pattern is expected to persist unless we pool these resources, taking into account the dynamic nature of this issue. In this paper, we propose integrating the available resources, leveraging their dynamic nature to break this pattern, and introduce a continual learning framework and benchmark for problematic content detection. Our benchmark, comprising 84 related tasks, creates a novel measure of progress: prioritizing the adaptability of classifiers to evolving tasks over excelling in specific tasks. To ensure continuous relevance, our benchmark is designed for seamless integration of new tasks. Our results demonstrate that continual learning methods outperform static approaches by up to 17% and 4% AUC in capturing the evolving content and adapting to novel forms of problematic content- Anthology ID:
- 2024.woah-1.7
- Volume:
- Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024)
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Yi-Ling Chung, Zeerak Talat, Debora Nozza, Flor Miriam Plaza-del-Arco, Paul Röttger, Aida Mostafazadeh Davani, Agostina Calabrese
- Venues:
- WOAH | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 68–109
- Language:
- URL:
- https://aclanthology.org/2024.woah-1.7
- DOI:
- Cite (ACL):
- Ali Omrani, Alireza Salkhordeh Ziabari, Preni Golazizian, Jeffrey Sorensen, and Morteza Dehghani. 2024. Towards a Unified Framework for Adaptable Problematic Content Detection via Continual Learning. In Proceedings of the 8th Workshop on Online Abuse and Harms (WOAH 2024), pages 68–109, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- Towards a Unified Framework for Adaptable Problematic Content Detection via Continual Learning (Omrani et al., WOAH-WS 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.woah-1.7.pdf