SemEval-2021 Task 5: Toxic Spans Detection
John Pavlopoulos, Jeffrey Sorensen, Léo Laugier, Ion Androutsopoulos
Abstract
The Toxic Spans Detection task of SemEval-2021 required participants to predict the spans of toxic posts that were responsible for the toxic label of the posts. The task could be addressed as supervised sequence labeling, using training data with gold toxic spans provided by the organisers. It could also be treated as rationale extraction, using classifiers trained on potentially larger external datasets of posts manually annotated as toxic or not, without toxic span annotations. For the supervised sequence labeling approach and evaluation purposes, posts previously labeled as toxic were crowd-annotated for toxic spans. Participants submitted their predicted spans for a held-out test set and were scored using character-based F1. This overview summarises the work of the 36 teams that provided system descriptions.- Anthology ID:
- 2021.semeval-1.6
- Volume:
- Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)
- Month:
- August
- Year:
- 2021
- Address:
- Online
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 59–69
- Language:
- URL:
- https://aclanthology.org/2021.semeval-1.6
- DOI:
- 10.18653/v1/2021.semeval-1.6
- Cite (ACL):
- John Pavlopoulos, Jeffrey Sorensen, Léo Laugier, and Ion Androutsopoulos. 2021. SemEval-2021 Task 5: Toxic Spans Detection. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 59–69, Online. Association for Computational Linguistics.
- Cite (Informal):
- SemEval-2021 Task 5: Toxic Spans Detection (Pavlopoulos et al., SemEval 2021)
- PDF:
- https://preview.aclanthology.org/remove-xml-comments/2021.semeval-1.6.pdf