CLEANANERCorp: Identifying and Correcting Incorrect Labels in the ANERcorp Dataset

Mashael AlDuwais, Hend Al-Khalifa, Abdulmalik AlSalman


Abstract
Label errors are a common issue in machine learning datasets, particularly for tasks such as Named Entity Recognition. Such label erros might hurt model training, affect evaluation results, and lead to an inaccurate assessment of model performance. In this study, we dived deep into one of the widely adopted Arabic NER benchmark datasets (ANERcorp) and found a significant number of annotation errors, missing labels, and inconsistencies. Therefore, in this study, we conducted empirical research to understand these erros, correct them and propose a cleaner version of the dataset named CLEANANERCorp. CLEANANERCorp will serve the research community as a more accurate and consistent benchmark.
Anthology ID:
2024.osact-1.2
Volume:
Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Hend Al-Khalifa, Kareem Darwish, Hamdy Mubarak, Mona Ali, Tamer Elsayed
Venues:
OSACT | WS
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
13–19
Language:
URL:
https://aclanthology.org/2024.osact-1.2
DOI:
Bibkey:
Cite (ACL):
Mashael AlDuwais, Hend Al-Khalifa, and Abdulmalik AlSalman. 2024. CLEANANERCorp: Identifying and Correcting Incorrect Labels in the ANERcorp Dataset. In Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024, pages 13–19, Torino, Italia. ELRA and ICCL.
Cite (Informal):
CLEANANERCorp: Identifying and Correcting Incorrect Labels in the ANERcorp Dataset (AlDuwais et al., OSACT-WS 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2024.osact-1.2.pdf