Abstract
The extraction of valuable information from the vast amount of digital data available today has become increasingly important, making Named Entity Recognition models an essential component of information extraction tasks. This emphasizes the importance of understanding the factors that can compromise the performance of these models. Many studies have examined the impact of data annotation errors on NER models, leaving the broader implication of overall data quality on these models unexplored. In this work, we evaluate the robustness of three prominent NER models on datasets with varying amounts of textual noise types. The results show that as the noise in the dataset increases, model performance declines, with a minor impact for some noise types and a significant drop in performance for others. The findings of this research can be used as a foundation for building robust NER systems by enhancing dataset quality beforehand.- Anthology ID:
- 2024.wnut-1.8
- Volume:
- Proceedings of the Ninth Workshop on Noisy and User-generated Text (W-NUT 2024)
- Month:
- March
- Year:
- 2024
- Address:
- San Ġiljan, Malta
- Editors:
- Rob van der Goot, JinYeong Bak, Max Müller-Eberstein, Wei Xu, Alan Ritter, Tim Baldwin
- Venues:
- WNUT | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 79–88
- Language:
- URL:
- https://aclanthology.org/2024.wnut-1.8
- DOI:
- Cite (ACL):
- Divya Bhadauria, Alejandro Sierra Múnera, and Ralf Krestel. 2024. The Effects of Data Quality on Named Entity Recognition. In Proceedings of the Ninth Workshop on Noisy and User-generated Text (W-NUT 2024), pages 79–88, San Ġiljan, Malta. Association for Computational Linguistics.
- Cite (Informal):
- The Effects of Data Quality on Named Entity Recognition (Bhadauria et al., WNUT-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.wnut-1.8.pdf