Named Entity Recognition - Is There a Glass Ceiling?

Tomasz Stanislawek, Anna Wróblewska, Alicja Wójcicka, Daniel Ziembicki, Przemyslaw Biecek


Abstract
Recent developments in Named Entity Recognition (NER) have resulted in better and better models. However, is there a glass ceiling? Do we know which types of errors are still hard or even impossible to correct? In this paper, we present a detailed analysis of the types of errors in state-of-the-art machine learning (ML) methods. Our study illustrates weak and strong points of the Stanford, CMU, FLAIR, ELMO and BERT models, as well as their shared limitations. We also introduce new techniques for improving annotation, training process, and for checking model quality and stability.
Anthology ID:
K19-1058
Volume:
Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)
Month:
November
Year:
2019
Address:
Hong Kong, China
Venue:
CoNLL
SIG:
SIGNLL
Publisher:
Association for Computational Linguistics
Note:
Pages:
624–633
Language:
URL:
https://aclanthology.org/K19-1058
DOI:
10.18653/v1/K19-1058
Bibkey:
Cite (ACL):
Tomasz Stanislawek, Anna Wróblewska, Alicja Wójcicka, Daniel Ziembicki, and Przemyslaw Biecek. 2019. Named Entity Recognition - Is There a Glass Ceiling?. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 624–633, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Named Entity Recognition - Is There a Glass Ceiling? (Stanislawek et al., CoNLL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/K19-1058.pdf
Attachment:
 K19-1058.Attachment.pdf
Supplementary material:
 K19-1058.Supplementary_Material.pdf