How to tackle an emerging topic? Combining strong and weak labels for Covid news NER

Aleksander Ficek, Fangyu Liu, Nigel Collier


Abstract
Being able to train Named Entity Recognition (NER) models for emerging topics is crucial for many real-world applications especially in the medical domain where new topics are continuously evolving out of the scope of existing models and datasets. For a realistic evaluation setup, we introduce a novel COVID-19 news NER dataset (COVIDNEWS-NER) and release 3000 entries of hand annotated strongly labelled sentences and 13000 auto-generated weakly labelled sentences. Besides the dataset, we propose CONTROSTER, a recipe to strategically combine weak and strong labels in improving NER in an emerging topic through transfer learning. We show the effectiveness of CONTROSTER on COVIDNEWS-NER while providing analysis on combining weak and strong labels for training. Our key findings are: (1) Using weak data to formulate an initial backbone before tuning on strong data outperforms methods trained on only strong or weak data. (2) A combination of out-of-domain and in-domain weak label training is crucial and can overcome saturation when being training on weak labels from a single source.
Anthology ID:
2022.aacl-short.60
Volume:
Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Month:
November
Year:
2022
Address:
Online only
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
488–496
Language:
URL:
https://aclanthology.org/2022.aacl-short.60
DOI:
Bibkey:
Cite (ACL):
Aleksander Ficek, Fangyu Liu, and Nigel Collier. 2022. How to tackle an emerging topic? Combining strong and weak labels for Covid news NER. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 488–496, Online only. Association for Computational Linguistics.
Cite (Informal):
How to tackle an emerging topic? Combining strong and weak labels for Covid news NER (Ficek et al., AACL-IJCNLP 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.aacl-short.60.pdf
Dataset:
 2022.aacl-short.60.Dataset.zip