Continual Named Entity Recognition without Catastrophic Forgetting
Duzhen Zhang, Wei Cong, Jiahua Dong, Yahan Yu, Xiuyi Chen, Yonggang Zhang, Zhen Fang
Abstract
Continual Named Entity Recognition (CNER) is a burgeoning area, which involves updating an existing model by incorporating new entity types sequentially. Nevertheless, continual learning approaches are often severely afflicted by catastrophic forgetting. This issue is intensified in CNER due to the consolidation of old entity types from previous steps into the non-entity type at each step, leading to what is known as the semantic shift problem of the non-entity type. In this paper, we introduce a pooled feature distillation loss that skillfully navigates the trade-off between retaining knowledge of old entity types and acquiring new ones, thereby more effectively mitigating the problem of catastrophic forgetting. Additionally, we develop a confidence-based pseudo-labeling for the non-entity type, i.e., predicting entity types using the old model to handle the semantic shift of the non-entity type. Following the pseudo-labeling process, we suggest an adaptive re-weighting type-balanced learning strategy to handle the issue of biased type distribution. We carried out comprehensive experiments on ten CNER settings using three different datasets. The results illustrate that our method significantly outperforms prior state-of-the-art approaches, registering an average improvement of 6.3% and 8.0% in Micro and Macro F1 scores, respectively.- Anthology ID:
- 2023.emnlp-main.509
- Volume:
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 8186–8197
- Language:
- URL:
- https://aclanthology.org/2023.emnlp-main.509
- DOI:
- 10.18653/v1/2023.emnlp-main.509
- Cite (ACL):
- Duzhen Zhang, Wei Cong, Jiahua Dong, Yahan Yu, Xiuyi Chen, Yonggang Zhang, and Zhen Fang. 2023. Continual Named Entity Recognition without Catastrophic Forgetting. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8186–8197, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Continual Named Entity Recognition without Catastrophic Forgetting (Zhang et al., EMNLP 2023)
- PDF:
- https://preview.aclanthology.org/improve-issue-templates/2023.emnlp-main.509.pdf