Flexible Weight Tuning and Weight Fusion Strategies for Continual Named Entity Recognition

Yahan Yu, Duzhen Zhang, Xiuyi Chen, Chenhui Chu

Abstract
Continual Named Entity Recognition (CNER) is dedicated to sequentially learning new entity types while mitigating catastrophic forgetting of old entity types. Traditional CNER approaches commonly employ knowledge distillation to retain old knowledge within the current model. However, because only the representations of old and new models are constrained to be consistent, the reliance solely on distillation in existing methods still suffers from catastrophic forgetting. To further alleviate the forgetting issue of old entity types, this paper introduces flexible Weight Tuning (WT) and Weight Fusion (WF) strategies for CNER. The WT strategy, applied at each training step, employs a learning rate schedule on the parameters of the current model. After learning the current task, the WF strategy dynamically integrates knowledge from both the current and previous models for inference. Notably, these two strategies are model-agnostic and seamlessly integrate with existing State-Of-The-Art (SOTA) models. Extensive experiments demonstrate that the WT and WF strategies consistently enhance the performance of previous SOTA methods across ten CNER settings in three datasets.
Anthology ID:
2024.findings-acl.79
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1351–1358
Language:
URL:
https://aclanthology.org/2024.findings-acl.79
DOI:
Bibkey:
Cite (ACL):
Yahan Yu, Duzhen Zhang, Xiuyi Chen, and Chenhui Chu. 2024. Flexible Weight Tuning and Weight Fusion Strategies for Continual Named Entity Recognition. In Findings of the Association for Computational Linguistics ACL 2024, pages 1351–1358, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Flexible Weight Tuning and Weight Fusion Strategies for Continual Named Entity Recognition (Yu et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2024.findings-acl.79.pdf
PDF Search