Abstract
Incorporating large-scale pre-trained models with the prototypical neural networks is a de-facto paradigm in few-shot named entity recognition. Existing methods, unfortunately, are not aware of the fact that embeddings from pre-trained models contain a prominently large amount of information regarding word frequencies, biasing prototypical neural networks against learning word entities. This discrepancy constrains the two models’ synergy. Thus, we propose a one-line-code normalization method to reconcile such a mismatch with empirical and theoretical grounds. Our experiments based on nine benchmark datasets show the superiority of our method over the counterpart models and are comparable to the state-of-the-art methods. In addition to the model enhancement, our work also provides an analytical viewpoint for addressing the general problems in few-shot name entity recognition or other tasks that rely on pre-trained models or prototypical neural networks.- Anthology ID:
- 2022.findings-emnlp.129
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2022
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Editors:
- Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1793–1807
- Language:
- URL:
- https://aclanthology.org/2022.findings-emnlp.129
- DOI:
- 10.18653/v1/2022.findings-emnlp.129
- Cite (ACL):
- Youcheng Huang, Wenqiang Lei, Jie Fu, and Jiancheng Lv. 2022. Reconciliation of Pre-trained Models and Prototypical Neural Networks in Few-shot Named Entity Recognition. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1793–1807, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- Reconciliation of Pre-trained Models and Prototypical Neural Networks in Few-shot Named Entity Recognition (Huang et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2022.findings-emnlp.129.pdf