Meta-Learning for Few-Shot Named Entity Recognition

Cyprien de Lichy, Hadrien Glaude, William Campbell


Abstract
Meta-learning has recently been proposed to learn models and algorithms that can generalize from a handful of examples. However, applications to structured prediction and textual tasks pose challenges for meta-learning algorithms. In this paper, we apply two meta-learning algorithms, Prototypical Networks and Reptile, to few-shot Named Entity Recognition (NER), including a method for incorporating language model pre-training and Conditional Random Fields (CRF). We propose a task generation scheme for converting classical NER datasets into the few-shot setting, for both training and evaluation. Using three public datasets, we show these meta-learning algorithms outperform a reasonable fine-tuned BERT baseline. In addition, we propose a novel combination of Prototypical Networks and Reptile.
Anthology ID:
2021.metanlp-1.6
Volume:
Proceedings of the 1st Workshop on Meta Learning and Its Applications to Natural Language Processing
Month:
August
Year:
2021
Address:
Online
Venue:
MetaNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
44–58
Language:
URL:
https://aclanthology.org/2021.metanlp-1.6
DOI:
10.18653/v1/2021.metanlp-1.6
Bibkey:
Cite (ACL):
Cyprien de Lichy, Hadrien Glaude, and William Campbell. 2021. Meta-Learning for Few-Shot Named Entity Recognition. In Proceedings of the 1st Workshop on Meta Learning and Its Applications to Natural Language Processing, pages 44–58, Online. Association for Computational Linguistics.
Cite (Informal):
Meta-Learning for Few-Shot Named Entity Recognition (de Lichy et al., MetaNLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2021.metanlp-1.6.pdf
Optional supplementary material:
 2021.metanlp-1.6.OptionalSupplementaryMaterial.zip
Data
SGD