Meta Learning for Natural Language Processing: A Survey

Hung-yi Lee, Shang-Wen Li, Thang Vu


Abstract
Deep learning has been the mainstream technique in the natural language processing (NLP) area. However, deep learning requires many labeled data and is less generalizable across domains. Meta-learning is an arising field in machine learning. It studies approaches to learning better learning algorithms and aims to improve algorithms in various aspects, including data efficiency and generalizability. The efficacy of meta-learning has been shown in many NLP tasks, but there is no systematic survey of these approaches in NLP, which hinders more researchers from joining the field. Our goal with this survey paper is to offer researchers pointers to relevant meta-learning works in NLP and attract more attention from the NLP community to drive future innovation. This paper first introduces the general concepts of meta-learning and the common approaches. Then we summarize task construction settings, applications of meta-learning for various NLP problems and review the development of meta-learning in the NLP community.
Anthology ID:
2022.naacl-main.49
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
666–684
Language:
URL:
https://aclanthology.org/2022.naacl-main.49
DOI:
10.18653/v1/2022.naacl-main.49
Bibkey:
Cite (ACL):
Hung-yi Lee, Shang-Wen Li, and Thang Vu. 2022. Meta Learning for Natural Language Processing: A Survey. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 666–684, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
Meta Learning for Natural Language Processing: A Survey (Lee et al., NAACL 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/2022.naacl-main.49.pdf
Video:
 https://preview.aclanthology.org/auto-file-uploads/2022.naacl-main.49.mp4
Data
GLUE