Abstract
Intent classification (IC) and slot filling (SF) are core components in most goal-oriented dialogue systems. Current IC/SF models perform poorly when the number of training examples per class is small. We propose a new few-shot learning task, few-shot IC/SF, to study and improve the performance of IC and SF models on classes not seen at training time in ultra low resource scenarios. We establish a few-shot IC/SF benchmark by defining few-shot splits for three public IC/SF datasets, ATIS, TOP, and Snips. We show that two popular few-shot learning algorithms, model agnostic meta learning (MAML) and prototypical networks, outperform a fine-tuning baseline on this benchmark. Prototypical networks achieves significant gains in IC performance on the ATIS and TOP datasets, while both prototypical networks and MAML outperform the baseline with respect to SF on all three datasets. In addition, we demonstrate that joint training as well as the use of pre-trained language models, ELMo and BERT in our case, are complementary to these few-shot learning methods and yield further gains.- Anthology ID:
- 2020.nlp4convai-1.12
- Volume:
- Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI
- Month:
- July
- Year:
- 2020
- Address:
- Online
- Editors:
- Tsung-Hsien Wen, Asli Celikyilmaz, Zhou Yu, Alexandros Papangelis, Mihail Eric, Anuj Kumar, Iñigo Casanueva, Rushin Shah
- Venue:
- NLP4ConvAI
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 96–108
- Language:
- URL:
- https://aclanthology.org/2020.nlp4convai-1.12
- DOI:
- 10.18653/v1/2020.nlp4convai-1.12
- Cite (ACL):
- Jason Krone, Yi Zhang, and Mona Diab. 2020. Learning to Classify Intents and Slot Labels Given a Handful of Examples. In Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, pages 96–108, Online. Association for Computational Linguistics.
- Cite (Informal):
- Learning to Classify Intents and Slot Labels Given a Handful of Examples (Krone et al., NLP4ConvAI 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2020.nlp4convai-1.12.pdf
- Data
- ATIS