Multi-Label Intent Detection via Contrastive Task Specialization of Sentence Encoders
Ivan Vulić, Iñigo Casanueva, Georgios Spithourakis, Avishek Mondal, Tsung-Hsien Wen, Paweł Budzianowski
Abstract
Deploying task-oriented dialog ToD systems for new domains and tasks requires natural language understanding models that are 1) resource-efficient and work under low-data regimes; 2) adaptable, efficient, and quick-to-train; 3) expressive and can handle complex ToD scenarios with multiple user intents in a single utterance. Motivated by these requirements, we introduce a novel framework for multi-label intent detection (mID): MultI-ConvFiT (Multi-Label Intent Detection via Contrastive Conversational Fine-Tuning). While previous work on efficient single-label intent detection learns a classifier on top of a fixed sentence encoder (SE), we propose to 1) transform general-purpose SEs into task-specialized SEs via contrastive fine-tuning on annotated multi-label data, 2) where task specialization knowledge can be stored into lightweight adapter modules without updating the original parameters of the input SE, and then 3) we build improved mID classifiers stacked on top of fixed specialized SEs. Our main results indicate that MultI-ConvFiT yields effective mID models, with large gains over non-specialized SEs reported across a spectrum of different mID datasets, both in low-data and high-data regimes.- Anthology ID:
- 2022.emnlp-main.512
- Volume:
- Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, United Arab Emirates
- Editors:
- Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 7544–7559
- Language:
- URL:
- https://aclanthology.org/2022.emnlp-main.512
- DOI:
- 10.18653/v1/2022.emnlp-main.512
- Cite (ACL):
- Ivan Vulić, Iñigo Casanueva, Georgios Spithourakis, Avishek Mondal, Tsung-Hsien Wen, and Paweł Budzianowski. 2022. Multi-Label Intent Detection via Contrastive Task Specialization of Sentence Encoders. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7544–7559, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
- Cite (Informal):
- Multi-Label Intent Detection via Contrastive Task Specialization of Sentence Encoders (Vulić et al., EMNLP 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2022.emnlp-main.512.pdf