A Framework to Generate High-Quality Datapoints for Multiple Novel Intent Detection
Ankan Mullick, Sukannya Purkayastha, Pawan Goyal, Niloy Ganguly
Abstract
Systems like Voice-command based conversational agents are characterized by a pre-defined set of skills or intents to perform user specified tasks. In the course of time, newer intents may emerge requiring retraining. However, the newer intents may not be explicitly announced and need to be inferred dynamically. Thus, there are two important tasks at hand (a). identifying emerging new intents, (b). annotating data of the new intents so that the underlying classifier can be retrained efficiently. The tasks become specially challenging when a large number of new intents emerge simultaneously and there is a limited budget of manual annotation. In this paper, we propose MNID (Multiple Novel Intent Detection) which is a cluster based framework to detect multiple novel intents with budgeted human annotation cost. Empirical results on various benchmark datasets (of different sizes) demonstrate that MNID, by intelligently using the budget for annotation, outperforms the baseline methods in terms of accuracy and F1-score.- Anthology ID:
- 2022.findings-naacl.21
- Volume:
- Findings of the Association for Computational Linguistics: NAACL 2022
- Month:
- July
- Year:
- 2022
- Address:
- Seattle, United States
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 282–292
- Language:
- URL:
- https://aclanthology.org/2022.findings-naacl.21
- DOI:
- 10.18653/v1/2022.findings-naacl.21
- Cite (ACL):
- Ankan Mullick, Sukannya Purkayastha, Pawan Goyal, and Niloy Ganguly. 2022. A Framework to Generate High-Quality Datapoints for Multiple Novel Intent Detection. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 282–292, Seattle, United States. Association for Computational Linguistics.
- Cite (Informal):
- A Framework to Generate High-Quality Datapoints for Multiple Novel Intent Detection (Mullick et al., Findings 2022)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2022.findings-naacl.21.pdf
- Code
- sukannyapurkayastha/mnid