Gaining Insights into Unrecognized User Utterances in Task-Oriented Dialog Systems
Ella Rabinovich, Matan Vetzler, David Boaz, Vineet Kumar, Gaurav Pandey, Ateret Anaby Tavor
Abstract
The rapidly growing market demand for automatic dialogue agents capable of goal-oriented behavior has caused many tech-industry leaders to invest considerable efforts into task-oriented dialog systems. The success of these systems is highly dependent on the accuracy of their intent identification – the process of deducing the goal or meaning of the user’s request and mapping it to one of the known intents for further processing. Gaining insights into unrecognized utterances – user requests the systems fails to attribute to a known intent – is therefore a key process in continuous improvement of goal-oriented dialog systems. We present an end-to-end pipeline for processing unrecognized user utterances, deployed in a real-world, commercial task-oriented dialog system, including a specifically-tailored clustering algorithm, a novel approach to cluster representative extraction, and cluster naming. We evaluated the proposed components, demonstrating their benefits in the analysis of unrecognized user requests.- Anthology ID:
- 2022.emnlp-industry.22
- Volume:
- Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track
- Month:
- December
- Year:
- 2022
- Address:
- Abu Dhabi, UAE
- Editors:
- Yunyao Li, Angeliki Lazaridou
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 218–225
- Language:
- URL:
- https://aclanthology.org/2022.emnlp-industry.22
- DOI:
- 10.18653/v1/2022.emnlp-industry.22
- Cite (ACL):
- Ella Rabinovich, Matan Vetzler, David Boaz, Vineet Kumar, Gaurav Pandey, and Ateret Anaby Tavor. 2022. Gaining Insights into Unrecognized User Utterances in Task-Oriented Dialog Systems. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 218–225, Abu Dhabi, UAE. Association for Computational Linguistics.
- Cite (Informal):
- Gaining Insights into Unrecognized User Utterances in Task-Oriented Dialog Systems (Rabinovich et al., EMNLP 2022)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2022.emnlp-industry.22.pdf