BETOLD: A Task-Oriented Dialog Dataset for Breakdown Detection
Silvia Terragni, Bruna Guedes, Andre Manso, Modestas Filipavicius, Nghia Khau, Roland Mathis
Abstract
Task-Oriented Dialog (TOD) systems often suffer from dialog breakdowns - situations in which users cannot or do not want to proceed with the conversation. Ideally TOD systems should be able to detect dialog breakdowns to prevent users from quitting a conversation and to encourage them to interact with the system again. In this paper, we present BETOLD, a privacy-preserving dataset for breakdown detection. The dataset consists of user and system turns represented by intents and entity annotations, derived from NLU and NLG dialog manager components. We also propose an attention-based model that detects potential breakdowns using these annotations, instead of the utterances’ text. This approach achieves a comparable performance to the corresponding utterance-only model, while ensuring data privacy.- Anthology ID:
- 2022.cai-1.4
- Volume:
- Proceedings of the Second Workshop on When Creative AI Meets Conversational AI
- Month:
- October
- Year:
- 2022
- Address:
- Gyeongju, Republic of Korea
- Venue:
- CAI
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 23–34
- Language:
- URL:
- https://aclanthology.org/2022.cai-1.4
- DOI:
- Cite (ACL):
- Silvia Terragni, Bruna Guedes, Andre Manso, Modestas Filipavicius, Nghia Khau, and Roland Mathis. 2022. BETOLD: A Task-Oriented Dialog Dataset for Breakdown Detection. In Proceedings of the Second Workshop on When Creative AI Meets Conversational AI, pages 23–34, Gyeongju, Republic of Korea. Association for Computational Linguistics.
- Cite (Informal):
- BETOLD: A Task-Oriented Dialog Dataset for Breakdown Detection (Terragni et al., CAI 2022)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/2022.cai-1.4.pdf