Abstract
In goal-driven dialogue systems, success is often defined based on a structured definition of the goal. This requires that the dialogue system be constrained to handle a specific class of goals and that there be a mechanism to measure success with respect to that goal. However, in many human-human dialogues the diversity of goals makes it infeasible to define success in such a way. To address this scenario, we consider the task of automatically predicting success in goal-driven human-human dialogues using only the information communicated between participants in the form of text. We build a dataset from stackoverflow.com which consists of exchanges between two users in the technical domain where ground-truth success labels are available. We then propose a turn-based hierarchical neural network model that can be used to predict success without requiring a structured goal definition. We show this model outperforms rule-based heuristics and other baselines as it is able to detect patterns over the course of a dialogue and capture notions such as gratitude.- Anthology ID:
- W17-5531
- Volume:
- Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue
- Month:
- August
- Year:
- 2017
- Address:
- Saarbrücken, Germany
- Editors:
- Kristiina Jokinen, Manfred Stede, David DeVault, Annie Louis
- Venue:
- SIGDIAL
- SIG:
- SIGDIAL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 253–262
- Language:
- URL:
- https://aclanthology.org/W17-5531
- DOI:
- 10.18653/v1/W17-5531
- Cite (ACL):
- Michael Noseworthy, Jackie Chi Kit Cheung, and Joelle Pineau. 2017. Predicting Success in Goal-Driven Human-Human Dialogues. In Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 253–262, Saarbrücken, Germany. Association for Computational Linguistics.
- Cite (Informal):
- Predicting Success in Goal-Driven Human-Human Dialogues (Noseworthy et al., SIGDIAL 2017)
- PDF:
- https://preview.aclanthology.org/corrections-2024-07/W17-5531.pdf