Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluation
Takyoung Kim, Jamin Shin, Young-Ho Kim, Sanghwan Bae, Sungdong Kim
Abstract
Most task-oriented dialogue (TOD) benchmarks assume users that know exactly how to use the system by constraining the user behaviors within the system’s capabilities via strict user goals, namely “user familiarity” bias. This data bias deepens when it combines with data-driven TOD systems, as it is impossible to fathom the effect of it with existing static evaluations. Hence, we conduct an interactive user study to unveil how vulnerable TOD systems are against realistic scenarios. In particular, we compare users with 1) detailed goal instructions that conform to the system boundaries (closed-goal) and 2) vague goal instructions that are often unsupported but realistic (open-goal). Our study reveals that conversations in open-goal settings lead to catastrophic failures of the system, in which 92% of the dialogues had significant issues. Moreover, we conduct a thorough analysis to identify distinctive features between the two settings through error annotation. From this, we discover a novel “pretending” behavior, in which the system pretends to handle the user requests even though they are beyond the system’s capabilities. We discuss its characteristics and toxicity while showing recent large language models can also suffer from this behavior.- Anthology ID:
- 2024.nlp4convai-1.3
- Volume:
- Proceedings of the 6th Workshop on NLP for Conversational AI (NLP4ConvAI 2024)
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Elnaz Nouri, Abhinav Rastogi, Georgios Spithourakis, Bing Liu, Yun-Nung Chen, Yu Li, Alon Albalak, Hiromi Wakaki, Alexandros Papangelis
- Venues:
- NLP4ConvAI | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 37–55
- Language:
- URL:
- https://aclanthology.org/2024.nlp4convai-1.3
- DOI:
- Cite (ACL):
- Takyoung Kim, Jamin Shin, Young-Ho Kim, Sanghwan Bae, and Sungdong Kim. 2024. Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluation. In Proceedings of the 6th Workshop on NLP for Conversational AI (NLP4ConvAI 2024), pages 37–55, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluation (Kim et al., NLP4ConvAI-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.nlp4convai-1.3.pdf