Abstract
User Satisfaction Estimation (USE) is crucial in helping measure the quality of a task-oriented dialogue system. However, the complex nature of implicit responses poses challenges in detecting user satisfaction, and most datasets are limited in size or not available to the public due to user privacy policies. Unlike task-oriented dialogue, large-scale annotated chitchat with emotion labels is publicly available. Therefore, we present a novel user satisfaction model with domain adaptation (USMDA) to utilize this chitchat. We adopt a dialogue Transformer encoder to capture contextual features from the dialogue. And we reduce domain discrepancy to learn dialogue-related invariant features. Moreover, USMDA jointly learns satisfaction signals in the chitchat context with user satisfaction estimation, and user actions in task-oriented dialogue with dialogue action recognition. Experimental results on two benchmarks show that our proposed framework for the USE task outperforms existing unsupervised domain adaptation methods. To the best of our knowledge, this is the first work to study user satisfaction estimation with unsupervised domain adaptation from chitchat to task-oriented dialogue.- Anthology ID:
- 2022.sigdial-1.59
- Volume:
- Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue
- Month:
- September
- Year:
- 2022
- Address:
- Edinburgh, UK
- Editors:
- Oliver Lemon, Dilek Hakkani-Tur, Junyi Jessy Li, Arash Ashrafzadeh, Daniel Hernández Garcia, Malihe Alikhani, David Vandyke, Ondřej Dušek
- Venue:
- SIGDIAL
- SIG:
- SIGDIAL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 630–636
- Language:
- URL:
- https://aclanthology.org/2022.sigdial-1.59
- DOI:
- 10.18653/v1/2022.sigdial-1.59
- Cite (ACL):
- Yan Pan, Mingyang Ma, Bernhard Pflugfelder, and Georg Groh. 2022. User Satisfaction Modeling with Domain Adaptation in Task-oriented Dialogue Systems. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 630–636, Edinburgh, UK. Association for Computational Linguistics.
- Cite (Informal):
- User Satisfaction Modeling with Domain Adaptation in Task-oriented Dialogue Systems (Pan et al., SIGDIAL 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-3/2022.sigdial-1.59.pdf
- Data
- EmoryNLP, SGD