Abstract
While task-oriented dialogue systems have improved, not all users can fully accomplish their tasks. Users with limited knowledge about the system may experience dialogue breakdowns or fail to achieve their tasks because they do not know how to interact with the system. For addressing this issue, it would be desirable to construct a system that can estimate the user’s task success ability and adapt to that ability. In this study, we propose a method that estimates this ability by applying item response theory (IRT), commonly used in education for estimating examinee abilities, to task-oriented dialogue systems. Through experiments predicting the probability of a correct answer to each slot by using the estimated task success ability, we found that the proposed method significantly outperformed baselines.- Anthology ID:
- 2023.sigdial-1.39
- Volume:
- Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue
- Month:
- September
- Year:
- 2023
- Address:
- Prague, Czechia
- Editors:
- Svetlana Stoyanchev, Shafiq Joty, David Schlangen, Ondrej Dusek, Casey Kennington, Malihe Alikhani
- Venue:
- SIGDIAL
- SIG:
- SIGDIAL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 421–427
- Language:
- URL:
- https://aclanthology.org/2023.sigdial-1.39
- DOI:
- 10.18653/v1/2023.sigdial-1.39
- Cite (ACL):
- Ryu Hirai, Ao Guo, and Ryuichiro Higashinaka. 2023. Applying Item Response Theory to Task-oriented Dialogue Systems for Accurately Determining User’s Task Success Ability. In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 421–427, Prague, Czechia. Association for Computational Linguistics.
- Cite (Informal):
- Applying Item Response Theory to Task-oriented Dialogue Systems for Accurately Determining User’s Task Success Ability (Hirai et al., SIGDIAL 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-5/2023.sigdial-1.39.pdf