Investigating person-specific errors in chat-oriented dialogue systems
Koh Mitsuda, Ryuichiro Higashinaka, Tingxuan Li, Sen Yoshida
Abstract
Creating chatbots to behave like real people is important in terms of believability. Errors in general chatbots and chatbots that follow a rough persona have been studied, but those in chatbots that behave like real people have not been thoroughly investigated. We collected a large amount of user interactions of a generation-based chatbot trained from large-scale dialogue data of a specific character, i.e., target person, and analyzed errors related to that person. We found that person-specific errors can be divided into two types: errors in attributes and those in relations, each of which can be divided into two levels: self and other. The correspondence with an existing taxonomy of errors was also investigated, and person-specific errors that should be addressed in the future were clarified.- Anthology ID:
- 2022.acl-short.50
- Volume:
- Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Smaranda Muresan, Preslav Nakov, Aline Villavicencio
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 464–469
- Language:
- URL:
- https://aclanthology.org/2022.acl-short.50
- DOI:
- 10.18653/v1/2022.acl-short.50
- Cite (ACL):
- Koh Mitsuda, Ryuichiro Higashinaka, Tingxuan Li, and Sen Yoshida. 2022. Investigating person-specific errors in chat-oriented dialogue systems. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 464–469, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Investigating person-specific errors in chat-oriented dialogue systems (Mitsuda et al., ACL 2022)
- PDF:
- https://preview.aclanthology.org/improve-issue-templates/2022.acl-short.50.pdf