N-best Response-based Analysis of Contradiction-awareness in Neural Response Generation Models
Shiki Sato, Reina Akama, Hiroki Ouchi, Ryoko Tokuhisa, Jun Suzuki, Kentaro Inui
Abstract
Avoiding the generation of responses that contradict the preceding context is a significant challenge in dialogue response generation. One feasible method is post-processing, such as filtering out contradicting responses from a resulting n-best response list. In this scenario, the quality of the n-best list considerably affects the occurrence of contradictions because the final response is chosen from this n-best list. This study quantitatively analyzes the contextual contradiction-awareness of neural response generation models using the consistency of the n-best lists. Particularly, we used polar questions as stimulus inputs for concise and quantitative analyses. Our tests illustrate the contradiction-awareness of recent neural response generation models and methodologies, followed by a discussion of their properties and limitations.- Anthology ID:
- 2022.sigdial-1.60
- Volume:
- Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue
- Month:
- September
- Year:
- 2022
- Address:
- Edinburgh, UK
- Editors:
- Oliver Lemon, Dilek Hakkani-Tur, Junyi Jessy Li, Arash Ashrafzadeh, Daniel Hernández Garcia, Malihe Alikhani, David Vandyke, Ondřej Dušek
- Venue:
- SIGDIAL
- SIG:
- SIGDIAL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 637–644
- Language:
- URL:
- https://preview.aclanthology.org/add_missing_videos/2022.sigdial-1.60/
- DOI:
- 10.18653/v1/2022.sigdial-1.60
- Cite (ACL):
- Shiki Sato, Reina Akama, Hiroki Ouchi, Ryoko Tokuhisa, Jun Suzuki, and Kentaro Inui. 2022. N-best Response-based Analysis of Contradiction-awareness in Neural Response Generation Models. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 637–644, Edinburgh, UK. Association for Computational Linguistics.
- Cite (Informal):
- N-best Response-based Analysis of Contradiction-awareness in Neural Response Generation Models (Sato et al., SIGDIAL 2022)
- PDF:
- https://preview.aclanthology.org/add_missing_videos/2022.sigdial-1.60.pdf
- Code
- shiki-sato/nbest-contradiction-analysis