A Large Collection of Model-generated Contradictory Responses for Consistency-aware Dialogue Systems

Shiki Sato, Reina Akama, Jun Suzuki, Kentaro Inui

Abstract
Mitigating the generation of contradictory responses poses a substantial challenge in dialogue response generation. The quality and quantity of available contradictory response data play a vital role in suppressing these contradictions, offering two significant benefits. First, having access to large contradiction data enables a comprehensive examination of their characteristics. Second, data-driven methods to mitigate contradictions may be enhanced with large-scale contradiction data for training. Nevertheless, no attempt has been made to build an extensive collection of model-generated contradictory responses. In this paper, we build a large dataset of response generation models’ contradictions for the first time. Then, we acquire valuable insights into the characteristics of model-generated contradictions through an extensive analysis of the collected responses. Lastly, we also demonstrate how this dataset substantially enhances the performance of data-driven contradiction suppression methods.
Anthology ID:
2024.findings-acl.949
Volume:
Findings of the Association for Computational Linguistics: ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
16047–16062
Language:
URL:
https://aclanthology.org/2024.findings-acl.949
DOI:
10.18653/v1/2024.findings-acl.949
Bibkey:
Cite (ACL):
Shiki Sato, Reina Akama, Jun Suzuki, and Kentaro Inui. 2024. A Large Collection of Model-generated Contradictory Responses for Consistency-aware Dialogue Systems. In Findings of the Association for Computational Linguistics: ACL 2024, pages 16047–16062, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
A Large Collection of Model-generated Contradictory Responses for Consistency-aware Dialogue Systems (Sato et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/dois-2013-emnlp/2024.findings-acl.949.pdf
PDF Search