Post-hoc Utterance Refining Method by Entity Mining for Faithful Knowledge Grounded Conversations

Yoonna Jang, Suhyune Son, Jeongwoo Lee, Junyoung Son, Yuna Hur, Jungwoo Lim, Hyeonseok Moon, Kisu Yang, Heuiseok Lim


Abstract
Despite the striking advances in recent language generation performance, model-generated responses have suffered from the chronic problem of hallucinations that are either untrue or unfaithful to a given source. Especially in the task of knowledge grounded conversation, the models are required to generate informative responses, but hallucinated utterances lead to miscommunication. In particular, entity-level hallucination that causes critical misinformation and undesirable conversation is one of the major concerns. To address this issue, we propose a post-hoc refinement method called REM. It aims to enhance the quality and faithfulness of hallucinated utterances by refining them based on the source knowledge. If the generated utterance has a low source-faithfulness score with the given knowledge, REM mines the key entities in the knowledge and implicitly uses them for refining the utterances. We verify that our method reduces entity hallucination in the utterance. Also, we show the adaptability and efficacy of REM with extensive experiments and generative results. Our code is available at https://github.com/YOONNAJANG/REM.
Anthology ID:
2023.emnlp-main.295
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4844–4861
Language:
URL:
https://aclanthology.org/2023.emnlp-main.295
DOI:
10.18653/v1/2023.emnlp-main.295
Bibkey:
Cite (ACL):
Yoonna Jang, Suhyune Son, Jeongwoo Lee, Junyoung Son, Yuna Hur, Jungwoo Lim, Hyeonseok Moon, Kisu Yang, and Heuiseok Lim. 2023. Post-hoc Utterance Refining Method by Entity Mining for Faithful Knowledge Grounded Conversations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4844–4861, Singapore. Association for Computational Linguistics.
Cite (Informal):
Post-hoc Utterance Refining Method by Entity Mining for Faithful Knowledge Grounded Conversations (Jang et al., EMNLP 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-1/2023.emnlp-main.295.pdf
Video:
 https://preview.aclanthology.org/nschneid-patch-1/2023.emnlp-main.295.mp4