Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding
Nouha Dziri, Andrea Madotto, Osmar Zaïane, Avishek Joey Bose
Abstract
Dialogue systems powered by large pre-trained language models exhibit an innate ability to deliver fluent and natural-sounding responses. Despite their impressive performance, these models are fitful and can often generate factually incorrect statements impeding their widespread adoption. In this paper, we focus on the task of improving faithfulness and reducing hallucination of neural dialogue systems to known facts supplied by a Knowledge Graph (KG). We propose Neural Path Hunter which follows a generate-then-refine strategy whereby a generated response is amended using the KG. Neural Path Hunter leverages a separate token-level fact critic to identify plausible sources of hallucination followed by a refinement stage that retrieves correct entities by crafting a query signal that is propagated over a k-hop subgraph. We empirically validate our proposed approach on the OpenDialKG dataset (Moon et al., 2019) against a suite of metrics and report a relative improvement of faithfulness over dialogue responses by 20.35% based on FeQA (Durmus et al., 2020). The code is available at https://github.com/nouhadziri/Neural-Path-Hunter.- Anthology ID:
- 2021.emnlp-main.168
- Volume:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2021
- Address:
- Online and Punta Cana, Dominican Republic
- Editors:
- Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2197–2214
- Language:
- URL:
- https://aclanthology.org/2021.emnlp-main.168
- DOI:
- 10.18653/v1/2021.emnlp-main.168
- Cite (ACL):
- Nouha Dziri, Andrea Madotto, Osmar Zaïane, and Avishek Joey Bose. 2021. Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2197–2214, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding (Dziri et al., EMNLP 2021)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/2021.emnlp-main.168.pdf
- Code
- nouhadziri/neural-path-hunter
- Data
- OpenDialKG