Does the Generator Mind Its Contexts? An Analysis of Generative Model Faithfulness under Context Transfer
Xinshuo Hu, Dongfang Li, Xiaoguang Li, Yuxiang Wu, Lifeng Shang, Baotian Hu
Abstract
he present study introduces the knowledge-augmented generator, which is specifically designed to produce information that remains grounded in contextual knowledge, regardless of alterations in the context. Previous research has predominantly focused on examining hallucinations stemming from static input, such as in the domains of summarization or machine translation. However, our investigation delves into the faithfulness of generative question answering in the presence of dynamic knowledge. Our objective is to explore the existence of hallucinations arising from parametric memory when contextual knowledge undergoes changes, while also analyzing the underlying causes for their occurrence. In order to efficiently address this issue, we propose a straightforward yet effective measure for detecting such hallucinations. Intriguingly, our investigation uncovers that all models exhibit a tendency to generate previous answers as hallucinations. To gain deeper insights into the underlying causes of this phenomenon, we conduct a series of experiments that verify the critical role played by context in hallucination, both during training and testing, from various perspectives.- Anthology ID:
- 2024.lrec-main.463
- Volume:
- Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
- Venues:
- LREC | COLING
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 5202–5211
- Language:
- URL:
- https://aclanthology.org/2024.lrec-main.463
- DOI:
- Cite (ACL):
- Xinshuo Hu, Dongfang Li, Xiaoguang Li, Yuxiang Wu, Lifeng Shang, and Baotian Hu. 2024. Does the Generator Mind Its Contexts? An Analysis of Generative Model Faithfulness under Context Transfer. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 5202–5211, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- Does the Generator Mind Its Contexts? An Analysis of Generative Model Faithfulness under Context Transfer (Hu et al., LREC-COLING 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.lrec-main.463.pdf