Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification

Pritish Sahu, Karan Sikka, Ajay Divakaran

Abstract
Large Visual Language Models (LVLMs) struggle with hallucinations in visual instruction following task(s). These issues hinder their trustworthiness and real-world applicability. We propose Pelican – a novel framework designed to detect and mitigate hallucinations through claim verification. Pelican first decomposes the visual claim into a chain of sub-claims based on first-order predicates. These sub-claims consists of (predicate, question) pairs and can be conceptualized as nodes of a computational graph. We then use use Program-of-Thought prompting to generate Python code for answering these questions through flexible composition of external tools. Pelican improves over prior work by introducing (1) intermediate variables for precise grounding of object instances, and (2) shared computation for answering the sub-question to enable adaptive corrections and inconsistency identification. We finally use reasoning abilities of LLM to verify the correctness of the the claim by considering the consistency and confidence of the (question, answer) pairs from each sub-claim. Our experiments demonstrate consistent performance improvements over various baseline LVLMs and existing hallucination mitigation approaches across several benchmarks.
Anthology ID:
2024.emnlp-main.470
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8228–8248
Language:
URL:
https://aclanthology.org/2024.emnlp-main.470
DOI:
10.18653/v1/2024.emnlp-main.470
Bibkey:
Cite (ACL):
Pritish Sahu, Karan Sikka, and Ajay Divakaran. 2024. Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8228–8248, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification (Sahu et al., EMNLP 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/dois-2013-emnlp/2024.emnlp-main.470.pdf
PDF Search