Taewook Hwang
2026
Perceptual Hallucination in Vision–Language Models: Definition, Analysis and Verification
Taewook Hwang | Inbum Heo | Sung Jun Lee | Sangkeun Jung
Findings of the Association for Computational Linguistics: ACL 2026
Taewook Hwang | Inbum Heo | Sung Jun Lee | Sangkeun Jung
Findings of the Association for Computational Linguistics: ACL 2026
Vision-Language Models (VLMs) have demonstrated remarkable performance in document understanding tasks; however, VLMs also suffer from hallucinations inherited from LLMs. While prior work has focused on reasoning-stage hallucinations, the role of visual perception remains underexplored. In this work, we define perceptual hallucination as the phenomenon where VLMs generate information as if perceived, despite absent or damaged visual evidence. To analyze this, we construct DocHallu, a benchmark of 2,671 original–damaged image pairs across three tasks, available at https://huggingface.co/datasets/IB99/DocHallu. Experiments reveal that perceptual hallucination occurs across all models, with higher rates for numerical content than textual content. Activation patching analysis suggests that hallucinations are strongly associated with errors introduced in the vision encoder, which can subsequently propagate and become amplified through the text decoding process. We also demonstrate that LLM-based post-hoc filtering can reduce hallucination exposure by 36% on average, with reductions of up to 88%. This work extends VLM hallucination research by defining, analyzing, and verifying perceptual hallucination in document understanding.
2025
FEAT: A Preference Feedback Dataset through a Cost-Effective Auto-Generation and Labeling Framework for English AI Tutoring
Hyein Seo | Taewook Hwang | Yohan Lee | Sangkeun Jung
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Hyein Seo | Taewook Hwang | Yohan Lee | Sangkeun Jung
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
In English education tutoring, teacher feedback is essential for guiding students. Recently, AI-based tutoring systems have emerged to assist teachers; however, these systems require high-quality and large-scale teacher feedback data, which is both time-consuming and costly to generate manually. In this study, we propose FEAT, a cost-effective framework for generating teacher feedback, and have constructed three complementary datasets: (1) DIRECT-Manual (DM), where both humans and large language models (LLMs) collaboratively generate high-quality teacher feedback, albeit at a higher cost; (2) DIRECT-Generated (DG), an LLM-only generated, cost-effective dataset with lower quality;, and (3) DIRECT-Augmented (DA), primarily based on DG with a small portion of DM added to enhance quality while maintaining cost-efficiency. Experimental results showed that incorporating a small portion of DM (5–10%) into DG leads to superior performance compared to using 100% DM alone.