Byeonggeuk Lim
2026
Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarks
Junehyoung Kwon | MiHyeon Kim | Eunju Lee | JungMin Yun | Byeonggeuk Lim | YoungBin Kim
Findings of the Association for Computational Linguistics: ACL 2026
Junehyoung Kwon | MiHyeon Kim | Eunju Lee | JungMin Yun | Byeonggeuk Lim | YoungBin Kim
Findings of the Association for Computational Linguistics: ACL 2026
While Large Vision-Language Models (LVLMs) offer powerful capabilities, they pose privacy risks by unintentionally memorizing sensitive personal information. Current unlearning benchmarks attempt to mitigate this using fictitious identities but overlook a critical stage 1 failure: models fail to effectively memorize target information initially, rendering subsequent unlearning evaluations unreliable. Diagnosing under-memorization and the multi-hop curse as root causes, we introduce ReMem, a Reliable Multi-hop and Multi-image Memorization Benchmark. ReMem ensures robust foundational learning through principled data scaling, reasoning-aware QA pairs, and diverse visual contexts. Additionally, we propose a novel Exposure metric to quantify the depth of information erasure from the model’s internal probability distribution. Extensive experiments demonstrate that ReMem provides a rigorous and trustworthy framework for diagnosing both learning and unlearning behaviors in LVLMs.
Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs
Byeonggeuk Lim | JungMin Yun | Junehyoung Kwon | Kyeonghyun Kim | YoungBin Kim
Findings of the Association for Computational Linguistics: ACL 2026
Byeonggeuk Lim | JungMin Yun | Junehyoung Kwon | Kyeonghyun Kim | YoungBin Kim
Findings of the Association for Computational Linguistics: ACL 2026
Large Vision-Language Models (LVLMs) frequently suffer from hallucinations. Existing preference learning-based approaches largely rely on proprietary models to construct preference datasets. We identify that this reliance introduces a distributional mismatch between the proprietary and target models that hinders efficient alignment. To address this, we propose Alignment via VErified Self-correction DPO (AVES-DPO), a framework that aligns LVLMs using in-distribution data derived from the model’s intrinsic knowledge. Our approach employs a consensus-based verification mechanism to diagnose diverse hallucinations and guides the model to self-correct, thereby generating preference pairs strictly compatible with its internal distribution. Extensive experiments demonstrate that AVES-DPO surpasses existing baselines in hallucination mitigation while requiring only 5.2k samples.