Doc-PP: Document Policy Preservation Benchmark for Large Vision-Language Models

Haeun Jang, Hwan Chang, Hwanhee Lee


Abstract
The deployment of Large Vision-Language Models (LVLMs) for real-world document question answering is often constrained by dynamic, user-defined policies that dictate information disclosure based on context. While ensuring adherence to these explicit constraints is critical, existing safety research primarily focuses on implicit social norms or text-only settings, overlooking the complexities of multimodal documents. In this paper, we introduce Doc-PP (Document Policy Preservation Benchmark), a novel benchmark constructed from real-world reports requiring reasoning across heterogeneous visual and textual elements under strict non-disclosure policies. Our evaluation highlights a systemic Reasoning-Induced Safety Gap: models frequently leak sensitive information when answers must be inferred through complex synthesis or aggregated across modalities, effectively circumventing existing safety constraints. Furthermore, we identify that providing extracted text improves perception but inadvertently facilitates leakage. To address these vulnerabilities, we propose DVA (Decompose–Verify–Aggregation), a structural inference framework that decouples reasoning from policy verification. Experimental results demonstrate that DVA significantly outperforms standard prompting defenses, offering a robust baseline for policy-compliant document understanding.
Anthology ID:
2026.findings-acl.832
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
16859–16881
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.832/
DOI:
Bibkey:
Cite (ACL):
Haeun Jang, Hwan Chang, and Hwanhee Lee. 2026. Doc-PP: Document Policy Preservation Benchmark for Large Vision-Language Models. In Findings of the Association for Computational Linguistics: ACL 2026, pages 16859–16881, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Doc-PP: Document Policy Preservation Benchmark for Large Vision-Language Models (Jang et al., Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.832.pdf
Checklist:
 2026.findings-acl.832.checklist.pdf