@inproceedings{shahariar-etal-2026-pii,
    title = "{PII}-{V}is{B}ench: Evaluating Personally Identifiable Information Safety in Vision Language Models Along a Continuum of Visibility",
    author = "Shahariar, G M  and
      Al Nazi, Zabir  and
      Bhuiyan, Md Olid Hasan  and
      Shi, Zhouxing",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.501/",
    pages = "10294--10316",
    ISBN = "979-8-89176-395-1",
    abstract = "Vision Language Models (VLMs) are increasingly integrated into privacy-critical domains, yet existing evaluations of personally identifiable information (PII) leakage largely treat privacy as a static extraction task and ignore how a subject{'}s online presence{---}the volume of their data available online{---}influences privacy alignment. We introduce **PII-VisBench**, a novel benchmark containing 4,000 unique probes designed to evaluate VLM safety through the *continuum of online presence*. The benchmark stratifies 200 subjects into four visibility categories: *high, medium, low,* and *zero*{---}based on the extent and nature of their information available online. We evaluate 18 open-source VLMs (0.3B{--}32B) based on two key metrics: percentage of PII probing queries refused (*Refusal Rate*) and the fraction of non-refusal responses flagged for containing PII (*Conditional PII Disclosure Rate*). Across models, we observe a consistent pattern: refusals increase and PII disclosures decrease (9.10{\%} high $\rightarrow$ 5.34{\%} low) as subject visibility drops. We identify that models are more likely to disclose PII for high-visibility subjects, alongside substantial model-family heterogeneity and PII-type disparities. Finally, paraphrasing and jailbreak-style prompts expose attack- and model-dependent failures, motivating visibility-aware safety evaluation and training interventions."
}