@inproceedings{ying-etal-2026-securewebarena,
    title = "{S}ecure{W}eb{A}rena: A Holistic Security Evaluation Benchmark for {LVLM}-based Web Agents",
    author = "Ying, Zonghao  and
      Shao, Yangguang  and
      Gan, Jianle  and
      Xu, Gan  and
      Zhang, Wenxin  and
      Zou, Quanchen  and
      Shi, Junzheng  and
      Yin, Zhenfei  and
      Zhang, Mingchuan  and
      Liu, Aishan  and
      Liu, Xianglong",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.582/",
    pages = "11986--11998",
    ISBN = "979-8-89176-395-1",
    abstract = "Large vision{--}language model (LVLM)-based web agents are emerging as powerful automation tools but face severe security risks in real-world deployment. Existing benchmarks offer limited coverage, typically isolating user-level prompts from environmental threats, thus failing to capture the full spectrum of vulnerabilities. To address this, we present SecureWebArena, the first holistic security benchmark for web agents. SecureWebArena features a unified suite of six realistic web environments with 2,970 adversarial trajectories, covering a structured taxonomy of six attack vectors that span both user-level and environment-level manipulations. Crucially, we introduce a multi-layered evaluation protocol that dissects agent failures across internal reasoning, behavioral execution, and task outcomes, enabling fine-grained risk analysis beyond simple success metrics. Experiments on 9 representative LVLMs reveal universal vulnerabilities to subtle manipulations and uncover significant trade-offs between model specialization and security. SecureWebArena establishes a rigorous foundation for advancing the development of trustworthy web agents."
}