@inproceedings{lu-etal-2026-eva,
    title = "{EVA}: Evolving Semantic Adversaries for Red-Teaming {GUI} Agents Against Environmental Injection Attacks",
    author = "Lu, Yijie  and
      Zhao, Manman  and
      Ju, Tianjie  and
      Yan, Zihe  and
      Ma, Xinbei  and
      Guo, Yuan  and
      Ding, Daizong  and
      Liu, Gongshen  and
      Zhang, Zhuosheng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1230/",
    pages = "24597--24615",
    ISBN = "979-8-89176-395-1",
    abstract = "Autonomous GUI agents are inherently vulnerable to Environmental Injection Attacks (EIAs). However, existing red-teaming methods face a trade-off between requiring target-specific knowledge and incurring prohibitive computational costs. More fundamentally, a key question remains: $\textit{what factors determine attack success?}$ To answer this, we first analyze two dimensions: visual appearance (e.g., position, size, color) and semantic content. We find that semantic content dominates, while visual variations have negligible impact. Leveraging this insight, we introduce EVA, a framework that evolves payloads exclusively on the semantic dimension via a discovery-deployment pipeline. Experiments demonstrate that EVA significantly outperforms baselines, achieving 59{\%} to 85{\%} average Attack Success Rate (ASR) while evolving benign seeds into successful attacks within 1.18 to 1.71 iterations. This rapid convergence suggests a dense semantic attack space within the model{'}s latent space. Whenever an input falls into this space, the agent becomes inherently vulnerable, exposing a fundamental alignment flaw in current multimodal representations."
}