@inproceedings{wu-etal-2026-mirrorcaptcha,
    title = "{M}irror{CAPTCHA}: Wild {CAPTCHA}, Wild Distribution, Wild Web-based Platform Meet Multimodal {LLM} Agents",
    author = "Wu, Xiangyu  and
      Hu, Yuwei  and
      Cui, Tianyu  and
      Tian, Yueying  and
      Chen, Qing-Guo  and
      Xu, Zhao  and
      Luo, Weihua  and
      Zhang, Kaifu  and
      Yang, Yang  and
      Lu, Jianfeng",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.1431/",
    pages = "31001--31017",
    ISBN = "979-8-89176-390-6",
    abstract = "The path to fully autonomous web agents is currently hindered by a critical bottleneck: their limited ability to handle CAPTCHA. Existing agent benchmarks largely ignore this practical challenge, failing to evaluate an agent{'}s real-world capacity to solve CAPTCHA. To bridge this gap, we conduct a comprehensive analysis of real-world CAPTCHA distributions and introduce MirrorCAPTCHA, a benchmark annotated with Weighted Pass Rate and a newly proposed metric Completion Degree. MirrorCAPTCHA is designed to serve as a ``mirror'' that faithfully reflects the automation capabilities of agents in real scenarios. We filter 2095 websites from Common Crawl, identify the CAPTCHA deployed on these sites, and cluster them into 18 distinct categories using K-means algorithm. To ensure practicality, we extract a web subgraph from Common Crawl covering these websites and use random walks to simulate real-world CAPTCHA encounter frequencies, yielding a realistic measure of agents' ability. Additionally, we develop a lightweight synthetic data pipeline to train Ovis2-Agent-CAPTCHA-8B, which significantly outperforms current state-of-the-art closed-source models on MirrorCAPTCHA, achieving a 9.4{\%} higher average Weighted Pass Rate and a 2.13{\%} higher average Completion Degree than the runner-up, Gemini-2.5-Pro."
}