@inproceedings{wang-etal-2026-measure,
    title = "Measure Twice, Click Once: Co-evolving Proposer and Visual Critic via Reinforcement Learning for {GUI} Grounding",
    author = "Wang, Wenkai  and
      Li, Xiyun  and
      Guo, Hongcan  and
      Yu, Wenhao  and
      Fang, Tianqing  and
      Mi, Haitao  and
      Yu, Dong  and
      Zhang, Shengyu",
    editor = "Liakata, Maria  and
      Moreira, Viviane P.  and
      Zhang, Jiajun  and
      Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-long.960/",
    pages = "20964--20984",
    ISBN = "979-8-89176-390-6",
    abstract = "Graphical User Interface (GUI) grounding requires mapping natural language instructions to precise pixel coordinates. However, due to visually homogeneous elements and dense layouts, models typically grasp semantic intent yet fail to achieve precise localization. While scaling sampling attempts (Pass@k) reveals potential gains, static self-consistency strategies derived from geometric clustering often yield limited improvements, as the model{'}s predictions tend to be spatially dispersed. In this paper, we propose replacing static consistency strategies with a learnable selection mechanism that selects the optimal target by critiquing its own proposals rendered on the screenshot. Given the significant disparity between the model{'}s grounding and critiquing capabilities, we propose a co-evolving Propose-and-Critic framework. To jointly optimize these, we introduce a maturity-aware adaptive co-evolutionary reinforcement learning paradigm. This approach dynamically balances the training objectives of proposer and critic, where the diversity of the proposer{'}s outputs enhances critic robustness, while the critic{'}s maturing discrimination capability conversely unlocks the proposer{'}s potential for extensive spatial exploration, fostering the mutual reinforcement and co-evolution of both capabilities, thereby ensuring generalizability to adapt to diverse and complex interface layouts. Extensive experiments over 6 benchmarks show that our method significantly enhances both capabilities."
}