@inproceedings{xie-etal-2025-gui,
    title = "{GUI}-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for {GUI} Agent",
    author = "Xie, Bin  and
      Shao, Rui  and
      Chen, Gongwei  and
      Zhou, Kaiwen  and
      Li, Yinchuan  and
      Liu, Jie  and
      Zhang, Min  and
      Nie, Liqiang",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.282/",
    pages = "5650--5667",
    ISBN = "979-8-89176-251-0",
    abstract = "GUI automation faces critical challenges in dynamic environments. MLLMs suffer from two key issues: misinterpreting UI components and outdated knowledge. Traditional fine-tuning methods are costly for app-specific knowledge updates. We propose GUI-explorer, a training-free GUI agent that incorporates two fundamental mechanisms: $\textbf{(1) Autonomous Exploration of Function-aware Trajectory}$. To comprehensively cover all application functionalities, we design a $\textbf{Function-aware Task Goal Generator}$ that automatically constructs exploration goals by analyzing GUI structural information (e.g., screenshots and activity hierarchies). This enables systematic exploration to collect diverse trajectories. $\textbf{(2) Unsupervised Mining of Transition-aware Knowledge}$. To establish precise screen-operation logic, we develop a $\textbf{Transition-aware Knowledge Extractor}$ that extracts effective screen-operation logic through unsupervised analysis the state transition of structured interaction triples (observation, action, outcome). This eliminates the need for human involvement in knowledge extraction. With a task success rate of 53.7{\%} on SPA-Bench and 47.4{\%} on AndroidWorld, GUI-explorer shows significant improvements over SOTA agents. It requires no parameter updates for new apps. GUI-explorer is open-sourced and publicly available at https://github.com/JiuTian-VL/GUI-explorer."
}