@inproceedings{sun-etal-2025-os,
    title = "{OS}-Genesis: Automating {GUI} Agent Trajectory Construction via Reverse Task Synthesis",
    author = "Sun, Qiushi  and
      Cheng, Kanzhi  and
      Ding, Zichen  and
      Jin, Chuanyang  and
      Wang, Yian  and
      Xu, Fangzhi  and
      Wu, Zhenyu  and
      Jia, Chengyou  and
      Chen, Liheng  and
      Liu, Zhoumianze  and
      Kao, Ben  and
      Li, Guohao  and
      He, Junxian  and
      Qiao, Yu  and
      Wu, Zhiyong",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.277/",
    pages = "5555--5579",
    ISBN = "979-8-89176-251-0",
    abstract = "Graphical User Interface (GUI) agents powered by Vision-Language Models (VLMs) have demonstrated human-like computer control capability. Despite their utility in advancing digital automation, the development of such agents faces a critical bottleneck: collecting high-quality trajectory data for training. Common practices for collecting such data rely on human supervision or synthetic data generation through executing pre-defined tasks, which are either resource-intensive or unable to guarantee data quality. Further, these approaches exhibit significant gaps between the generated data and online environments, alongside limited data diversity. To address this issue, we introduce OS-Genesis, a novel GUI data synthesis pipeline that overcomes the challenges above. Unlike prior methods that rely on preset tasks, OS-Genesis reverse engineers the GUI trajectory construction process. Agents first perceive environments and perform step-level interactions, then retrospectively derive high-quality tasks to enable trajectory-level exploration. A trajectory reward model is then employed to ensure the quality of the generated trajectories. We demonstrate that training GUI agents with OS-Genesis significantly improves their performance on highly challenging online benchmarks. In-depth analysis further validates OS-Genesis{'}s cost-effectiveness and its superior data quality and diversity compared to existing synthesis methods."
}