@inproceedings{hu-etal-2025-os,
    title = "{OS} Agents: A Survey on {MLLM}-based Agents for Computer, Phone and Browser Use",
    author = "Hu, Xueyu  and
      Xiong, Tao  and
      Yi, Biao  and
      Wei, Zishu  and
      Xiao, Ruixuan  and
      Chen, Yurun  and
      Ye, Jiasheng  and
      Tao, Meiling  and
      Zhou, Xiangxin  and
      Zhao, Ziyu  and
      Li, Yuhuai  and
      Xu, Shengze  and
      Wang, Shenzhi  and
      Xu, Xinchen  and
      Qiao, Shuofei  and
      Wang, Zhaokai  and
      Kuang, Kun  and
      Zeng, Tieyong  and
      Wang, Liang  and
      Li, Jiwei  and
      Jiang, Yuchen Eleanor  and
      Zhou, Wangchunshu  and
      Wang, Guoyin  and
      Yin, Keting  and
      Zhao, Zhou  and
      Yang, Hongxia  and
      Wu, Fan  and
      Zhang, Shengyu  and
      Wu, Fei",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.369/",
    pages = "7436--7465",
    ISBN = "979-8-89176-251-0",
    abstract = "The dream to create AI assistants as capable and versatile as the fictional J.A.R.V.I.S from Iron Man has long captivated imaginations. With the evolution of multi-modal large language models ((M)LLMs), this dream is closer to reality, as (M)LLM-based Agents using computers, mobile phones and web browsers by operating within the environments and interfaces (e.g., Graphical User Interface (GUI) and Command Line Interface (CLI)) provided by operating systems (OS) to automate tasks have significantly advanced. This paper presents a comprehensive survey on these advanced agents, designated as OS Agents. We begin by elucidating the fundamentals of OS Agents, exploring their key components and capabilities. We then examine methodologies for constructing OS Agents, focusing on domain-specific foundation models and agent frameworks. A detailed review of evaluation metrics and benchmarks highlights how OS Agents are assessed across diverse platforms and tasks. Finally, we discuss current challenges and identify promising directions for future research. An open-source GitHub repository is maintained as a dynamic resource to foster further innovation in this field."
}