@inproceedings{xi-etal-2025-agentgym,
    title = "{A}gent{G}ym: Evaluating and Training Large Language Model-based Agents across Diverse Environments",
    author = "Xi, Zhiheng  and
      Ding, Yiwen  and
      Chen, Wenxiang  and
      Hong, Boyang  and
      Guo, Honglin  and
      Wang, Junzhe  and
      Guo, Xin  and
      Yang, Dingwen  and
      Liao, Chenyang  and
      He, Wei  and
      Gao, Songyang  and
      Chen, Lu  and
      Zheng, Rui  and
      Zou, Yicheng  and
      Gui, Tao  and
      Zhang, Qi  and
      Qiu, Xipeng  and
      Huang, Xuanjing  and
      Wu, Zuxuan  and
      Jiang, Yu-Gang",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1355/",
    pages = "27914--27961",
    ISBN = "979-8-89176-251-0",
    abstract = "Large language models (LLMs) have emerged as a promising foundation to build generally-capable agents (LLM-based agents) that can handle multi-turn decision-making tasks across various environments. However, the community lacks a unified interactive framework that covers diverse environments for comprehensive evaluation of agents, and enables exploration and learning for their self-improvement. To address this, we propose AgentGym, a framework featuring 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction. We construct expanded instruction set, high-quality trajectories, and comprehensive benchmarking suite for developing LLM-based agents. Moreover, AgentGym supports interactive exploration and learning for agents through multi-turn interactions and real-time feedback. Based on AgentGym, we take the initial step to develop LLM-based agents that can handle diverse tasks via methods like self-improvement or reinforcement learning. Experimental results show that the trained agents can achieve results comparable to commercial models. We hope our work can help the community develop more advanced LLM-based agents. We release the code, dataset, benchmark, and checkpoints at https://agentgym.github.io/."
}