@inproceedings{cao-etal-2025-pgpo,
    title = "{PGPO}: Enhancing Agent Reasoning via Pseudocode-style Planning Guided Preference Optimization",
    author = "Cao, Zouying  and
      Wang, Runze  and
      Yang, Yifei  and
      Ma, Xinbei  and
      Zhu, Xiaoyong  and
      Zheng, Bo  and
      Zhao, Hai",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/display_plenaries/2025.findings-acl.774/",
    pages = "14966--14985",
    ISBN = "979-8-89176-256-5",
    abstract = "Large Language Model (LLM) agents have demonstrated impressive capabilities in handling complex interactive problems. Existing LLM agents mainly generate natural language plans to guide reasoning, which is verbose and inefficient. NL plans are also tailored to specific tasks and restrict agents' ability to generalize across similar tasks. To this end, we explore pseudocode-style plans (P-code Plan) to capture the structural logic of reasoning. We find that P-code Plan empowers LLM agents with stronger generalization ability and more efficiency. Inspired by this finding, we propose a pseudocode-style $\underline{P}$lanning $\underline{G}$uided $\underline{P}$reference $\underline{O}$ptimization method called PGPO for effective agent learning. With two planning-oriented rewards, PGPO further enhances LLM agents' ability to generate high-quality P-code Plans and subsequent reasoning. Experiments show that PGPO achieves superior performance on representative agent benchmarks and outperforms the current leading baselines. Analyses reveal the advantage of PGPO in reducing action errors and omissions during reasoning."
}