Xueqi Ma
2026
EVOTOOL: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selection
Shuo Yang | Caren Han | Xueqi Ma | Yan Li | Mohammad Reza Ghasemi Madani | Eduard Hovy
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Shuo Yang | Caren Han | Xueqi Ma | Yan Li | Mohammad Reza Ghasemi Madani | Eduard Hovy
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
LLM-based agents depend on effective tool-use policies to solve complex tasks, yet optimizing these policies remains challenging dueto delayed supervision and the difficulty ofcredit assignment in long-horizon trajectories.Existing optimization approaches tend to beeither monolithic, which are prone to entangling behaviors, or single-aspect, which ignorecross-module error propagation. To addressthese limitations, we propose EVOTOOL, a self-evolving framework that optimizes a modulartool-use policy via a gradient-free evolutionary paradigm. EVOTOOL decomposes agent’stool-use policy into four modules, includingPlanner, Selector, Caller, and Synthesizer, anditeratively improves them through three mechanisms. Trajectory-Grounded Blame Attribution uses diagnostic traces to localize failuresto a specific module. Feedback-Guided Targeted Mutation then edits only that modulevia natural-language critique. Diversity-AwarePopulation Selection preserves complementarycandidates to ensure solution diversity. Acrossfour diverse benchmarks, EVOTOOL outperforms strong baselines by over 5 points on bothGPT-4.1 and Qwen3-8B, while achieving superior efficiency and transferability.