@inproceedings{zeng-etal-2025-itool,
    title = "i{T}ool: Reinforced Fine-Tuning with Dynamic Deficiency Calibration for Advanced Tool Use",
    author = "Zeng, Yirong  and
      Ding, Xiao  and
      Wang, Yuxian  and
      Liu, Weiwen  and
      Hou, Yutai  and
      Ning, Wu  and
      Huang, Xu  and
      Tang, Duyu  and
      Tu, Dandan  and
      Qin, Bing  and
      Liu, Ting",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.701/",
    pages = "13901--13916",
    ISBN = "979-8-89176-332-6",
    abstract = "Augmenting large language models (LLMs) with external tools is a promising approach to enhance their capabilities, especially for complex tasks. Synthesizing tool-use data through real-world simulations is an effective way to achieve this. However, our investigation reveals that training gains significantly decay as synthetic data increases. The model struggles to benefit from more synthetic data, and it can not equip the model with advanced tool-use capabilities in complex scenarios. Moreover, we discovered that the above limitation usually manifests as a fragment deficiency (i.e., parameter errors) in response. To this end, we propose an iterative reinforced fine-tuning strategy designed to alleviate this limitation. This strategy involves: (1) enhancing the diversity of response for synthetic data through path exploration of Monte Carlo Tree Search. (2) iteratively pinpointing the model{'}s deficiency by constructing fine-grained preference pairs, and then improving it by preference optimization algorithms for targeted improvement. The experiments show that our method achieves 13.11{\%} better performance than the same-size base model. It achieves an improvement of 6.5{\%} in complex scenarios compared to the baseline, and it also outperforms larger open-source and closed-source models."
}