Zhangchi Xue
2026
ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback
Yutao Mou | Zhangchi Xue | Lijun Li | Peiyang Liu | Shikun Zhang | Wei Ye | Jing Shao
Findings of the Association for Computational Linguistics: ACL 2026
Yutao Mou | Zhangchi Xue | Lijun Li | Peiyang Liu | Shikun Zhang | Wei Ye | Jing Shao
Findings of the Association for Computational Linguistics: ACL 2026
While LLM-based agents can interact with environments via invoking external tools, their expanded capabilities also amplify security risks. Monitoring step-level tool invocation behaviors in real time and proactively intervening before unsafe execution is critical for agent deployment, yet remains underexplored. In this work, we first construct TS-Bench, a novel benchmark for step-level tool invocation safety detection in LLM agents. We then develop a guardrail model, TS-Guard, using multi-task reinforcement learning. The model proactively detects unsafe tool invocation actions before execution by reasoning over the interaction history. It assesses request harmfulness and action–attack correlations, producing interpretable and generalizable safety judgments and feedback. Furthermore, We introduce TS-Flow, a guardrail-feedback-driven reasoning framework for LLM agents, which reduces harmful tool invocations of ReAct-style agents by 65% on average and improves benign task completion by approximately 10% under prompt injection attacks.