@inproceedings{chuang-etal-2026-toward,
    title = "Toward Scalable Verifiable Reward: Proxy State-Based Evaluation for Multi-turn Tool-Calling {LLM} Agents",
    author = "Chuang, Yun-Shiuan  and
      Kulkarni, Chaitanya  and
      Chiu, Alec M.  and
      Thangali, Avinash  and
      Pan, Zijie  and
      Shekhar, Shivani  and
      Ge, Yirou  and
      Li, Yixi  and
      Kona, Uma  and
      Pang, Linsey  and
      Mehrotra, Prakhar",
    editor = "Li, Yunyao  and
      Rehm, Georg  and
      Tu, Mei",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics ({ACL} 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl/2026.acl-industry.87/",
    pages = "1251--1264",
    ISBN = "979-8-89176-394-4",
    abstract = "Interactive large language model (LLM) agents operating via multi-turn dialogue and multi-step tool calling are increasingly used in production. Benchmarks for these agents must both reliably compare models and yield on-policy training data. Prior agentic benchmarks (e.g., tau-bench, tau{\textasciicircum}2-bench, AppWorld) rely on fully deterministic backends, which are costly to build and iterate. We propose Proxy State-Based Evaluation, an LLM-driven simulation framework that preserves final state-based evaluation without a deterministic database. Specifically, a scenario specifies the user goal, user/system facts, expected final state, and expected agent behavior, and an LLM state tracker infers a structured proxy state from the full interaction trace. LLM judges then verify goal completion and detect tool/user hallucinations against scenario constraints. Empirically, our benchmark produces stable, model-differentiating rankings across families and inference-time reasoning efforts, and its on- and off-policy rollouts provide supervision that transfers to unseen scenarios. Careful scenario specification yields near-zero simulator hallucination rates, as supported by ablation studies. The framework also supports sensitivity analyses over user personas. Human-LLM judge agreement exceeds 90{\%}, indicating reliable automated evaluation. Overall, proxy state-based evaluation offers a practical, scalable alternative to deterministic agentic benchmarks for industrial LLM agents."
}