@inproceedings{zhao-etal-2025-uncertainty,
    title = "Uncertainty Propagation on {LLM} Agent",
    author = "Zhao, Qiwei  and
      Li, Dong  and
      Liu, Yanchi  and
      Cheng, Wei  and
      Sun, Yiyou  and
      Oishi, Mika  and
      Osaki, Takao  and
      Matsuda, Katsushi  and
      Yao, Huaxiu  and
      Zhao, Chen  and
      Chen, Haifeng  and
      Zhao, Xujiang",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.302/",
    pages = "6064--6073",
    ISBN = "979-8-89176-251-0",
    abstract = "Large language models (LLMs) integrated into multi-step agent systems enable complex decision-making processes across various applications. However, their outputs often lack reliability, making uncertainty estimation crucial. Existing uncertainty estimation methods primarily focus on final-step outputs, which fail to account for cumulative uncertainty over the multi-step decision-making process and the dynamic interactions between agents and their environments. To address these limitations, we propose SAUP (Situation Awareness Uncertainty Propagation), a novel framework that propagates uncertainty through each step of an LLM-based agent{'}s reasoning process. SAUP incorporates situational awareness by assigning situational weights to each step{'}s uncertainty during the propagation. Our method, compatible with various one-step uncertainty estimation techniques, provides a comprehensive and accurate uncertainty measure. Extensive experiments on benchmark datasets demonstrate that SAUP significantly outperforms existing state-of-the-art methods, achieving up to 20{\%} improvement in AUROC."
}