@inproceedings{vazhentsev-etal-2025-unconditional,
    title = "Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Models",
    author = "Vazhentsev, Artem  and
      Fadeeva, Ekaterina  and
      Xing, Rui  and
      Kuzmin, Gleb  and
      Lazichny, Ivan  and
      Panchenko, Alexander  and
      Nakov, Preslav  and
      Baldwin, Timothy  and
      Panov, Maxim  and
      Shelmanov, Artem",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1807/",
    pages = "35661--35682",
    ISBN = "979-8-89176-332-6",
    abstract = "Uncertainty quantification (UQ) has emerged as a promising approach for detecting hallucinations and low-quality output of Large Language Models (LLMs). However, obtaining proper uncertainty scores is complicated by the conditional dependency between the generation steps of an autoregressive LLM, because it is hard to model it explicitly. Here, we propose to learn this dependency from attention-based features. In particular, we train a regression model that leverages LLM attention maps, probabilities on the current generation step, and recurrently computed uncertainty scores from previously generated tokens. To incorporate the recurrent features, we also suggest a two-staged training procedure. Our experimental evaluation on ten datasets and three LLMs shows that the proposed method is highly effective for selective generation, achieving substantial improvements over rivaling unsupervised and supervised approaches."
}