@inproceedings{wang-etal-2024-conu,
    title = "{C}on{U}: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees",
    author = "Wang, Zhiyuan  and
      Duan, Jinhao  and
      Cheng, Lu  and
      Zhang, Yue  and
      Wang, Qingni  and
      Shi, Xiaoshuang  and
      Xu, Kaidi  and
      Shen, Heng Tao  and
      Zhu, Xiaofeng",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/add-emnlp-2024-awards/2024.findings-emnlp.404/",
    doi = "10.18653/v1/2024.findings-emnlp.404",
    pages = "6886--6898",
    abstract = "Uncertainty quantification (UQ) in natural language generation (NLG) tasks remains an open challenge, exacerbated by the closed-source nature of the latest large language models (LLMs). This study investigates applying conformal prediction (CP), which can transform any heuristic uncertainty notion into rigorous prediction sets, to black-box LLMs in open-ended NLG tasks. We introduce a novel uncertainty measure based on self-consistency theory, and then develop a conformal uncertainty criterion by integrating the uncertainty condition aligned with correctness into the CP algorithm. Empirical evaluations indicate that our uncertainty measure outperforms prior state-of-the-art methods. Furthermore, we achieve strict control over the correctness coverage rate utilizing 7 popular LLMs on 4 free-form NLG datasets, spanning general-purpose and medical scenarios. Additionally, the calibrated prediction sets with small size further highlights the efficiency of our method in providing trustworthy guarantees for practical open-ended NLG applications."
}