Conformal Prediction for Natural Language Processing: A Survey
Margarida Campos, António Farinhas, Chrysoula Zerva, Mário A. T. Figueiredo, André F. T. Martins
Abstract
The rapid proliferation of large language models and natural language processing (NLP) applications creates a crucial need for uncertainty quantification to mitigate risks such as Hallucinations and to enhance decision-making reliability in critical applications. Conformal prediction is emerging as a theoretically sound and practically useful framework, combining flexibility with strong statistical guarantees. Its model-agnostic and distribution-free nature makes it particularly promising to address the current shortcomings of NLP systems that stem from the absence of uncertainty quantification. This paper provides a comprehensive survey of conformal prediction techniques, their guarantees, and existing applications in NLP, pointing to directions for future research and open challenges.- Anthology ID:
- 2024.tacl-1.82
- Volume:
- Transactions of the Association for Computational Linguistics, Volume 12
- Month:
- Year:
- 2024
- Address:
- Cambridge, MA
- Venue:
- TACL
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 1497–1516
- Language:
- URL:
- https://preview.aclanthology.org/add-emnlp-2024-awards/2024.tacl-1.82/
- DOI:
- 10.1162/tacl_a_00715
- Cite (ACL):
- Margarida Campos, António Farinhas, Chrysoula Zerva, Mário A. T. Figueiredo, and André F. T. Martins. 2024. Conformal Prediction for Natural Language Processing: A Survey. Transactions of the Association for Computational Linguistics, 12:1497–1516.
- Cite (Informal):
- Conformal Prediction for Natural Language Processing: A Survey (Campos et al., TACL 2024)
- PDF:
- https://preview.aclanthology.org/add-emnlp-2024-awards/2024.tacl-1.82.pdf