Abstract
State-of-the-art classification and regression models are often not well calibrated, and cannot reliably provide uncertainty estimates, limiting their utility in safety-critical applications such as clinical decision-making. While recent work has focused on calibration of classifiers, there is almost no work in NLP on calibration in a regression setting. In this paper, we quantify the calibration of pre- trained language models for text regression, both intrinsically and extrinsically. We further apply uncertainty estimates to augment training data in low-resource domains. Our experiments on three regression tasks in both self-training and active-learning settings show that uncertainty estimation can be used to increase overall performance and enhance model generalization.- Anthology ID:
- 2022.tacl-1.39
- Volume:
- Transactions of the Association for Computational Linguistics, Volume 10
- Month:
- Year:
- 2022
- Address:
- Cambridge, MA
- Editors:
- Brian Roark, Ani Nenkova
- Venue:
- TACL
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 680–696
- Language:
- URL:
- https://aclanthology.org/2022.tacl-1.39
- DOI:
- 10.1162/tacl_a_00483
- Cite (ACL):
- Yuxia Wang, Daniel Beck, Timothy Baldwin, and Karin Verspoor. 2022. Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression. Transactions of the Association for Computational Linguistics, 10:680–696.
- Cite (Informal):
- Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression (Wang et al., TACL 2022)
- PDF:
- https://preview.aclanthology.org/add_acl24_videos/2022.tacl-1.39.pdf