Abstract
Conversational Question Answering (ConvQA) models aim at answering a question with its relevant paragraph and previous question-answer pairs that occurred during conversation multiple times. To apply such models to a real-world scenario, some existing work uses predicted answers, instead of unavailable ground-truth answers, as the conversation history for inference. However, since these models usually predict wrong answers, using all the predictions without filtering significantly hampers the model performance. To address this problem, we propose to filter out inaccurate answers in the conversation history based on their estimated confidences and uncertainties from the ConvQA model, without making any architectural changes. Moreover, to make the confidence and uncertainty values more reliable, we propose to further calibrate them, thereby smoothing the model predictions. We validate our models, Answer Selection-based realistic Conversation Question Answering, on two standard ConvQA datasets, and the results show that our models significantly outperform relevant baselines. Code is available at: https://github.com/starsuzi/AS-ConvQA.- Anthology ID:
- 2023.eacl-main.35
- Volume:
- Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Andreas Vlachos, Isabelle Augenstein
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 477–490
- Language:
- URL:
- https://aclanthology.org/2023.eacl-main.35
- DOI:
- 10.18653/v1/2023.eacl-main.35
- Cite (ACL):
- Soyeong Jeong, Jinheon Baek, Sung Ju Hwang, and Jong Park. 2023. Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 477–490, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- Realistic Conversational Question Answering with Answer Selection based on Calibrated Confidence and Uncertainty Measurement (Jeong et al., EACL 2023)
- PDF:
- https://preview.aclanthology.org/revert-3132-ingestion-checklist/2023.eacl-main.35.pdf