Unsupervised Adaptation of Question Answering Systems via Generative Self-training
Steven Rennie, Etienne Marcheret, Neil Mallinar, David Nahamoo, Vaibhava Goel
Abstract
BERT-era question answering systems have recently achieved impressive performance on several question-answering (QA) tasks. These systems are based on representations that have been pre-trained on self-supervised tasks such as word masking and sentence entailment, using massive amounts of data. Nevertheless, additional pre-training closer to the end-task, such as training on synthetic QA pairs, has been shown to improve performance. While recent work has considered augmenting labelled data and leveraging large unlabelled datasets to generate synthetic QA data, directly adapting to target data has received little attention. In this paper we investigate the iterative generation of synthetic QA pairs as a way to realize unsupervised self adaptation. Motivated by the success of the roundtrip consistency method for filtering generated QA pairs, we present iterative generalizations of the approach, which maximize an approximation of a lower bound on the probability of the adaptation data. By adapting on synthetic QA pairs generated on the target data, our method is able to improve QA systems significantly, using an order of magnitude less synthetic data and training computation than existing augmentation approaches.- Anthology ID:
- 2020.emnlp-main.87
- Volume:
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1148–1157
- Language:
- URL:
- https://aclanthology.org/2020.emnlp-main.87
- DOI:
- 10.18653/v1/2020.emnlp-main.87
- Cite (ACL):
- Steven Rennie, Etienne Marcheret, Neil Mallinar, David Nahamoo, and Vaibhava Goel. 2020. Unsupervised Adaptation of Question Answering Systems via Generative Self-training. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1148–1157, Online. Association for Computational Linguistics.
- Cite (Informal):
- Unsupervised Adaptation of Question Answering Systems via Generative Self-training (Rennie et al., EMNLP 2020)
- PDF:
- https://preview.aclanthology.org/naacl-24-ws-corrections/2020.emnlp-main.87.pdf