Generative Context Pair Selection for Multi-hop Question Answering
Dheeru Dua, Cicero Nogueira dos Santos, Patrick Ng, Ben Athiwaratkun, Bing Xiang, Matt Gardner, Sameer Singh
Abstract
Compositional reasoning tasks such as multi-hop question answering require models to learn how to make latent decisions using only weak supervision from the final answer. Crowdsourced datasets gathered for these tasks, however, often contain only a slice of the underlying task distribution, which can induce unanticipated biases such as shallow word overlap between the question and context. Recent works have shown that discriminative training results in models that exploit these underlying biases to achieve a better held-out performance, without learning the right way to reason. We propose a generative context selection model for multi-hop QA that reasons about how the given question could have been generated given a context pair and not just independent contexts. We show that on HotpotQA, while being comparable to the state-of-the-art answering performance, our proposed generative passage selection model has a better performance (4.9% higher than baseline) on adversarial held-out set which tests robustness of model’s multi-hop reasoning capabilities.- Anthology ID:
- 2021.emnlp-main.561
- Volume:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2021
- Address:
- Online and Punta Cana, Dominican Republic
- Editors:
- Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 7009–7015
- Language:
- URL:
- https://aclanthology.org/2021.emnlp-main.561
- DOI:
- 10.18653/v1/2021.emnlp-main.561
- Cite (ACL):
- Dheeru Dua, Cicero Nogueira dos Santos, Patrick Ng, Ben Athiwaratkun, Bing Xiang, Matt Gardner, and Sameer Singh. 2021. Generative Context Pair Selection for Multi-hop Question Answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7009–7015, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Generative Context Pair Selection for Multi-hop Question Answering (Dua et al., EMNLP 2021)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-5/2021.emnlp-main.561.pdf
- Data
- HotpotQA, WikiHop