Towards Inference-Oriented Reading Comprehension: ParallelQA

Soumya Wadhwa, Varsha Embar, Matthias Grabmair, Eric Nyberg


Abstract
In this paper, we investigate the tendency of end-to-end neural Machine Reading Comprehension (MRC) models to match shallow patterns rather than perform inference-oriented reasoning on RC benchmarks. We aim to test the ability of these systems to answer questions which focus on referential inference. We propose ParallelQA, a strategy to formulate such questions using parallel passages. We also demonstrate that existing neural models fail to generalize well to this setting.
Anthology ID:
W18-1001
Volume:
Proceedings of the Workshop on Generalization in the Age of Deep Learning
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Venue:
Gen-Deep
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–7
Language:
URL:
https://aclanthology.org/W18-1001
DOI:
10.18653/v1/W18-1001
Bibkey:
Cite (ACL):
Soumya Wadhwa, Varsha Embar, Matthias Grabmair, and Eric Nyberg. 2018. Towards Inference-Oriented Reading Comprehension: ParallelQA. In Proceedings of the Workshop on Generalization in the Age of Deep Learning, pages 1–7, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Towards Inference-Oriented Reading Comprehension: ParallelQA (Wadhwa et al., Gen-Deep 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/starsem-semeval-split/W18-1001.pdf
Data
NewsQASQuAD