Abstract
In this paper, we investigate the tendency of end-to-end neural Machine Reading Comprehension (MRC) models to match shallow patterns rather than perform inference-oriented reasoning on RC benchmarks. We aim to test the ability of these systems to answer questions which focus on referential inference. We propose ParallelQA, a strategy to formulate such questions using parallel passages. We also demonstrate that existing neural models fail to generalize well to this setting.- Anthology ID:
- W18-1001
- Volume:
- Proceedings of the Workshop on Generalization in the Age of Deep Learning
- Month:
- June
- Year:
- 2018
- Address:
- New Orleans, Louisiana
- Venue:
- Gen-Deep
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1–7
- Language:
- URL:
- https://aclanthology.org/W18-1001
- DOI:
- 10.18653/v1/W18-1001
- Cite (ACL):
- Soumya Wadhwa, Varsha Embar, Matthias Grabmair, and Eric Nyberg. 2018. Towards Inference-Oriented Reading Comprehension: ParallelQA. In Proceedings of the Workshop on Generalization in the Age of Deep Learning, pages 1–7, New Orleans, Louisiana. Association for Computational Linguistics.
- Cite (Informal):
- Towards Inference-Oriented Reading Comprehension: ParallelQA (Wadhwa et al., Gen-Deep 2018)
- PDF:
- https://preview.aclanthology.org/auto-file-uploads/W18-1001.pdf
- Data
- NewsQA, SQuAD