Neural Conversational QA: Learning to Reason vs Exploiting Patterns
Nikhil Verma, Abhishek Sharma, Dhiraj Madan, Danish Contractor, Harshit Kumar, Sachindra Joshi
Abstract
Neural Conversational QA tasks such as ShARC require systems to answer questions based on the contents of a given passage. On studying recent state-of-the-art models on the ShARC QA task, we found indications that the model(s) learn spurious clues/patterns in the data-set. Further, a heuristic-based program, built to exploit these patterns, had comparative performance to that of the neural models. In this paper we share our findings about the four types of patterns in the ShARC corpus and how the neural models exploit them. Motivated by the above findings, we create and share a modified data-set that has fewer spurious patterns than the original data-set, consequently allowing models to learn better.- Anthology ID:
- 2020.emnlp-main.589
- Volume:
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 7263–7269
- Language:
- URL:
- https://aclanthology.org/2020.emnlp-main.589
- DOI:
- 10.18653/v1/2020.emnlp-main.589
- Cite (ACL):
- Nikhil Verma, Abhishek Sharma, Dhiraj Madan, Danish Contractor, Harshit Kumar, and Sachindra Joshi. 2020. Neural Conversational QA: Learning to Reason vs Exploiting Patterns. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7263–7269, Online. Association for Computational Linguistics.
- Cite (Informal):
- Neural Conversational QA: Learning to Reason vs Exploiting Patterns (Verma et al., EMNLP 2020)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/2020.emnlp-main.589.pdf
- Data
- Neural Conversational QA