MemoReader: Large-Scale Reading Comprehension through Neural Memory Controller

Seohyun Back, Seunghak Yu, Sathish Reddy Indurthi, Jihie Kim, Jaegul Choo


Abstract
Machine reading comprehension helps machines learn to utilize most of the human knowledge written in the form of text. Existing approaches made a significant progress comparable to human-level performance, but they are still limited in understanding, up to a few paragraphs, failing to properly comprehend lengthy document. In this paper, we propose a novel deep neural network architecture to handle a long-range dependency in RC tasks. In detail, our method has two novel aspects: (1) an advanced memory-augmented architecture and (2) an expanded gated recurrent unit with dense connections that mitigate potential information distortion occurring in the memory. Our proposed architecture is widely applicable to other models. We have performed extensive experiments with well-known benchmark datasets such as TriviaQA, QUASAR-T, and SQuAD. The experimental results demonstrate that the proposed method outperforms existing methods, especially for lengthy documents.
Anthology ID:
D18-1237
Volume:
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Month:
October-November
Year:
2018
Address:
Brussels, Belgium
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
2131–2140
Language:
URL:
https://aclanthology.org/D18-1237
DOI:
10.18653/v1/D18-1237
Bibkey:
Cite (ACL):
Seohyun Back, Seunghak Yu, Sathish Reddy Indurthi, Jihie Kim, and Jaegul Choo. 2018. MemoReader: Large-Scale Reading Comprehension through Neural Memory Controller. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2131–2140, Brussels, Belgium. Association for Computational Linguistics.
Cite (Informal):
MemoReader: Large-Scale Reading Comprehension through Neural Memory Controller (Back et al., EMNLP 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp-22-attachments/D18-1237.pdf
Attachment:
 D18-1237.Attachment.pdf
Data
SQuADTriviaQA