Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension

Kai Sun, Dian Yu, Dong Yu, Claire Cardie


Abstract
Machine reading comprehension tasks require a machine reader to answer questions relevant to the given document. In this paper, we present the first free-form multiple-Choice Chinese machine reading Comprehension dataset (C3), containing 13,369 documents (dialogues or more formally written mixed-genre texts) and their associated 19,577 multiple-choice free-form questions collected from Chinese-as-a-second-language examinations. We present a comprehensive analysis of the prior knowledge (i.e., linguistic, domain-specific, and general world knowledge) needed for these real-world problems. We implement rule-based and popular neural methods and find that there is still a significant performance gap between the best performing model (68.5%) and human readers (96.0%), especiallyon problems that require prior knowledge. We further study the effects of distractor plausibility and data augmentation based on translated relevant datasets for English on model performance. We expect C3 to present great challenges to existing systems as answering 86.8% of questions requires both knowledge within and beyond the accompanying document, and we hope that C3 can serve as a platform to study how to leverage various kinds of prior knowledge to better understand a given written or orally oriented text. C3 is available at https://dataset.org/c3/.
Anthology ID:
2020.tacl-1.10
Volume:
Transactions of the Association for Computational Linguistics, Volume 8
Month:
Year:
2020
Address:
Cambridge, MA
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
141–155
Language:
URL:
https://aclanthology.org/2020.tacl-1.10
DOI:
10.1162/tacl_a_00305
Bibkey:
Cite (ACL):
Kai Sun, Dian Yu, Dong Yu, and Claire Cardie. 2020. Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension. Transactions of the Association for Computational Linguistics, 8:141–155.
Cite (Informal):
Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension (Sun et al., TACL 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/2020.tacl-1.10.pdf
Data
C3DuReader