Abstract
The paper introduces two Russian machine reading comprehension (MRC) datasets, called MuSeRC and RuCoS, which require reasoning over multiple sentences and commonsense knowledge to infer the answer. The former follows the design of MultiRC, while the latter is a counterpart of the ReCoRD dataset. The datasets are included in RussianSuperGLUE, the Russian general language understanding benchmark. We provide a comparative analysis and demonstrate that the proposed tasks are relatively more complex as compared to the original ones for English. Besides, performance results of human solvers and BERT-based models show that MuSeRC and RuCoS represent a challenge for recent advanced neural models. We thus hope to facilitate research in the field of MRC for Russian and prompt the study of multi-hop reasoning in a cross-lingual scenario.- Anthology ID:
- 2020.coling-main.570
- Volume:
- Proceedings of the 28th International Conference on Computational Linguistics
- Month:
- December
- Year:
- 2020
- Address:
- Barcelona, Spain (Online)
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 6481–6497
- Language:
- URL:
- https://aclanthology.org/2020.coling-main.570
- DOI:
- 10.18653/v1/2020.coling-main.570
- Cite (ACL):
- Alena Fenogenova, Vladislav Mikhailov, and Denis Shevelev. 2020. Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6481–6497, Barcelona, Spain (Online). International Committee on Computational Linguistics.
- Cite (Informal):
- Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian (Fenogenova et al., COLING 2020)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2020.coling-main.570.pdf
- Data
- ReCoRD, SberQuAD, SuperGLUE, Taiga Corpus