D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension
Hongyu Li, Xiyuan Zhang, Yibing Liu, Yiming Zhang, Quan Wang, Xiangyang Zhou, Jing Liu, Hua Wu, Haifeng Wang
Abstract
In this paper, we introduce a simple system Baidu submitted for MRQA (Machine Reading for Question Answering) 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models. Our system is built on a framework of pretraining and fine-tuning, namely D-NET. The techniques of pre-trained language models and multi-task learning are explored to improve the generalization of MRC models and we conduct experiments to examine the effectiveness of these strategies. Our system is ranked at top 1 of all the participants in terms of averaged F1 score. Our codes and models will be released at PaddleNLP.- Anthology ID:
- D19-5828
- Volume:
- Proceedings of the 2nd Workshop on Machine Reading for Question Answering
- Month:
- November
- Year:
- 2019
- Address:
- Hong Kong, China
- Editors:
- Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, Danqi Chen
- Venue:
- WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 212–219
- Language:
- URL:
- https://aclanthology.org/D19-5828
- DOI:
- 10.18653/v1/D19-5828
- Cite (ACL):
- Hongyu Li, Xiyuan Zhang, Yibing Liu, Yiming Zhang, Quan Wang, Xiangyang Zhou, Jing Liu, Hua Wu, and Haifeng Wang. 2019. D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension. In Proceedings of the 2nd Workshop on Machine Reading for Question Answering, pages 212–219, Hong Kong, China. Association for Computational Linguistics.
- Cite (Informal):
- D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension (Li et al., 2019)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/D19-5828.pdf