Abstract
Machine reading comprehension is a heavily-studied research and test field for evaluating new pre-trained language models (PrLMs) and fine-tuning strategies, and recent studies have enriched the pre-trained language models with syntactic, semantic and other linguistic information to improve the performance of the models. In this paper, we imitate the human reading process in connecting the anaphoric expressions and explicitly leverage the coreference information of the entities to enhance the word embeddings from the pre-trained language model, in order to highlight the coreference mentions of the entities that must be identified for coreference-intensive question answering in QUOREF, a relatively new dataset that is specifically designed to evaluate the coreference-related performance of a model. We use two strategies to fine-tune a pre-trained language model, namely, placing an additional encoder layer after a pre-trained language model to focus on the coreference mentions or constructing a relational graph convolutional network to model the coreference relations. We demonstrate that the explicit incorporation of coreference information in the fine-tuning stage performs better than the incorporation of the coreference information in pre-training a language model.- Anthology ID:
- 2022.acl-long.91
- Volume:
- Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Smaranda Muresan, Preslav Nakov, Aline Villavicencio
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1281–1292
- Language:
- URL:
- https://aclanthology.org/2022.acl-long.91
- DOI:
- 10.18653/v1/2022.acl-long.91
- Cite (ACL):
- Baorong Huang, Zhuosheng Zhang, and Hai Zhao. 2022. Tracing Origins: Coreference-aware Machine Reading Comprehension. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1281–1292, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Tracing Origins: Coreference-aware Machine Reading Comprehension (Huang et al., ACL 2022)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-1/2022.acl-long.91.pdf
- Code
- bright2013/CorefAwareMRC
- Data
- NewsQA, QuAC, Quoref, SQuAD, SearchQA