xMoCo: Cross Momentum Contrastive Learning for Open-Domain Question Answering

Nan Yang, Furu Wei, Binxing Jiao, Daxing Jiang, Linjun Yang


Abstract
Dense passage retrieval has been shown to be an effective approach for information retrieval tasks such as open domain question answering. Under this paradigm, a dual-encoder model is learned to encode questions and passages separately into vector representations, and all the passage vectors are then pre-computed and indexed, which can be efficiently retrieved by vector space search during inference time. In this paper, we propose a new contrastive learning method called Cross Momentum Contrastive learning (xMoCo), for learning a dual-encoder model for question-passage matching. Our method efficiently maintains a large pool of negative samples like the original MoCo, and by jointly optimizing question-to-passage and passage-to-question matching tasks, enables using separate encoders for questions and passages. We evaluate our method on various open-domain question answering dataset, and the experimental results show the effectiveness of the proposed method.
Anthology ID:
2021.acl-long.477
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Editors:
Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6120–6129
Language:
URL:
https://aclanthology.org/2021.acl-long.477
DOI:
10.18653/v1/2021.acl-long.477
Bibkey:
Cite (ACL):
Nan Yang, Furu Wei, Binxing Jiao, Daxing Jiang, and Linjun Yang. 2021. xMoCo: Cross Momentum Contrastive Learning for Open-Domain Question Answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6120–6129, Online. Association for Computational Linguistics.
Cite (Informal):
xMoCo: Cross Momentum Contrastive Learning for Open-Domain Question Answering (Yang et al., ACL-IJCNLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl-24-ws-corrections/2021.acl-long.477.pdf
Video:
 https://preview.aclanthology.org/naacl-24-ws-corrections/2021.acl-long.477.mp4
Data
Natural QuestionsSQuADTriviaQAWebQuestions