Cross-lingual Information Retrieval with BERT
Zhuolin Jiang, Amro El-Jaroudi, William Hartmann, Damianos Karakos, Lingjun Zhao
Abstract
Multiple neural language models have been developed recently, e.g., BERT and XLNet, and achieved impressive results in various NLP tasks including sentence classification, question answering and document ranking. In this paper, we explore the use of the popular bidirectional language model, BERT, to model and learn the relevance between English queries and foreign-language documents in the task of cross-lingual information retrieval. A deep relevance matching model based on BERT is introduced and trained by finetuning a pretrained multilingual BERT model with weak supervision, using home-made CLIR training data derived from parallel corpora. Experimental results of the retrieval of Lithuanian documents against short English queries show that our model is effective and outperforms the competitive baseline approaches.- Anthology ID:
- 2020.clssts-1.5
- Volume:
- Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020)
- Month:
- May
- Year:
- 2020
- Address:
- Marseille, France
- Venue:
- CLSSTS
- SIG:
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 26–31
- Language:
- English
- URL:
- https://aclanthology.org/2020.clssts-1.5
- DOI:
- Cite (ACL):
- Zhuolin Jiang, Amro El-Jaroudi, William Hartmann, Damianos Karakos, and Lingjun Zhao. 2020. Cross-lingual Information Retrieval with BERT. In Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020), pages 26–31, Marseille, France. European Language Resources Association.
- Cite (Informal):
- Cross-lingual Information Retrieval with BERT (Jiang et al., CLSSTS 2020)
- PDF:
- https://preview.aclanthology.org/starsem-semeval-split/2020.clssts-1.5.pdf