The 2019 BBN Cross-lingual Information Retrieval System
Le Zhang, Damianos Karakos, William Hartmann, Manaj Srivastava, Lee Tarlin, David Akodes, Sanjay Krishna Gouda, Numra Bathool, Lingjun Zhao, Zhuolin Jiang, Richard Schwartz, John Makhoul
Abstract
In this paper, we describe a cross-lingual information retrieval (CLIR) system that, given a query in English, and a set of audio and text documents in a foreign language, can return a scored list of relevant documents, and present findings in a summary form in English. Foreign audio documents are first transcribed by a state-of-the-art pretrained multilingual speech recognition model that is finetuned to the target language. For text documents, we use multiple multilingual neural machine translation (MT) models to achieve good translation results, especially for low/medium resource languages. The processed documents and queries are then scored using a probabilistic CLIR model that makes use of the probability of translation from GIZA translation tables and scores from a Neural Network Lexical Translation Model (NNLTM). Additionally, advanced score normalization, combination, and thresholding schemes are employed to maximize the Average Query Weighted Value (AQWV) scores. The CLIR output, together with multiple translation renderings, are selected and translated into English snippets via a summarization model. Our turnkey system is language agnostic and can be quickly trained for a new low-resource language in few days.- Anthology ID:
- 2020.clssts-1.8
- Volume:
- Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020)
- Month:
- May
- Year:
- 2020
- Address:
- Marseille, France
- Editors:
- Kathy McKeown, Douglas W. Oard, Elizabeth, Richard Schwartz
- Venue:
- CLSSTS
- SIG:
- Publisher:
- European Language Resources Association
- Note:
- Pages:
- 44–51
- Language:
- English
- URL:
- https://aclanthology.org/2020.clssts-1.8
- DOI:
- Cite (ACL):
- Le Zhang, Damianos Karakos, William Hartmann, Manaj Srivastava, Lee Tarlin, David Akodes, Sanjay Krishna Gouda, Numra Bathool, Lingjun Zhao, Zhuolin Jiang, Richard Schwartz, and John Makhoul. 2020. The 2019 BBN Cross-lingual Information Retrieval System. In Proceedings of the workshop on Cross-Language Search and Summarization of Text and Speech (CLSSTS2020), pages 44–51, Marseille, France. European Language Resources Association.
- Cite (Informal):
- The 2019 BBN Cross-lingual Information Retrieval System (Zhang et al., CLSSTS 2020)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-2/2020.clssts-1.8.pdf