@inproceedings{patidar-etal-2019-monolingual,
    title = "From Monolingual to Multilingual {FAQ} Assistant using Multilingual Co-training",
    author = "Patidar, Mayur  and
      Kumari, Surabhi  and
      Patwardhan, Manasi  and
      Karande, Shirish  and
      Agarwal, Puneet  and
      Vig, Lovekesh  and
      Shroff, Gautam",
    editor = "Cherry, Colin  and
      Durrett, Greg  and
      Foster, George  and
      Haffari, Reza  and
      Khadivi, Shahram  and
      Peng, Nanyun  and
      Ren, Xiang  and
      Swayamdipta, Swabha",
    booktitle = "Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)",
    month = nov,
    year = "2019",
    address = "Hong Kong, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/D19-6113/",
    doi = "10.18653/v1/D19-6113",
    pages = "115--123",
    abstract = "Recent research on cross-lingual transfer show state-of-the-art results on benchmark datasets using pre-trained language representation models (PLRM) like BERT. These results are achieved with the traditional training approaches, such as Zero-shot with no data, Translate-train or Translate-test with machine translated data. In this work, we propose an approach of {\textquotedblleft}Multilingual Co-training{\textquotedblright} (MCT) where we augment the expert annotated dataset in the source language (English) with the corresponding machine translations in the target languages (e.g. Arabic, Spanish) and fine-tune the PLRM jointly. We observe that the proposed approach provides consistent gains in the performance of BERT for multiple benchmark datasets (e.g. 1.0{\%} gain on MLDocs, and 1.2{\%} gain on XNLI over translate-train with BERT), while requiring a single model for multiple languages. We further consider a FAQ dataset where the available English test dataset is translated by experts into Arabic and Spanish. On such a dataset, we observe an average gain of 4.9{\%} over all other cross-lingual transfer protocols with BERT. We further observe that domain-specific joint pre-training of the PLRM using HR policy documents in English along with the machine translations in the target languages, followed by the joint finetuning, provides a further improvement of 2.8{\%} in average accuracy."
}