A Robust Self-Learning Framework for Cross-Lingual Text Classification

Xin Dong, Gerard de Melo

Abstract
Based on massive amounts of data, recent pretrained contextual representation models have made significant strides in advancing a number of different English NLP tasks. However, for other languages, relevant training data may be lacking, while state-of-the-art deep learning methods are known to be data-hungry. In this paper, we present an elegantly simple robust self-learning framework to include unlabeled non-English samples in the fine-tuning process of pretrained multilingual representation models. We leverage a multilingual model’s own predictions on unlabeled non-English data in order to obtain additional information that can be used during further fine-tuning. Compared with original multilingual models and other cross-lingual classification models, we observe significant gains in effectiveness on document and sentiment classification for a range of diverse languages.
Anthology ID:
D19-1658
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
6306–6310
Language:
URL:
https://preview.aclanthology.org/icon-24-ingestion/D19-1658/
DOI:
10.18653/v1/D19-1658
Bibkey:
Cite (ACL):
Xin Dong and Gerard de Melo. 2019. A Robust Self-Learning Framework for Cross-Lingual Text Classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6306–6310, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
A Robust Self-Learning Framework for Cross-Lingual Text Classification (Dong & de Melo, EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/icon-24-ingestion/D19-1658.pdf
PDF Search Fix data