Meta-Learning for Effective Multi-task and Multilingual Modelling
Ishan Tarunesh, Sushil Khyalia, Vishwajeet Kumar, Ganesh Ramakrishnan, Preethi Jyothi
Abstract
Natural language processing (NLP) tasks (e.g. question-answering in English) benefit from knowledge of other tasks (e.g., named entity recognition in English) and knowledge of other languages (e.g., question-answering in Spanish). Such shared representations are typically learned in isolation, either across tasks or across languages. In this work, we propose a meta-learning approach to learn the interactions between both tasks and languages. We also investigate the role of different sampling strategies used during meta-learning. We present experiments on five different tasks and six different languages from the XTREME multilingual benchmark dataset. Our meta-learned model clearly improves in performance compared to competitive baseline models that also include multi-task baselines. We also present zero-shot evaluations on unseen target languages to demonstrate the utility of our proposed model.- Anthology ID:
- 2021.eacl-main.314
- Volume:
- Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
- Month:
- April
- Year:
- 2021
- Address:
- Online
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3600–3612
- Language:
- URL:
- https://aclanthology.org/2021.eacl-main.314
- DOI:
- 10.18653/v1/2021.eacl-main.314
- Cite (ACL):
- Ishan Tarunesh, Sushil Khyalia, Vishwajeet Kumar, Ganesh Ramakrishnan, and Preethi Jyothi. 2021. Meta-Learning for Effective Multi-task and Multilingual Modelling. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3600–3612, Online. Association for Computational Linguistics.
- Cite (Informal):
- Meta-Learning for Effective Multi-task and Multilingual Modelling (Tarunesh et al., EACL 2021)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2021.eacl-main.314.pdf
- Code
- ishan00/meta-learning-for-multi-task-multilingual
- Data
- GLUE