MetaTS: Meta Teacher-Student Network for Multilingual Sequence Labeling with Minimal Supervision
Zheng Li, Danqing Zhang, Tianyu Cao, Ying Wei, Yiwei Song, Bing Yin
Abstract
Sequence labeling aims to predict a fine-grained sequence of labels for the text. However, such formulation hinders the effectiveness of supervised methods due to the lack of token-level annotated data. This is exacerbated when we meet a diverse range of languages. In this work, we explore multilingual sequence labeling with minimal supervision using a single unified model for multiple languages. Specifically, we propose a Meta Teacher-Student (MetaTS) Network, a novel meta learning method to alleviate data scarcity by leveraging large multilingual unlabeled data. Prior teacher-student frameworks of self-training rely on rigid teaching strategies, which may hardly produce high-quality pseudo-labels for consecutive and interdependent tokens. On the contrary, MetaTS allows the teacher to dynamically adapt its pseudo-annotation strategies by the student’s feedback on the generated pseudo-labeled data of each language and thus mitigate error propagation from noisy pseudo-labels. Extensive experiments on both public and real-world multilingual sequence labeling datasets empirically demonstrate the effectiveness of MetaTS.- Anthology ID:
- 2021.emnlp-main.255
- Volume:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2021
- Address:
- Online and Punta Cana, Dominican Republic
- Editors:
- Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3183–3196
- Language:
- URL:
- https://aclanthology.org/2021.emnlp-main.255
- DOI:
- 10.18653/v1/2021.emnlp-main.255
- Cite (ACL):
- Zheng Li, Danqing Zhang, Tianyu Cao, Ying Wei, Yiwei Song, and Bing Yin. 2021. MetaTS: Meta Teacher-Student Network for Multilingual Sequence Labeling with Minimal Supervision. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3183–3196, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- MetaTS: Meta Teacher-Student Network for Multilingual Sequence Labeling with Minimal Supervision (Li et al., EMNLP 2021)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2021.emnlp-main.255.pdf