Crosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments
Minh Van Nguyen, Tuan Ngo Nguyen, Bonan Min, Thien Huu Nguyen
Abstract
Previous work on crosslingual Relation and Event Extraction (REE) suffers from the monolingual bias issue due to the training of models on only the source language data. An approach to overcome this issue is to use unlabeled data in the target language to aid the alignment of crosslingual representations, i.e., via fooling a language discriminator. However, as this approach does not condition on class information, a target language example of a class could be incorrectly aligned to a source language example of a different class. To address this issue, we propose a novel crosslingual alignment method that leverages class information of REE tasks for representation learning. In particular, we propose to learn two versions of representation vectors for each class in an REE task based on either source or target language examples. Representation vectors for corresponding classes will then be aligned to achieve class-aware alignment for crosslingual representations. In addition, we propose to further align representation vectors for language-universal word categories (i.e., parts of speech and dependency relations). As such, a novel filtering mechanism is presented to facilitate the learning of word category representations from contextualized representations on input texts based on adversarial learning. We conduct extensive crosslingual experiments with English, Chinese, and Arabic over REE tasks. The results demonstrate the benefits of the proposed method that significantly advances the state-of-the-art performance in these settings.- Anthology ID:
- 2021.emnlp-main.440
- Volume:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2021
- Address:
- Online and Punta Cana, Dominican Republic
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5414–5426
- Language:
- URL:
- https://aclanthology.org/2021.emnlp-main.440
- DOI:
- 10.18653/v1/2021.emnlp-main.440
- Cite (ACL):
- Minh Van Nguyen, Tuan Ngo Nguyen, Bonan Min, and Thien Huu Nguyen. 2021. Crosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5414–5426, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Crosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments (Nguyen et al., EMNLP 2021)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2021.emnlp-main.440.pdf