Exploring Pre-Trained Transformers and Bilingual Transfer Learning for Arabic Coreference Resolution
Abstract
In this paper, we develop bilingual transfer learning approaches to improve Arabic coreference resolution by leveraging additional English annotation via bilingual or multilingual pre-trained transformers. We show that bilingual transfer learning improves the strong transformer-based neural coreference models by 2-4 F1. We also systemically investigate the effectiveness of several pre-trained transformer models that differ in training corpora, languages covered, and model capacity. Our best model achieves a new state-of-the-art performance of 64.55 F1 on the Arabic OntoNotes dataset. Our code is publicly available at https://github.com/bnmin/arabic_coref.- Anthology ID:
- 2021.crac-1.10
- Volume:
- Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference
- Month:
- November
- Year:
- 2021
- Address:
- Punta Cana, Dominican Republic
- Venue:
- CRAC
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 94–99
- Language:
- URL:
- https://aclanthology.org/2021.crac-1.10
- DOI:
- 10.18653/v1/2021.crac-1.10
- Cite (ACL):
- Bonan Min. 2021. Exploring Pre-Trained Transformers and Bilingual Transfer Learning for Arabic Coreference Resolution. In Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference, pages 94–99, Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Exploring Pre-Trained Transformers and Bilingual Transfer Learning for Arabic Coreference Resolution (Min, CRAC 2021)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/2021.crac-1.10.pdf
- Code
- bnmin/arabic_coref