Octopus: A Multitask Model and Toolkit for Arabic Natural Language Generation

AbdelRahim Elmadany, El Moatez Billah Nagoudi, Muhammad Abdul-Mageed

Abstract
Understanding Arabic text and generating human-like responses is a challenging task. While many researchers have proposed models and solutions for individual problems, there is an acute shortage of a comprehensive Arabic natural language generation toolkit that is capable of handling a wide range of tasks. In this work, we present a robust Arabic text-to-text Transformer model, namely AraT5v2, methodically trained on extensive and diverse data, utilizing an extended sequence length of 2,048 tokens. We explore various pretraining strategies including unsupervised, supervised, and joint pertaining, under both single and multitask settings. Our models outperform competitive baselines with large margins. We take our work one step further by developing and publicly releasing OCTOPUS, a Python-based package and command-line toolkit tailored for eight Arabic generation tasks all exploiting a single model. We provide a link to the models and the toolkit through our public repository.
Anthology ID:
2023.arabicnlp-1.20
Volume:
Proceedings of ArabicNLP 2023
Month:
December
Year:
2023
Address:
Singapore (Hybrid)
Editors:
Hassan Sawaf, Samhaa El-Beltagy, Wajdi Zaghouani, Walid Magdy, Ahmed Abdelali, Nadi Tomeh, Ibrahim Abu Farha, Nizar Habash, Salam Khalifa, Amr Keleg, Hatem Haddad, Imed Zitouni, Khalil Mrini, Rawan Almatham
Venues:
ArabicNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
232–243
Language:
URL:
https://aclanthology.org/2023.arabicnlp-1.20
DOI:
10.18653/v1/2023.arabicnlp-1.20
Bibkey:
Cite (ACL):
AbdelRahim Elmadany, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2023. Octopus: A Multitask Model and Toolkit for Arabic Natural Language Generation. In Proceedings of ArabicNLP 2023, pages 232–243, Singapore (Hybrid). Association for Computational Linguistics.
Cite (Informal):
Octopus: A Multitask Model and Toolkit for Arabic Natural Language Generation (Elmadany et al., ArabicNLP-WS 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/dois-2013-emnlp/2023.arabicnlp-1.20.pdf
PDF Search