MANorm: A Normalization Dictionary for Moroccan Arabic Dialect Written in Latin Script

Randa Zarnoufi, Hamid Jaafar, Walid Bachri, Mounia Abik

Abstract
Social media user generated text is actually the main resource for many NLP tasks. This text, however, does not follow the standard rules of writing. Moreover, the use of dialect such as Moroccan Arabic in written communications increases further NLP tasks complexity. A dialect is a verbal language that does not have a standard orthography. The written dialect is based on the phonetic transliteration of spoken words which leads users to improvise spelling while writing. Thus, for the same word we can find multiple forms of transliterations. Subsequently, it is mandatory to normalize these different transliterations to one canonical word form. To reach this goal, we have exploited the powerfulness of word embedding models generated with a corpus of YouTube comments. Besides, using a Moroccan Arabic dialect dictionary that provides the canonical forms, we have built a normalization dictionary that we refer to as MANorm. We have conducted several experiments to demonstrate the efficiency of MANorm, which have shown its usefulness in dialect normalization. We made MANorm freely available online.
Anthology ID:
2020.wanlp-1.14
Volume:
Proceedings of the Fifth Arabic Natural Language Processing Workshop
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Imed Zitouni, Muhammad Abdul-Mageed, Houda Bouamor, Fethi Bougares, Mahmoud El-Haj, Nadi Tomeh, Wajdi Zaghouani
Venue:
WANLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
155–166
Language:
URL:
https://aclanthology.org/2020.wanlp-1.14
DOI:
Bibkey:
Cite (ACL):
Randa Zarnoufi, Hamid Jaafar, Walid Bachri, and Mounia Abik. 2020. MANorm: A Normalization Dictionary for Moroccan Arabic Dialect Written in Latin Script. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 155–166, Barcelona, Spain (Online). Association for Computational Linguistics.
Cite (Informal):
MANorm: A Normalization Dictionary for Moroccan Arabic Dialect Written in Latin Script (Zarnoufi et al., WANLP 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/naacl24-info/2020.wanlp-1.14.pdf
PDF Search