Qalam: A Multimodal LLM for Arabic Optical Character and Handwriting Recognition
Gagan Bhatia, El Moatez Billah Nagoudi, Fakhraddin Alwajih, Muhammad Abdul-Mageed
Abstract
Arabic Optical Character Recognition (OCR) and Handwriting Recognition (HWR) pose unique challenges due to the cursive and context-sensitive nature of the Arabic script. This study introduces ***Qalam***, a novel foundation model designed for Arabic OCR and HWR, built on a SwinV2 encoder and RoBERTa decoder architecture. Our model significantly outperforms existing methods, achieving a Word Error Rate (WER) of just 0.80% in HWR tasks and 1.18% in OCR tasks. We train ***Qalam*** on a diverse dataset, including over 4.5 million images from Arabic manuscripts and a synthetic dataset comprising 60k image-text pairs. Notably, ***Qalam*** demonstrates exceptional handling of Arabic diacritics, a critical feature in Arabic scripts. Furthermore, it shows a remarkable ability to process high-resolution inputs, addressing a common limitation in current OCR systems. These advancements underscore ***Qalam***’s potential as a leading solution for Arabic script recognition, offering a significant leap in accuracy and efficiency.- Anthology ID:
- 2024.arabicnlp-1.19
- Volume:
- Proceedings of The Second Arabic Natural Language Processing Conference
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Nizar Habash, Houda Bouamor, Ramy Eskander, Nadi Tomeh, Ibrahim Abu Farha, Ahmed Abdelali, Samia Touileb, Injy Hamed, Yaser Onaizan, Bashar Alhafni, Wissam Antoun, Salam Khalifa, Hatem Haddad, Imed Zitouni, Badr AlKhamissi, Rawan Almatham, Khalil Mrini
- Venues:
- ArabicNLP | WS
- SIG:
- SIGARAB
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 210–224
- Language:
- URL:
- https://preview.aclanthology.org/icon-24-ingestion/2024.arabicnlp-1.19/
- DOI:
- 10.18653/v1/2024.arabicnlp-1.19
- Cite (ACL):
- Gagan Bhatia, El Moatez Billah Nagoudi, Fakhraddin Alwajih, and Muhammad Abdul-Mageed. 2024. Qalam: A Multimodal LLM for Arabic Optical Character and Handwriting Recognition. In Proceedings of The Second Arabic Natural Language Processing Conference, pages 210–224, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Qalam: A Multimodal LLM for Arabic Optical Character and Handwriting Recognition (Bhatia et al., ArabicNLP 2024)
- PDF:
- https://preview.aclanthology.org/icon-24-ingestion/2024.arabicnlp-1.19.pdf