A New State-of-the-Art BERT Model for Judeo-Arabic
Elisha Rosensweig, Yitzchak Lindenbaum, Hillel Gershuni, Vered Raziel-Kretzmer, Daniel Caine, Avi Shmidman
Abstract
We present JABERT, the first BERT model pretrained specifically for historical Judeo-Arabic texts. We demonstrate that JABERT outperforms Arabic and multilingual models on the downstream task of Judeo-Arabic homograph disambiguation. Furthermore, in order to test the latter, we have curated and annotated the first Judeo-Arabic homograph test set. We release both JABERT and the Judeo-Arabic homograph test to the public for unrestricted use.- Anthology ID:
- 2026.lt4hala-1.6
- Volume:
- Proceedings of the Fourth Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA 2026) @ LREC 2026
- Month:
- May
- Year:
- 2026
- Address:
- Palma, Mallorca (Spain)
- Editors:
- Rachele Sprugnoli, Marco Passarotti
- Venues:
- LT4HALA | WS
- SIG:
- Publisher:
- ELRA Language Resources Association (ELRA)
- Note:
- Pages:
- 58–71
- Language:
- External URL:
- https://lrec.elra.info/lrec2026-ws-lt4hala-06
- DOI:
- 10.63317/52qpsk7c5bfc
- Cite (ACL):
- Elisha Rosensweig, Yitzchak Lindenbaum, Hillel Gershuni, Vered Raziel-Kretzmer, Daniel Caine, and Avi Shmidman. 2026. A New State-of-the-Art BERT Model for Judeo-Arabic. In Proceedings of the Fourth Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA 2026) @ LREC 2026, pages 58–71, Palma, Mallorca (Spain). ELRA Language Resources Association (ELRA).
- Cite (Informal):
- A New State-of-the-Art BERT Model for Judeo-Arabic (Rosensweig et al., LT4HALA 2026)