QAES: First Publicly-Available Trait-Specific Annotations for Automated Scoring of Arabic Essays
May Bashendy, Salam Albatarni, Sohaila Eltanbouly, Eman Zahran, Hamdo Elhuseyin, Tamer Elsayed, Walid Massoud, Houda Bouamor
Abstract
Automated Essay Scoring (AES) has emerged as a significant research problem within natural language processing, providing valuable support for educators in assessing student writing skills. In this paper, we introduce QAES, the first publicly available trait-specific annotations for Arabic AES, built on the Qatari Corpus of Argumentative Writing (QCAW). QAES includes a diverse collection of essays in Arabic, each of them annotated with holistic and trait-specific scores, including relevance, organization, vocabulary, style, development, mechanics, and grammar. In total, it comprises 195 Arabic essays (with lengths ranging from 239 to 806 words) across two distinct argumentative writing tasks. We benchmark our dataset against the state-of-the-art English baselines and a feature-based approach. In addition, we discuss the adopted guidelines and the challenges encountered during the annotation process. Finally, we provide insights into potential areas for improvement and future directions in Arabic AES research.- Anthology ID:
- 2024.arabicnlp-1.28
- Volume:
- Proceedings of The Second Arabic Natural Language Processing Conference
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Nizar Habash, Houda Bouamor, Ramy Eskander, Nadi Tomeh, Ibrahim Abu Farha, Ahmed Abdelali, Samia Touileb, Injy Hamed, Yaser Onaizan, Bashar Alhafni, Wissam Antoun, Salam Khalifa, Hatem Haddad, Imed Zitouni, Badr AlKhamissi, Rawan Almatham, Khalil Mrini
- Venues:
- ArabicNLP | WS
- SIG:
- SIGARAB
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 337–351
- Language:
- URL:
- https://aclanthology.org/2024.arabicnlp-1.28
- DOI:
- 10.18653/v1/2024.arabicnlp-1.28
- Cite (ACL):
- May Bashendy, Salam Albatarni, Sohaila Eltanbouly, Eman Zahran, Hamdo Elhuseyin, Tamer Elsayed, Walid Massoud, and Houda Bouamor. 2024. QAES: First Publicly-Available Trait-Specific Annotations for Automated Scoring of Arabic Essays. In Proceedings of The Second Arabic Natural Language Processing Conference, pages 337–351, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- QAES: First Publicly-Available Trait-Specific Annotations for Automated Scoring of Arabic Essays (Bashendy et al., ArabicNLP-WS 2024)
- PDF:
- https://preview.aclanthology.org/autopr/2024.arabicnlp-1.28.pdf