AraDetector at ArAIEval Shared Task: An Ensemble of Arabic-specific pre-trained BERT and GPT-4 for Arabic Disinformation Detection
Ahmed Bahaaulddin, Vian Sabeeh, Hanan Belhaj, Serry Sibaee, Samar Ahmad, Ibrahim Khurfan, Abdullah Alharbi
Abstract
The rapid proliferation of disinformation through social media has become one of the most dangerous means to deceive and influence people’s thoughts, viewpoints, or behaviors due to social media’s facilities, such as rapid access, lower cost, and ease of use. Disinformation can spread through social media in different ways, such as fake news stories, doctored images or videos, deceptive data, and even conspiracy theories, thus making detecting disinformation challenging. This paper is a part of participation in the ArAIEval competition that relates to disinformation detection. This work evaluated four models: MARBERT, the proposed ensemble model, and two tests over GPT-4 (zero-shot and Few-shot). GPT-4 achieved micro-F1 79.01% while the ensemble method obtained 76.83%. Despite no improvement in the micro-F1 score on the dev dataset using the ensemble approach, we still used it for the test dataset predictions. We believed that merging different classifiers might enhance the system’s prediction accuracy.- Anthology ID:
- 2023.arabicnlp-1.51
- Volume:
- Proceedings of ArabicNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore (Hybrid)
- Editors:
- Hassan Sawaf, Samhaa El-Beltagy, Wajdi Zaghouani, Walid Magdy, Ahmed Abdelali, Nadi Tomeh, Ibrahim Abu Farha, Nizar Habash, Salam Khalifa, Amr Keleg, Hatem Haddad, Imed Zitouni, Khalil Mrini, Rawan Almatham
- Venues:
- ArabicNLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 530–535
- Language:
- URL:
- https://aclanthology.org/2023.arabicnlp-1.51
- DOI:
- 10.18653/v1/2023.arabicnlp-1.51
- Cite (ACL):
- Ahmed Bahaaulddin, Vian Sabeeh, Hanan Belhaj, Serry Sibaee, Samar Ahmad, Ibrahim Khurfan, and Abdullah Alharbi. 2023. AraDetector at ArAIEval Shared Task: An Ensemble of Arabic-specific pre-trained BERT and GPT-4 for Arabic Disinformation Detection. In Proceedings of ArabicNLP 2023, pages 530–535, Singapore (Hybrid). Association for Computational Linguistics.
- Cite (Informal):
- AraDetector at ArAIEval Shared Task: An Ensemble of Arabic-specific pre-trained BERT and GPT-4 for Arabic Disinformation Detection (Bahaaulddin et al., ArabicNLP-WS 2023)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2023.arabicnlp-1.51.pdf