LeCun at SemEval-2021 Task 6: Detecting Persuasion Techniques in Text Using Ensembled Pretrained Transformers and Data Augmentation

Dia Abujaber, Ahmed Qarqaz, Malak A. Abdullah

Abstract
We developed a system for task 6 sub-task 1 for detecting propaganda in memes. An external dataset and augmentation data-set were used to extend the official competition data-set. Data augmentation techniques were applied on the external data-set and competition data-set to come up with the augmented data-set. We trained 5 transformers (DeBERTa, and 4 RoBERTa) and ensembled them to make the prediction. We trained 1 RoBERTa model initially on the augmented data-set for a few epochs and then fine-tuned it on the competition data-set which improved the f1-micro up to 0.1 scores. After that, another initial RoBERTa model was trained on the external data-set merged with the augmented data-set for few epochs and fine-tuned it on the competition data-set. Furthermore, we ensembled the initial models with the models after fine-tuning. For the final model in the ensemble, we trained a DeBERTa model on the augmented data-set without fine-tuning it on the competition data-set. Finally, we averaged the output of each model in the ensemble to make the prediction.
Anthology ID:
2021.semeval-1.148
Volume:
Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)
Month:
August
Year:
2021
Address:
Online
Venue:
SemEval
SIGs:
SIGLEX | SIGSEM
Publisher:
Association for Computational Linguistics
Note:
Pages:
1068–1074
Language:
URL:
https://aclanthology.org/2021.semeval-1.148
DOI:
10.18653/v1/2021.semeval-1.148
Bibkey:
Cite (ACL):
Dia Abujaber, Ahmed Qarqaz, and Malak A. Abdullah. 2021. LeCun at SemEval-2021 Task 6: Detecting Persuasion Techniques in Text Using Ensembled Pretrained Transformers and Data Augmentation. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1068–1074, Online. Association for Computational Linguistics.
Cite (Informal):
LeCun at SemEval-2021 Task 6: Detecting Persuasion Techniques in Text Using Ensembled Pretrained Transformers and Data Augmentation (Abujaber et al., SemEval 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2021.semeval-1.148.pdf
PDF Search