Detecting generated scientific papers using an ensemble of transformer models

Anna Glazkova, Maksim Glazkov


Abstract
The paper describes neural models developed for the DAGPap22 shared task hosted at the Third Workshop on Scholarly Document Processing. This shared task targets the automatic detection of generated scientific papers. Our work focuses on comparing different transformer-based models as well as using additional datasets and techniques to deal with imbalanced classes. As a final submission, we utilized an ensemble of SciBERT, RoBERTa, and DeBERTa fine-tuned using random oversampling technique. Our model achieved 99.24% in terms of F1-score. The official evaluation results have put our system at the third place.
Anthology ID:
2022.sdp-1.28
Volume:
Proceedings of the Third Workshop on Scholarly Document Processing
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
sdp
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
223–228
Language:
URL:
https://aclanthology.org/2022.sdp-1.28
DOI:
Bibkey:
Cite (ACL):
Anna Glazkova and Maksim Glazkov. 2022. Detecting generated scientific papers using an ensemble of transformer models. In Proceedings of the Third Workshop on Scholarly Document Processing, pages 223–228, Gyeongju, Republic of Korea. Association for Computational Linguistics.
Cite (Informal):
Detecting generated scientific papers using an ensemble of transformer models (Glazkova & Glazkov, sdp 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2022.sdp-1.28.pdf
Code
 oldaandozerskaya/dagpap22
Data
KP20k