SubmissionNumber#=%=#227
FinalPaperTitle#=%=#SINAI at SemEval-2024 Task 8: Fine-tuning on Words and Perplexity as Features for Detecting Machine Written Text
ShortPaperTitle#=%=#
NumberOfPages#=%=#6
CopyrightSigned#=%=#Alberto José Gutiérrez Megías
JobTitle#==#
Organization#==#
Abstract#==#This work presents the proposed systems of the SINAI team for the subtask A of the Task 8 in SemEval 2024. We present the evaluation of two disparate systems, and our final submitted system. We claim that the perplexity value of a text may be used as classification signal. Accordingly, we conduct a study on the utility of perplexity for discerning text authorship, and we perform a comparative analysis of the results obtained on the datasets of the task. This comparative evaluation includes results derived from the systems evaluated, such as fine-tuning using an XLM-RoBERTa-Large transformer or using perplexity as a classification criterion. In addition, we discuss the results reached on the test set, where we show that there is large differences among the language probability distribution of the training and test sets. These analysis allows us to open new research lines to improve the detection of machine-generated text.
Author{1}{Firstname}#=%=#Alberto José
Author{1}{Lastname}#=%=#Gutiérrez Megías
Author{1}{Username}#=%=#sinai
Author{1}{Email}#=%=#agmegias@ujaen.es
Author{1}{Affiliation}#=%=#University of Jaén
Author{2}{Firstname}#=%=#L. Alfonso
Author{2}{Lastname}#=%=#Ureña-López
Author{2}{Username}#=%=#laurena
Author{2}{Email}#=%=#laurena@ujaen.es
Author{2}{Affiliation}#=%=#University of Jaen
Author{3}{Firstname}#=%=#Eugenio
Author{3}{Lastname}#=%=#Martínez Cámara
Author{3}{Email}#=%=#emcamara@ujaen.es
Author{3}{Affiliation}#=%=#University of Jaén

==========
èéáğö