SubmissionNumber#=%=#223
FinalPaperTitle#=%=#AISPACE at SemEval-2024 task 8: A Class-balanced Soft-voting System for Detecting Multi-generator Machine-generated Text
ShortPaperTitle#=%=#
NumberOfPages#=%=#6
CopyrightSigned#=%=#Renhua Gu
JobTitle#==#
Organization#==#
Abstract#==#SemEval-2024 Task 8 provides a challenge to detect human-written and machine-generated text. There are 3 subtasks for different detection scenarios. This paper proposes a system that mainly deals with Subtask B. It aims to detect if given full text is written by human or is generated by a specific Large Language Model (LLM), which is actually a multi-class text classification task. Our team AISPACE conducted a systematic study of fine-tuning transformer-based models, including encoder-only, decoder-only and encoder-decoder models. We compared their performance on this task and identified that encoder-only models performed exceptionally well. We also applied a weighted Cross Entropy loss function to address the issue of data imbalance of different class samples. Additionally, we employed soft-voting strategy over multi-models ensemble to enhance the reliability of our predictions. Our system ranked top 1 in Subtask B, which sets a state-of-the-art benchmark for this new challenge.
Author{1}{Firstname}#=%=#Renhua
Author{1}{Lastname}#=%=#Gu
Author{1}{Username}#=%=#joeblack
Author{1}{Email}#=%=#renhua.gu@samsung.com
Author{1}{Affiliation}#=%=#Samsung R&D Institute China-Beijing
Author{2}{Firstname}#=%=#Xiangfeng
Author{2}{Lastname}#=%=#Meng
Author{2}{Username}#=%=#ericmxf
Author{2}{Email}#=%=#xf.meng@samsung.com
Author{2}{Affiliation}#=%=#Samsung R&D Institute China-Beijing

==========
èéáğö