SubmissionNumber#=%=#24
FinalPaperTitle#=%=#SPADE: Structured Prompting Augmentation for Dialogue Enhancement in Machine-Generated Text Detection
ShortPaperTitle#=%=#
NumberOfPages#=%=#26
CopyrightSigned#=%=#haoyi li
JobTitle#==#
Organization#==#
Abstract#==#The increasing capability of large language models (LLMs) to generate synthetic content has heightened concerns about their misuse, driving the development of Machine-Generated Text (MGT) detection models. However, these detectors face significant challenges due to the lack of high-quality synthetic datasets for training.
To address this issue, we propose SPADE, a structured framework for detecting synthetic dialogues using prompt-based adversarial samples. Our proposed methods yield 14 new dialogue datasets, which we benchmark against eight MGT detection models. The results demonstrate improved generalization performance when utilizing a mixed dataset produced by proposed augmentation frameworks, offering a practical approach to enhancing LLM application security. Considering that real-world agents lack knowledge of future opponent utterances, we simulate online dialogue detection and examine the relationship between chat history length and detection accuracy. Our open-source datasets can be downloaded.
Author{1}{Firstname}#=%=#Haoyi
Author{1}{Lastname}#=%=#Li
Author{1}{Username}#=%=#haoyili
Author{1}{Email}#=%=#haoyil4@student.unimelb.edu.au
Author{1}{Affiliation}#=%=#The University of Melbourne
Author{2}{Firstname}#=%=#Angela Yifei
Author{2}{Lastname}#=%=#Yuan
Author{2}{Email}#=%=#yuanay@student.unimelb.edu.au
Author{2}{Affiliation}#=%=#The University of Melbourne
Author{3}{Firstname}#=%=#Soyeon Caren
Author{3}{Lastname}#=%=#Han
Author{3}{Email}#=%=#caren.han@unimelb.edu.au
Author{3}{Affiliation}#=%=#The University of Melbourne
Author{4}{Firstname}#=%=#Chirstopher
Author{4}{Lastname}#=%=#Leckie
Author{4}{Email}#=%=#caleckie@unimelb.edu.au
Author{4}{Affiliation}#=%=#The University of Melbourne

==========
èéáğö