SubmissionNumber#=%=#85 FinalPaperTitle#=%=#Werkzeug at SemEval-2024 Task 8: LLM-Generated Text Detection via Gated Mixture-of-Experts Fine-Tuning ShortPaperTitle#=%=# NumberOfPages#=%=#6 CopyrightSigned#=%=#Youlin Wu JobTitle#==# Organization#==# Abstract#==#Recent advancements in Large Language Models (LLMs) have propelled text generation to unprecedented heights, approaching human-level quality. However, it poses a new challenge to distinguish LLM-generated text from human-written text. Presently, most methods address this issue through classification, achieved by fine-tuning on small language models. Unfortunately, small language models suffer from anisotropy issue, where encoded text embeddings become difficult to differentiate in the latent space. Moreover, LLMs possess the ability to alter language styles with versatility, further complicating the classification task. To tackle these challenges, we propose Gated Mixture-of-Experts Fine-tuning (GMoEF) to detect LLM-generated text. GMoEF leverages parametric whitening to normalize text embeddings, thereby mitigating the anisotropy problem. Additionally, GMoEF employs the mixture-of-experts framework equipped with gating router to capture features of LLM-generated text from multiple perspectives. Our GMoEF achieved an impressive ranking of #8 out of 70 teams. The source code is available on https://gitlab.com/sigrs/gmoef. Author{1}{Firstname}#=%=#Youlin Author{1}{Lastname}#=%=#Wu Author{1}{Username}#=%=#wuyoulin Author{1}{Email}#=%=#wuyoulin@mail.dlut.edu.cn Author{1}{Affiliation}#=%=#Dalian University of Technology Author{2}{Firstname}#=%=#Kaichun Author{2}{Lastname}#=%=#Wang Author{2}{Username}#=%=#wangkc Author{2}{Email}#=%=#wangkc@mail.dlut.edu.cn Author{2}{Affiliation}#=%=#Dalian University of Technology Author{3}{Firstname}#=%=#Kai Author{3}{Lastname}#=%=#Ma Author{3}{Username}#=%=#ostmbhwd Author{3}{Email}#=%=#ostmbh@mail.dlut.edu.cn Author{3}{Affiliation}#=%=#Dalian Univercity of technology Author{4}{Firstname}#=%=#Liang Author{4}{Lastname}#=%=#Yang Author{4}{Username}#=%=#yangliang1986 Author{4}{Email}#=%=#liang@dlut.edu.cn Author{4}{Affiliation}#=%=#Dalian University of Technology Author{5}{Firstname}#=%=#Hongfei Author{5}{Lastname}#=%=#LIN Author{5}{Username}#=%=#hflin Author{5}{Email}#=%=#hflin@dlut.edu.cn Author{5}{Affiliation}#=%=#Dalian University of Technology ========== èéáğö