George Altshuler
2024
Team MLab at SemEval-2024 Task 8: Analyzing Encoder Embeddings for Detecting LLM-generated Text
Kevin Li
|
Kenan Hasanaliyev
|
Sally Zhu
|
George Altshuler
|
Alden Eberts
|
Eric Chen
|
Kate Wang
|
Emily Xia
|
Eli Browne
|
Ian Chen
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
This paper explores solutions to the challenges posed by the widespread use of LLMs, particularly in the context of identifying human-written versus machine-generated text. Focusing on Subtask B of SemEval 2024 Task 8, we compare the performance of RoBERTa and DeBERTa models. Subtask B involved identifying not only human or machine text but also the specific LLM responsible for generating text, where our DeBERTa model outperformed the RoBERTa baseline by over 10% in leaderboard accuracy. The results highlight the rapidly growing capabilities of LLMs and importance of keeping up with the latest advancements. Additionally, our paper presents visualizations using PCA and t-SNE that showcase the DeBERTa model’s ability to cluster different LLM outputs effectively. These findings contribute to understanding and improving AI methods for detecting machine-generated text, allowing us to build more robust and traceable AI systems in the language ecosystem.
Search
Co-authors
- Kevin Li 1
- Kenan Hasanaliyev 1
- Sally Zhu 1
- Alden Eberts 1
- Eric Chen 1
- show all...