AIpom at SemEval-2024 Task 8: Detecting AI-produced Outputs in M4

Alexander Shirnin, Nikita Andreev, Vladislav Mikhailov, Ekaterina Artemova


Abstract
This paper describes AIpom, a system designed to detect a boundary between human-written and machine-generated text (SemEval-2024 Task 8, Subtask C: Human-Machine Mixed Text Detection). We propose a two-stage pipeline combining predictions from an instruction-tuned decoder-only model and encoder-only sequence taggers. AIpom is ranked second on the leaderboard while achieving a Mean Absolute Error of 15.94. Ablation studies confirm the benefits of pipelining encoder and decoder models, particularly in terms of improved performance.
Anthology ID:
2024.semeval-1.238
Volume:
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
1667–1672
Language:
URL:
https://aclanthology.org/2024.semeval-1.238
DOI:
10.18653/v1/2024.semeval-1.238
Bibkey:
Cite (ACL):
Alexander Shirnin, Nikita Andreev, Vladislav Mikhailov, and Ekaterina Artemova. 2024. AIpom at SemEval-2024 Task 8: Detecting AI-produced Outputs in M4. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 1667–1672, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
AIpom at SemEval-2024 Task 8: Detecting AI-produced Outputs in M4 (Shirnin et al., SemEval 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2024.semeval-1.238.pdf
Supplementary material:
 2024.semeval-1.238.SupplementaryMaterial.txt
Supplementary material:
 2024.semeval-1.238.SupplementaryMaterial.zip