AIpom at SemEval-2024 Task 8: Detecting AI-produced Outputs in M4
Alexander Shirnin, Nikita Andreev, Vladislav Mikhailov, Ekaterina Artemova
Abstract
This paper describes AIpom, a system designed to detect a boundary between human-written and machine-generated text (SemEval-2024 Task 8, Subtask C: Human-Machine Mixed Text Detection). We propose a two-stage pipeline combining predictions from an instruction-tuned decoder-only model and encoder-only sequence taggers. AIpom is ranked second on the leaderboard while achieving a Mean Absolute Error of 15.94. Ablation studies confirm the benefits of pipelining encoder and decoder models, particularly in terms of improved performance.- Anthology ID:
- 2024.semeval-1.238
- Volume:
- Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1667–1672
- Language:
- URL:
- https://aclanthology.org/2024.semeval-1.238
- DOI:
- Cite (ACL):
- Alexander Shirnin, Nikita Andreev, Vladislav Mikhailov, and Ekaterina Artemova. 2024. AIpom at SemEval-2024 Task 8: Detecting AI-produced Outputs in M4. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 1667–1672, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- AIpom at SemEval-2024 Task 8: Detecting AI-produced Outputs in M4 (Shirnin et al., SemEval 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.semeval-1.238.pdf