TueSents at SemEval-2024 Task 8: Predicting the Shift from Human Authorship to Machine-generated Output in a Mixed Text

Valentin Pickard, Hoa Do


Abstract
This paper describes our approach and results for the SemEval 2024 task of identifying the token index in a mixed text where a switch from human authorship to machine-generated text occurs. We explore two BiLSTMs, one over sentence feature vectors to predict the index of the sentence containing such a change and another over character embeddings of the text. As sentence features, we compute token count, mean token length, standard deviation of token length, counts for punctuation and space characters, various readability scores, word frequency class and word part-of-speech class counts for each sentence. class counts. The evaluation is performed on mean absolute error (MAE) between predicted and actual boundary word index. While our competition results were notably below the baseline, there may still be useful aspects to our approach.
Anthology ID:
2024.semeval-1.118
Volume:
Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
829–832
Language:
URL:
https://preview.aclanthology.org/bulk-corrections-2025-11-25/2024.semeval-1.118/
DOI:
10.18653/v1/2024.semeval-1.118
Bibkey:
Cite (ACL):
Valentin Pickard and Hoa Do. 2024. TueSents at SemEval-2024 Task 8: Predicting the Shift from Human Authorship to Machine-generated Output in a Mixed Text. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 829–832, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
TueSents at SemEval-2024 Task 8: Predicting the Shift from Human Authorship to Machine-generated Output in a Mixed Text (Pickard & Do, SemEval 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/bulk-corrections-2025-11-25/2024.semeval-1.118.pdf
Supplementarymaterial:
 2024.semeval-1.118.SupplementaryMaterial.txt
Supplementarymaterial:
 2024.semeval-1.118.SupplementaryMaterial.zip