Saadeldine Eletter
2026
Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI
Yuxia Wang | Rui Xing | Jonibek Mansurov | Giovanni Puccetti | Zhuohan Xie | Minh Ngoc Ta | Jiahui Geng | Jinyan Su | Mervat Abassy | Saadeldine Eletter | Kareem Elozeiri | Nurkhan Laiyk | Maiya Goloburda | Tarek Mahmoud | Raj Vardhan Tomar | Alexander Aziz | Ryuto Koike | Masahiro Kaneko | Artem Shelmanov | Ekaterina Artemova | Vladislav Mikhailov | Akim Tsvigun | Alham Fikri Aji | Nizar Habash | Iryna Gurevych | Preslav Nakov
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yuxia Wang | Rui Xing | Jonibek Mansurov | Giovanni Puccetti | Zhuohan Xie | Minh Ngoc Ta | Jiahui Geng | Jinyan Su | Mervat Abassy | Saadeldine Eletter | Kareem Elozeiri | Nurkhan Laiyk | Maiya Goloburda | Tarek Mahmoud | Raj Vardhan Tomar | Alexander Aziz | Ryuto Koike | Masahiro Kaneko | Artem Shelmanov | Ekaterina Artemova | Vladislav Mikhailov | Akim Tsvigun | Alham Fikri Aji | Nizar Habash | Iryna Gurevych | Preslav Nakov
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Prior studies have shown that distinguishing text generated by Large Language Models (LLMs) from human-written one is highly challenging for humans, and often no better than random guessing. To verify the generalizability of this finding across languages and domains, we perform an extensive case study to identify the upper bound of human detection accuracy. Across 16 datasets covering 9 languages and 9 domains, 19 annotators achieved an average detection accuracy of 87.6%, thus challenging previous conclusions. We find that major gaps between human and machine text lie in concreteness, cultural nuances, and diversity. Prompting by explicitly explaining the distinctions in the prompts can partially bridge the gaps in over 50% of the cases. However, we also find that humans do not always prefer human-written text, particularly when they cannot clearly identify its source. We release our dataset, the human labels, and the annotator metadata at https://github.com/xnlp-lab/HumanEval-MGT.
Search
Fix author
Co-authors
- Mervat Abassy 1
- Alham Fikri Aji 1
- Ekaterina Artemova 1
- Alexander Aziz 1
- Kareem Elozeiri 1
- Jiahui Geng 1
- Maiya Goloburda 1
- Iryna Gurevych 1
- Nizar Habash 1
- Masahiro Kaneko 1
- Ryuto Koike 1
- Nurkhan Laiyk 1
- Tarek Mahmoud 1
- Jonibek Mansurov 1
- Vladislav Mikhailov 1
- Preslav Nakov 1
- Giovanni Puccetti 1
- Artem Shelmanov 1
- Jinyan Su 1
- Minh Ngoc Ta 1
- Raj Vardhan Tomar 1
- Akim Tsvigun 1
- Yuxia Wang 1
- Zhuohan Xie 1
- Rui Xing 1
Venues
- ACL1