HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoring
Zhixiong Su, Yichen Wang, Herun Wan, Zhaohan Zhang, Minnan Luo
Abstract
The misuse of large language models (LLMs) poses potential risks, motivating the development of machine-generated text (MGT) detection. Existing literature primarily concentrates on binary, document-level detection, thereby neglecting texts that are composed jointly by human and LLM contributions. Hence, this paper explores the possibility of fine-grained MGT detection under human-AI coauthoring.We suggest fine-grained detectors can pave pathways toward coauthored text detection with a numeric AI ratio.Specifically, we propose a dataset, HACo-Det, which produces human-AI coauthored texts via an automatic pipeline with word-level attribution labels. We retrofit seven prevailing document-level detectors to generalize them to word-level detection.Then we evaluate these detectors on HACo-Det on both word- and sentence-level detection tasks.Empirical results show that metric-based methods struggle to conduct fine-grained detection with a 0.462 average F1 score, while finetuned models show superior performance and better generalization across domains. However, we argue that fine-grained co-authored text detection is far from solved.We further analyze factors influencing performance, e.g., context window, and highlight the limitations of current methods, pointing to potential avenues for improvement.- Anthology ID:
- 2025.acl-long.1069
- Volume:
- Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- July
- Year:
- 2025
- Address:
- Vienna, Austria
- Editors:
- Wanxiang Che, Joyce Nabende, Ekaterina Shutova, Mohammad Taher Pilehvar
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 22015–22036
- Language:
- URL:
- https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1069/
- DOI:
- Cite (ACL):
- Zhixiong Su, Yichen Wang, Herun Wan, Zhaohan Zhang, and Minnan Luo. 2025. HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoring. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 22015–22036, Vienna, Austria. Association for Computational Linguistics.
- Cite (Informal):
- HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoring (Su et al., ACL 2025)
- PDF:
- https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1069.pdf