The Consistent Lack of Variance of Psychological Factors Expressed by LLMs and Spambots
Vasudha Varadarajan, Salvatore Giorgi, Siddharth Mangalik, Nikita Soni, Dave M. Markowitz, H. Andrew Schwartz
Abstract
In recent years, the proliferation of chatbots like ChatGPT and Claude has led to an increasing volume of AI-generated text. While the text itself is convincingly coherent and human-like, the variety of expressed of human attributes may still be limited. Using theoretical individual differences, the fundamental psychological traits which distinguish people, this study reveals a distinctive characteristic of such content: AI-generations exhibit remarkably limited variation in inferrable psychological traits compared to human-authored texts. We present a review and study across multiple datasets spanning various domains. We find that AI-generated text consistently models the authorship of an “average” human with such little variation that, on aggregate, it is clearly distinguishable from human-written texts using unsupervised methods (i.e., without using ground truth labels). Our results show that (1) fundamental human traits are able to accurately distinguish human- and machine-generated text and (2) current generation capabilities fail to capture a diverse range of human traits- Anthology ID:
- 2025.genaidetect-1.8
- Volume:
- Proceedings of the 1stWorkshop on GenAI Content Detection (GenAIDetect)
- Month:
- January
- Year:
- 2025
- Address:
- Abu Dhabi, UAE
- Editors:
- Firoj Alam, Preslav Nakov, Nizar Habash, Iryna Gurevych, Shammur Chowdhury, Artem Shelmanov, Yuxia Wang, Ekaterina Artemova, Mucahid Kutlu, George Mikros
- Venues:
- GenAIDetect | WS
- SIG:
- Publisher:
- International Conference on Computational Linguistics
- Note:
- Pages:
- 111–119
- Language:
- URL:
- https://preview.aclanthology.org/add-emnlp-2024-awards/2025.genaidetect-1.8/
- DOI:
- Cite (ACL):
- Vasudha Varadarajan, Salvatore Giorgi, Siddharth Mangalik, Nikita Soni, Dave M. Markowitz, and H. Andrew Schwartz. 2025. The Consistent Lack of Variance of Psychological Factors Expressed by LLMs and Spambots. In Proceedings of the 1stWorkshop on GenAI Content Detection (GenAIDetect), pages 111–119, Abu Dhabi, UAE. International Conference on Computational Linguistics.
- Cite (Informal):
- The Consistent Lack of Variance of Psychological Factors Expressed by LLMs and Spambots (Varadarajan et al., GenAIDetect 2025)
- PDF:
- https://preview.aclanthology.org/add-emnlp-2024-awards/2025.genaidetect-1.8.pdf