The Utility of LLM Text Generation in Longitudinal Psychological Datasets

Jari Zegers, Bennett Kleinberg


Abstract
As part of this ongoing work, we prompted an LLM with three waves of texts from a longitudinal panel dataset to generate a text for wave 4. We compared generated to ground truth texts using cosine similarity on embeddings and tested whether text similarity was associated with psychological variables. We found limited evidence for an association but do find differences in the topics used in generated versus ground-truth texts. An explanation for differences in text similarities remains the subject of ongoing investigation.
Anthology ID:
2025.nlpsi-1.10
Volume:
Proceedings of the First Workshop on Integrating NLP and Psychology to Study Social Interactions (NLPSI) @ICWSM ’25
Month:
June
Year:
2025
Address:
Copenhagen, Denmark
Editors:
Aswathy Velutharambath, Sofie Labat, Neele Falk, Flor Miriam Plaza-del-Arco, Roman Klinger, Véronique Hoste
Venues:
NLPSI | WS
SIG:
Publisher:
Association for the Advancement of Artificial Intelligence (www.aaai.org)
Note:
Pages:
92–94
Language:
URL:
https://preview.aclanthology.org/ingest-nlpsi/2025.nlpsi-1.10/
DOI:
Bibkey:
Cite (ACL):
Jari Zegers and Bennett Kleinberg. 2025. The Utility of LLM Text Generation in Longitudinal Psychological Datasets. In Proceedings of the First Workshop on Integrating NLP and Psychology to Study Social Interactions (NLPSI) @ICWSM ’25, pages 92–94, Copenhagen, Denmark. Association for the Advancement of Artificial Intelligence (www.aaai.org).
Cite (Informal):
The Utility of LLM Text Generation in Longitudinal Psychological Datasets (Zegers & Kleinberg, NLPSI 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-nlpsi/2025.nlpsi-1.10.pdf