@inproceedings{singh-etal-2025-systematic,
    title = "Systematic Evaluation of Auto-Encoding and Large Language Model Representations for Capturing Author States and Traits",
    author = "Singh, Khushboo  and
      Varadarajan, Vasudha  and
      V Ganesan, Adithya  and
      Nilsson, August H{\r{a}}kan  and
      Soni, Nikita  and
      Mahwish, Syeda  and
      Chitale, Pranav  and
      Boyd, Ryan L.  and
      Ungar, Lyle  and
      Rosenthal, Richard N  and
      Schwartz, H.",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.findings-acl.971/",
    pages = "18955--18973",
    ISBN = "979-8-89176-256-5",
    abstract = "Large Language Models (LLMs) are increasingly used in human-centered applications, yet their ability to model diverse psychological constructs is not well understood. In this study, we systematically evaluate a range of Transformer-LMs to predict psychological variables across five major dimensions: affect, substance use, mental health, sociodemographics, and personality. Analyses span three temporal levels{---}short daily text responses about current affect, text aggregated over two-weeks, and user-level text collected over two years{---}allowing us to examine how each model{'}s strengths align with the underlying stability of different constructs. The findings show that mental health signals emerge as the most accurately predicted dimensions (r=0.6) across all temporal scales. At the daily scale, smaller models like DeBERTa and HaRT often performed better, whereas, at longer scales or with greater context, larger model like Llama3-8B performed the best. Also, aggregating text over the entire study period yielded stronger correlations for outcomes, such as age and income. Overall, these results suggest the importance of selecting appropriate model architectures and temporal aggregation techniques based on the stability and nature of the target variable."
}