Aaron Marker
2026
Evaluating Document-Tuned Transformer Representations for Person-level Mental Health Assessment
Aaron Marker | Oscar Kjell | Vasudha Varadarajan | H. Andrew Schwartz
Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026)
Aaron Marker | Oscar Kjell | Vasudha Varadarajan | H. Andrew Schwartz
Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2026)
Person-level psychological assessment requires aggregating meaning across many messages from the same individual, a task that document-level training objectives were not explicitly designed for. We present a systematic, empirical comparison between architecturally matched traditional (a) base-transformers and (b) document-tuned-transformers (further contrastively fine-tuned at the document-level, sometimes referred to as "sentence transformers") under otherwise identical conditions. Comparing layer-wise and overall performance across two longitudinal mental health and psychological datasets, we find document-tuned models demonstrated a consistent improvement over base representations (increase in Pearson r of 13.4%, p=.015). Robustness analyses revealed document-tuned models remained more accurate under perturbations to word deletion, synonym replacement, typo injection, and back translation. Further, hedged language (e.g., ’usually’) was more characteristic of outcomes in document-tuned embeddings while abundance (e.g., ’lot’) was more characteristic of base-transformers, suggesting document-tuned models may better capture uncertainty.These results suggest representation choice impacts mental health prediction, document-tuned models often being more adept.