@inproceedings{puspo-etal-2026-role,
    title = "On the Role of Context in {LLM} Alignment to Mental Health Counseling Competencies",
    author = {Puspo, Sadiya Sayara Chowdhury  and
      Zampieri, Marcos  and
      Uzuner, {\"O}zlem},
    editor = "Zirikly, Aya  and
      Bar, Kfir  and
      MacAvaney, Sean  and
      Ireland, Molly  and
      Ophir, Yaakov  and
      Atzil-Slonim, Dana  and
      Varadarajan, Vasudha  and
      Bedrick, Steven  and
      Desmet, Bart",
    booktitle = "Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology ({CLP}sych 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.clpsych-1.6/",
    pages = "76--87",
    ISBN = "979-8-89176-421-7",
    abstract = "As Large Language Models (LLMs) demonstrate strong performance on clinical benchmarks, it remains unclear whether this reflects true patient-specific reasoning or reliance on generalized symptom patterns. To address this gap, we evaluate LLMs on a counseling competency benchmark to assess their use of patient-specific contextual information. Through controlled experiments with ablation experiments, role framing, Thread-of-Thought (ThoT) prompting, and input perturbations, we find that removing contextual details results in only modest performance drops, and predictions remain stable under input variations, indicating limited sensitivity to context. Although structured prompting increases explicit mention of patient details, it does not improve answer accuracy. Error analysis reveals systematic patterns where models favor general clinical associations over context-specific cues, even when such cues are correctly identified during intermediate reasoning. Our findings suggest that achieving passing-level performance does not guarantee context-sensitive decision-making revealing an important gap between apparent clinical competence and actual contextual reasoning. This indicates the need for evaluation frameworks that directly test context integration in mental health applications."
}