@inproceedings{cui-etal-2026-measuring,
    title = "Measuring the quality of therapy sessions against assessment scales using augmented semantic-similarity approaches",
    author = "Cui, Kejian  and
      D{'}alfonso, Simon  and
      Conway, Mike",
    editor = "Zirikly, Aya  and
      Bar, Kfir  and
      MacAvaney, Sean  and
      Ireland, Molly  and
      Ophir, Yaakov  and
      Atzil-Slonim, Dana  and
      Varadarajan, Vasudha  and
      Bedrick, Steven  and
      Desmet, Bart",
    booktitle = "Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology ({CLP}sych 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.clpsych-1.22/",
    pages = "271--281",
    ISBN = "979-8-89176-421-7",
    abstract = "Therapist fidelity and competence rating scales provide a way to measure quality assurance and therapist training outcomes. Scores on these scales reflect the extent to which a therapist adheres to specific therapeutic principles during a psychotherapy session. Existing research has employed natural language processing (NLP) techniques to automatically predict scale ratings. However, existing approaches require a model trained on a dataset of therapy sessions annotated with the target rating scale.Recent work has explored directly inferring therapeutic alliance by computing semantic similarity between therapy transcripts and the Working Alliance Inventory, via cosine similarity between sentence embeddings.In this paper, we extend this line of work by computing semantic similarity between therapist talk turns and therapist fidelity scale items to directly infer fidelity to specific therapeutic modalities. We further enhance this method by augmentation with LLM-generated example therapist utterances that instantiate target behaviours (as expressed by scale items) across varied therapeutic contexts.In evaluations on two independent datasets, our example-augmented semantic similarity approach consistently shows effectiveness in discriminating therapeutic modalities and levels of therapist fidelity."
}