Alex Guo


2025

pdf bib
Automated Coding of Counsellor and Client Behaviours in Motivational Interviewing Transcripts: Validation and Application
Soliman Ali | Jiading Zhu | Alex Guo | Xiao Nan Ye | Qilin Gu | Jodi Wolff | Carolynne Cooper | Osnat C. Melamed | Peter Selby | Jonathan Rose
NLP-AI4Health

Motivational Interviewing (MI) is a widely-used talk therapy approach employed by clinicians to guide clients toward healthy behaviour change. Both the automation of MI itself and the evaluation of human counsellors can benefit from high-quality automated classification of counsellor and client utterances. We show how to perform this ``coding'' of utterances using LLMs, by first performing utterance-level parsing and then hierarchical classification of counsellor and client language. Our system achieves an overall accuracy of 82% for the upper (coarse-grained) hierarchy of the counsellor codes and 88% for client codes. The lower (fine-grained) hierarchy scores at 68% and 76% respectively. We also show that these codes can be used to predict the session-level quality of a widely-used MI transcript dataset at 87% accuracy. As a demonstration of practical utility, we show that the slope of the amount of change/sustain talk in client speech across 106 MI transcripts from a human study has significant correlation with an independently surveyed week-later treatment outcome (r=0.28, p<0.005). Finally, we show how the codes can be used to visualize the trajectory of client motivation over a session alongside counsellor codes. The source code and several datasets of annotated MI transcripts are released.