Towards end-2-end learning for predicting behavior codes from spoken utterances in psychotherapy conversations
Karan Singla, Zhuohao Chen, David Atkins, Shrikanth Narayanan
Abstract
Spoken language understanding tasks usually rely on pipelines involving complex processing blocks such as voice activity detection, speaker diarization and Automatic speech recognition (ASR). We propose a novel framework for predicting utterance level labels directly from speech features, thus removing the dependency on first generating transcripts, and transcription free behavioral coding. Our classifier uses a pretrained Speech-2-Vector encoder as bottleneck to generate word-level representations from speech features. This pretrained encoder learns to encode speech features for a word using an objective similar to Word2Vec. Our proposed approach just uses speech features and word segmentation information for predicting spoken utterance-level target labels. We show that our model achieves competitive results to other state-of-the-art approaches which use transcribed text for the task of predicting psychotherapy-relevant behavior codes.- Anthology ID:
- 2020.acl-main.351
- Volume:
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2020
- Address:
- Online
- Editors:
- Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 3797–3803
- Language:
- URL:
- https://aclanthology.org/2020.acl-main.351
- DOI:
- 10.18653/v1/2020.acl-main.351
- Cite (ACL):
- Karan Singla, Zhuohao Chen, David Atkins, and Shrikanth Narayanan. 2020. Towards end-2-end learning for predicting behavior codes from spoken utterances in psychotherapy conversations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3797–3803, Online. Association for Computational Linguistics.
- Cite (Informal):
- Towards end-2-end learning for predicting behavior codes from spoken utterances in psychotherapy conversations (Singla et al., ACL 2020)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/2020.acl-main.351.pdf