Lasse Borgholt


2020

pdf
MultiQT: Multimodal learning for real-time question tracking in speech
Jakob D. Havtorn | Jan Latko | Joakim Edin | Lars Maaløe | Lasse Borgholt | Lorenzo Belgrano | Nicolai Jacobsen | Regitze Sdun | Željko Agić
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

We address a challenging and practical task of labeling questions in speech in real time during telephone calls to emergency medical services in English, which embeds within a broader decision support system for emergency call-takers. We propose a novel multimodal approach to real-time sequence labeling in speech. Our model treats speech and its own textual representation as two separate modalities or views, as it jointly learns from streamed audio and its noisy transcription into text via automatic speech recognition. Our results show significant gains of jointly learning from the two modalities when compared to text or audio only, under adverse noise and limited volume of training data. The results generalize to medical symptoms detection where we observe a similar pattern of improvements with multimodal learning.

2015

pdf
The Rating Game: Sentiment Rating Reproducibility from Text
Lasse Borgholt | Peter Simonsen | Dirk Hovy
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing