Although expressive TTS systems aim to capture human-like emotion, little is known about how well emotional signals in text correspond to those in speech. In this short paper, we investigate how emotion (Valence, Arousal, Dominance) in text relates to emotion in speech. We use 8 large language models for identifying emotion in text and two audio models for emotion in speech, across three genres: Podcasts, Audiobooks and TED talks. Findings show that while language models perform well on emotion recognition from situational text, and the audio models perform well on speech, they show a strong correlation for Valence only. Further, the genre of the content significantly impacts the correlation: audiobooks exhibit higher text-audio correlation than TED talks. Finally, we show that more context for LLMs fails to improve this correlation between text and speech emotion prediction. Our results highlight that emotional signals in text do not correspond well to those in speech: emotion prediction from text alone is insufficient for emotional TTS.

Metaphors and Metaphorical Languages (MLs) play an important role in healthcare for the information communication between clinicians, patients, and patients’ family members. In this work, we focus on the Dutch language and cancer patients’ data. We extract the metaphors used by patients using two data resources: 1) cancer patient storytelling interview data, 2) online forum, data including cancer patients’ posts, comments, and questions to professionals. We investigate how current state of the art LLMs and perform on this task by exploring different prompting strategies such as Chain of Thought, few-shot learning, and self-prompting. With human in the loop, we verify the extracted metaphors and collect the output as a corpus, named “HealthQuote.NL”. We believe the extracted metaphors can be useful for supporting better patient care, e.g. shared decision making, helping communication between patients and clinicians, patient health literacy, etc. It can also be integrated into the design of a care path. We share our prompts and resources at https://github.com/4dpicture/HealthQuote.NL