An Emotion-based Korean Multimodal Empathetic Dialogue System

Minyoung Jung, Yeongbeom Lim, San Kim, Jin Yea Jang, Saim Shin, Ki-Hoon Lee

Abstract
We propose a Korean multimodal dialogue system targeting emotion-based empathetic dialogues because most research in this field has been conducted in a few languages such as English and Japanese and in certain circumstances. Our dialogue system consists of an emotion detector, an empathetic response generator, a monitoring interface, a voice activity detector, a speech recognizer, a speech synthesizer, a gesture classification, and several controllers to provide both multimodality and empathy during a conversation between a human and a machine. For comparisons across visual influence on users, our dialogue system contains two versions of the user interface, a cat face-based user interface and an avatar-based user interface. We evaluated our dialogue system by investigating the dialogues in text and the average mean opinion scores under three different visual conditions, no visual, the cat face-based, and the avatar-based expressions. The experimental results stand for the importance of adequate visual expressions according to user utterances.
Anthology ID:
2022.cai-1.3
Volume:
Proceedings of the Second Workshop on When Creative AI Meets Conversational AI
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Editors:
Xianchao Wu, Peiying Ruan, Sheng Li, Yi Dong
Venue:
CAI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
16–22
Language:
URL:
https://aclanthology.org/2022.cai-1.3
DOI:
Bibkey:
Cite (ACL):
Minyoung Jung, Yeongbeom Lim, San Kim, Jin Yea Jang, Saim Shin, and Ki-Hoon Lee. 2022. An Emotion-based Korean Multimodal Empathetic Dialogue System. In Proceedings of the Second Workshop on When Creative AI Meets Conversational AI, pages 16–22, Gyeongju, Republic of Korea. Association for Computational Linguistics.
Cite (Informal):
An Emotion-based Korean Multimodal Empathetic Dialogue System (Jung et al., CAI 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-3/2022.cai-1.3.pdf
PDF Search