An Emotion-based Korean Multimodal Empathetic Dialogue System
Minyoung Jung, Yeongbeom Lim, San Kim, Jin Yea Jang, Saim Shin, Ki-Hoon Lee
Abstract
We propose a Korean multimodal dialogue system targeting emotion-based empathetic dialogues because most research in this field has been conducted in a few languages such as English and Japanese and in certain circumstances. Our dialogue system consists of an emotion detector, an empathetic response generator, a monitoring interface, a voice activity detector, a speech recognizer, a speech synthesizer, a gesture classification, and several controllers to provide both multimodality and empathy during a conversation between a human and a machine. For comparisons across visual influence on users, our dialogue system contains two versions of the user interface, a cat face-based user interface and an avatar-based user interface. We evaluated our dialogue system by investigating the dialogues in text and the average mean opinion scores under three different visual conditions, no visual, the cat face-based, and the avatar-based expressions. The experimental results stand for the importance of adequate visual expressions according to user utterances.- Anthology ID:
- 2022.cai-1.3
- Volume:
- Proceedings of the Second Workshop on When Creative AI Meets Conversational AI
- Month:
- October
- Year:
- 2022
- Address:
- Gyeongju, Republic of Korea
- Venue:
- CAI
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 16–22
- Language:
- URL:
- https://aclanthology.org/2022.cai-1.3
- DOI:
- Cite (ACL):
- Minyoung Jung, Yeongbeom Lim, San Kim, Jin Yea Jang, Saim Shin, and Ki-Hoon Lee. 2022. An Emotion-based Korean Multimodal Empathetic Dialogue System. In Proceedings of the Second Workshop on When Creative AI Meets Conversational AI, pages 16–22, Gyeongju, Republic of Korea. Association for Computational Linguistics.
- Cite (Informal):
- An Emotion-based Korean Multimodal Empathetic Dialogue System (Jung et al., CAI 2022)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2022.cai-1.3.pdf