@inproceedings{kaneko-2024-enhancing,
    title = "Enhancing Emotion Recognition in Spoken Dialogue Systems through Multimodal Integration and Personalization",
    author = "Kaneko, Takumasa",
    editor = "Inoue, Koji  and
      Fu, Yahui  and
      Axelsson, Agnes  and
      Ohashi, Atsumoto  and
      Madureira, Brielen  and
      Zenimoto, Yuki  and
      Mohapatra, Biswesh  and
      Stricker, Armand  and
      Khosla, Sopan",
    booktitle = "Proceedings of the 20th Workshop of Young Researchers' Roundtable on Spoken Dialogue Systems",
    month = sep,
    year = "2024",
    address = "Kyoto, Japan",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2024.yrrsds-1.2/",
    doi = "10.18653/v1/2024.yrrsds-1.2",
    pages = "5--7",
    abstract = "My research interests focus on multimodal emotion recognition and personalization in emotion recognition tasks. In multimodal emotion recognition, existing studies demonstrate that integrating various data types like speech, text, and video enhances accuracy. However, real-time constraints and high dataset costs limit their practical application. I propose constructing a multimodal emotion recognition model by combining available unimodal datasets. In terms of personalization, traditional discrete emotion labels often fail to capture the complexity of human emotions. Although recent methods embed speaker characteristics to boost prediction accuracy, they require extensive retraining. I introduce continuous prompt tuning, which updates only the speaker prompts while keeping the speech encoder weights fixed, enabling the addition of new speaker data without additional retraining. This paper discusses these existing research gaps and presents novel approaches to address them, aiming to significantly improve emotion recognition in spoken dialogue systems."
}