Muffin: Mitigating Unhelpfulness in Emotional Support Conversations with Multifaceted AI Feedback

Jiashuo Wang, Chunpu Xu, Chak Tou Leong, Wenjie Li, Jing Li

Abstract
Emotional support conversation systems are designed to alleviate users’ emotional distress and assist them in overcoming their challenges. While previous studies have made progress, their models occasionally generate unhelpful responses, which are intended to be supportive but instead have counterproductive effects. Since unhelpful responses can hinder the effectiveness of emotional support, it is crucial to mitigate them within conversations. Our solution is motivated by two principal considerations: (1) multiple facets of emotional support are expected to be considered when developing emotional support conversation models, and (2) directly reducing the probability of generating unhelpful responses can effectively mitigate their occurrence. Accordingly, we introduce a novel model-agnostic framework named  ̲Mitigating  ̲unhelpfulness with multifaceted AI  ̲feedback for emot ̲io ̲nal support (Muffin). It first employs a multifaceted AI feedback module designed to assess the helpfulness model responses across various facets of emotional support. Leveraging contrastive learning, Muffin then reduces the unhelpful responses’ likelihoods. To validate the effectiveness of our proposed framework, we apply Muffin to various previous emotional support generation models, including the state-of-the-art. Experimental results demonstrate that Muffin can significantly mitigate unhelpful response generation while enhancing response fluency and relevance.
Anthology ID:
2024.findings-acl.31
Volume:
Findings of the Association for Computational Linguistics ACL 2024
Month:
August
Year:
2024
Address:
Bangkok, Thailand and virtual meeting
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
567–585
Language:
URL:
https://aclanthology.org/2024.findings-acl.31
DOI:
10.18653/v1/2024.findings-acl.31
Bibkey:
Cite (ACL):
Jiashuo Wang, Chunpu Xu, Chak Tou Leong, Wenjie Li, and Jing Li. 2024. Muffin: Mitigating Unhelpfulness in Emotional Support Conversations with Multifaceted AI Feedback. In Findings of the Association for Computational Linguistics ACL 2024, pages 567–585, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.
Cite (Informal):
Muffin: Mitigating Unhelpfulness in Emotional Support Conversations with Multifaceted AI Feedback (Wang et al., Findings 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/2024.findings-acl.31.pdf
PDF Search