Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset
Hannah Rashkin, Eric Michael Smith, Margaret Li, Y-Lan Boureau
Abstract
One challenge for dialogue agents is recognizing feelings in the conversation partner and replying accordingly, a key communicative skill. While it is straightforward for humans to recognize and acknowledge others’ feelings in a conversation, this is a significant challenge for AI systems due to the paucity of suitable publicly-available datasets for training and evaluation. This work proposes a new benchmark for empathetic dialogue generation and EmpatheticDialogues, a novel dataset of 25k conversations grounded in emotional situations. Our experiments indicate that dialogue models that use our dataset are perceived to be more empathetic by human evaluators, compared to models merely trained on large-scale Internet conversation data. We also present empirical comparisons of dialogue model adaptations for empathetic responding, leveraging existing models or datasets without requiring lengthy re-training of the full model.- Anthology ID:
- P19-1534
- Volume:
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2019
- Address:
- Florence, Italy
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5370–5381
- Language:
- URL:
- https://aclanthology.org/P19-1534
- DOI:
- 10.18653/v1/P19-1534
- Cite (ACL):
- Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5370–5381, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset (Rashkin et al., ACL 2019)
- PDF:
- https://preview.aclanthology.org/nodalida-main-page/P19-1534.pdf
- Code
- facebookresearch/EmpatheticDialogues + additional community code
- Data
- EmpatheticDialogues