StyleDGPT: Stylized Response Generation with Pre-trained Language Models

Ze Yang, Wei Wu, Can Xu, Xinnian Liang, Jiaqi Bai, Liran Wang, Wei Wang, Zhoujun Li


Abstract
Generating responses following a desired style has great potentials to extend applications of open-domain dialogue systems, yet is refrained by lacking of parallel data for training. In this work, we explore the challenging task with pre-trained language models that have brought breakthrough to various natural language tasks. To this end, we introduce a KL loss and a style classifier to the fine-tuning step in order to steer response generation towards the target style in both a word-level and a sentence-level. Comprehensive empirical studies with two public datasets indicate that our model can significantly outperform state-of-the-art methods in terms of both style consistency and contextual coherence.
Anthology ID:
2020.findings-emnlp.140
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2020
Month:
November
Year:
2020
Address:
Online
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1548–1559
Language:
URL:
https://aclanthology.org/2020.findings-emnlp.140
DOI:
10.18653/v1/2020.findings-emnlp.140
Bibkey:
Cite (ACL):
Ze Yang, Wei Wu, Can Xu, Xinnian Liang, Jiaqi Bai, Liran Wang, Wei Wang, and Zhoujun Li. 2020. StyleDGPT: Stylized Response Generation with Pre-trained Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1548–1559, Online. Association for Computational Linguistics.
Cite (Informal):
StyleDGPT: Stylized Response Generation with Pre-trained Language Models (Yang et al., Findings 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/emnlp-22-attachments/2020.findings-emnlp.140.pdf
Code
 TobeyYang/StyleDGPT