Collaborative Learning of Bidirectional Decoders for Unsupervised Text Style Transfer

Yun Ma, Yangbin Chen, Xudong Mao, Qing Li


Abstract
Unsupervised text style transfer aims to alter the underlying style of the text to a desired value while keeping its style-independent semantics, without the support of parallel training corpora. Existing methods struggle to achieve both high style conversion rate and low content loss, exhibiting the over-transfer and under-transfer problems. We attribute these problems to the conflicting driving forces of the style conversion goal and content preservation goal. In this paper, we propose a collaborative learning framework for unsupervised text style transfer using a pair of bidirectional decoders, one decoding from left to right while the other decoding from right to left. In our collaborative learning mechanism, each decoder is regularized by knowledge from its peer which has a different knowledge acquisition process. The difference is guaranteed by their opposite decoding directions and a distinguishability constraint. As a result, mutual knowledge distillation drives both decoders to a better optimum and alleviates the over-transfer and under-transfer problems. Experimental results on two benchmark datasets show that our framework achieves strong empirical results on both style compatibility and content preservation.
Anthology ID:
2021.emnlp-main.729
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
9250–9266
Language:
URL:
https://aclanthology.org/2021.emnlp-main.729
DOI:
10.18653/v1/2021.emnlp-main.729
Bibkey:
Cite (ACL):
Yun Ma, Yangbin Chen, Xudong Mao, and Qing Li. 2021. Collaborative Learning of Bidirectional Decoders for Unsupervised Text Style Transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9250–9266, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Collaborative Learning of Bidirectional Decoders for Unsupervised Text Style Transfer (Ma et al., EMNLP 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/improve-issue-templates/2021.emnlp-main.729.pdf
Video:
 https://preview.aclanthology.org/improve-issue-templates/2021.emnlp-main.729.mp4
Code
 sunlight-ym/cbd_style_transfer