SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation

Ye Zhang, Nan Ding, Radu Soricut


Abstract
Supervised training of abstractive language generation models results in learning conditional probabilities over language sequences based on the supervised training signal. When the training signal contains a variety of writing styles, such models may end up learning an ‘average’ style that is directly influenced by the training data make-up and cannot be controlled by the needs of an application. We describe a family of model architectures capable of capturing both generic language characteristics via shared model parameters, as well as particular style characteristics via private model parameters. Such models are able to generate language according to a specific learned style, while still taking advantage of their power to model generic language phenomena. Furthermore, we describe an extension that uses a mixture of output distributions from all learned styles to perform on-the-fly style adaptation based on the textual input alone. Experimentally, we find that the proposed models consistently outperform models that encapsulate single-style or average-style language generation capabilities.
Anthology ID:
N18-1138
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1528–1538
Language:
URL:
https://aclanthology.org/N18-1138
DOI:
10.18653/v1/N18-1138
Bibkey:
Cite (ACL):
Ye Zhang, Nan Ding, and Radu Soricut. 2018. SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1528–1538, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation (Zhang et al., NAACL 2018)
Copy Citation:
PDF:
https://preview.aclanthology.org/remove-xml-comments/N18-1138.pdf
Video:
 https://vimeo.com/277348640