Low Level Linguistic Controls for Style Transfer and Content Preservation

Katy Gero, Chris Kedzie, Jonathan Reeve, Lydia Chilton

Abstract
Despite the success of style transfer in image processing, it has seen limited progress in natural language generation. Part of the problem is that content is not as easily decoupled from style in the text domain. Curiously, in the field of stylometry, content does not figure prominently in practical methods of discriminating stylistic elements, such as authorship and genre. Rather, syntax and function words are the most salient features. Drawing on this work, we model style as a suite of low-level linguistic controls, such as frequency of pronouns, prepositions, and subordinate clause constructions. We train a neural encoder-decoder model to reconstruct reference sentences given only content words and the setting of the controls. We perform style transfer by keeping the content words fixed while adjusting the controls to be indicative of another style. In experiments, we show that the model reliably responds to the linguistic controls and perform both automatic and manual evaluations on style transfer. We find we can fool a style classifier 84% of the time, and that our model produces highly diverse and stylistically distinctive outputs. This work introduces a formal, extendable model of style that can add control to any neural text generation system.
Anthology ID:
W19-8628
Volume:
Proceedings of the 12th International Conference on Natural Language Generation
Month:
October–November
Year:
2019
Address:
Tokyo, Japan
Editors:
Kees van Deemter, Chenghua Lin, Hiroya Takamura
Venue:
INLG
SIG:
SIGGEN
Publisher:
Association for Computational Linguistics
Note:
Pages:
208–218
Language:
URL:
https://aclanthology.org/W19-8628
DOI:
10.18653/v1/W19-8628
Bibkey:
Cite (ACL):
Katy Gero, Chris Kedzie, Jonathan Reeve, and Lydia Chilton. 2019. Low Level Linguistic Controls for Style Transfer and Content Preservation. In Proceedings of the 12th International Conference on Natural Language Generation, pages 208–218, Tokyo, Japan. Association for Computational Linguistics.
Cite (Informal):
Low Level Linguistic Controls for Style Transfer and Content Preservation (Gero et al., INLG 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-2/W19-8628.pdf
PDF Search