Abstract
While recent advances in language modeling has resulted in powerful generation models, their generation style remains implicitly dependent on the training data and can not emulate a specific target style. Leveraging the generative capabilities of a transformer-based language models, we present an approach to induce certain target-author attributes by incorporating continuous multi-dimensional lexical preferences of an author into generative language models. We introduce rewarding strategies in a reinforcement learning framework that encourages the use of words across multiple categorical dimensions, to varying extents. Our experiments demonstrate that the proposed approach can generate text that distinctively aligns with a given target author’s lexical style. We conduct quantitative and qualitative comparisons with competitive and relevant baselines to illustrate the benefits of the proposed approach.- Anthology ID:
- 2020.findings-emnlp.96
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2020
- Month:
- November
- Year:
- 2020
- Address:
- Online
- Editors:
- Trevor Cohn, Yulan He, Yang Liu
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1074–1079
- Language:
- URL:
- https://aclanthology.org/2020.findings-emnlp.96
- DOI:
- 10.18653/v1/2020.findings-emnlp.96
- Cite (ACL):
- Hrituraj Singh, Gaurav Verma, and Balaji Vasan Srinivasan. 2020. Incorporating Stylistic Lexical Preferences in Generative Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1074–1079, Online. Association for Computational Linguistics.
- Cite (Informal):
- Incorporating Stylistic Lexical Preferences in Generative Language Models (Singh et al., Findings 2020)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2020.findings-emnlp.96.pdf
- Data
- WebText