Abstract
Customizing text style or content typically involves extensive fine-tuning of large models, demanding significant data and training. Traditional unsupervised approaches using sampling often yield low diversity and creativity. We present a novel discrete Langevin proposal that samples directly from the categorical token distribution, overcoming these limitations. By adapting the continuous Langevin algorithm for discrete spaces, our approach enables efficient gradient-based sampling. Evaluations on style transfer tasks demonstrate superior performance over state-of-the-art methods in accuracy, BLEU, BERTScore, and diversity. Our proposed approach paves way for advanced customized text generation with desired styles as well as allows future scope for prompt generation for model safeguarding and jail-breaking.- Anthology ID:
- 2024.customnlp4u-1.12
- Volume:
- Proceedings of the 1st Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual (CustomNLP4U)
- Month:
- November
- Year:
- 2024
- Address:
- Miami, Florida, USA
- Editors:
- Sachin Kumar, Vidhisha Balachandran, Chan Young Park, Weijia Shi, Shirley Anugrah Hayati, Yulia Tsvetkov, Noah Smith, Hannaneh Hajishirzi, Dongyeop Kang, David Jurgens
- Venue:
- CustomNLP4U
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 150–155
- Language:
- URL:
- https://aclanthology.org/2024.customnlp4u-1.12
- DOI:
- 10.18653/v1/2024.customnlp4u-1.12
- Cite (ACL):
- Anugunj Naman. 2024. Customized Style Transfer using Discrete Sampling. In Proceedings of the 1st Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual (CustomNLP4U), pages 150–155, Miami, Florida, USA. Association for Computational Linguistics.
- Cite (Informal):
- Customized Style Transfer using Discrete Sampling (Naman, CustomNLP4U 2024)
- PDF:
- https://preview.aclanthology.org/dois-2013-emnlp/2024.customnlp4u-1.12.pdf