Diff4TST: Masked Diffusion Language Model for Text Style Transfer

Xinchen Ma, Gaole He, Yunshi Lan, Weining Qian


Abstract
Despite recent progress in LLMs for text style transfer, most existing methods rely on costly task-specific training and offer limited control over separating stylistic modification from content preservation. We propose Diff4TST, a diffusion-based language model that formulates text style transfer as an explicit copy-and-edit process. Built upon masked diffusion language models, Diff4TST introduces a style-aware noise schedule that selectively perturbs stylistic tokens while preserving content-bearing tokens during supervised fine-tuning.At inference time, we further introduce a generate-then-refine strategy that iteratively improves style compliance via gradient-based token re-masking, without reinforcement learning or external reward models. Extensive experiments on both fine-grained and polarity-based benchmarks show that Diff4TST achieves substantially improved style accuracy and controllability while maintaining strong content preservation and fluency. These results suggest diffusion-based language models as a principled and effective alternative to autoregressive pipelines for text style transfer.
Anthology ID:
2026.acl-long.306
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6736–6748
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.306/
DOI:
Bibkey:
Cite (ACL):
Xinchen Ma, Gaole He, Yunshi Lan, and Weining Qian. 2026. Diff4TST: Masked Diffusion Language Model for Text Style Transfer. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6736–6748, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
Diff4TST: Masked Diffusion Language Model for Text Style Transfer (Ma et al., ACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.306.pdf
Checklist:
 2026.acl-long.306.checklist.pdf