Abstract
Text revision refers to a family of natural language generation tasks, where the source and target sequences share moderate resemblance in surface form but differentiate in attributes, such as text formality and simplicity. Current state-of-the-art methods formulate these tasks as sequence-to-sequence learning problems, which rely on large-scale parallel training corpus. In this paper, we present an iterative inplace editing approach for text revision, which requires no parallel data. In this approach, we simply fine-tune a pre-trained Transformer with masked language modeling and attribute classification. During inference, the editing at each iteration is realized by two-step span replacement. At the first step, the distributed representation of the text optimizes on the fly towards an attribute function. At the second step, a text span is masked and another new one is proposed conditioned on the optimized representation. The empirical experiments on two typical and important text revision tasks, text formalization and text simplification, show the effectiveness of our approach. It achieves competitive and even better performance than state-of-the-art supervised methods on text simplification, and gains better performance than strong unsupervised methods on text formalization.- Anthology ID:
- 2022.in2writing-1.7
- Volume:
- Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022)
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Ting-Hao 'Kenneth' Huang, Vipul Raheja, Dongyeop Kang, John Joon Young Chung, Daniel Gissin, Mina Lee, Katy Ilonka Gero
- Venue:
- In2Writing
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 58–59
- Language:
- URL:
- https://aclanthology.org/2022.in2writing-1.7
- DOI:
- 10.18653/v1/2022.in2writing-1.7
- Cite (ACL):
- Jingjing Li, Zichao Li, Tao Ge, Irwin King, and Michael Lyu. 2022. Text Revision by On-the-Fly Representation Optimization. In Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022), pages 58–59, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Text Revision by On-the-Fly Representation Optimization (Li et al., In2Writing 2022)
- PDF:
- https://preview.aclanthology.org/proper-vol2-ingestion/2022.in2writing-1.7.pdf
- Code
- jingjingli01/oreo
- Data
- GYAFC, Newsela