Abstract
Text infilling aims to restore incomplete texts by filling in blanks, which has attracted more attention recently because of its wide application in ancient text restoration and text rewriting. However, attribute- aware text infilling is yet to be explored, and existing methods seldom focus on the infilling length of each blank or the number/location of blanks. In this paper, we propose an Attribute-aware Text Infilling method via a Pre-trained language model (A-TIP), which contains a text infilling component and a plug- and-play discriminator. Specifically, we first design a unified text infilling component with modified attention mechanisms and intra- and inter-blank positional encoding to better perceive the number of blanks and the infilling length for each blank. Then, we propose a plug-and-play discriminator to guide generation towards the direction of improving attribute relevance without decreasing text fluency. Finally, automatic and human evaluations on three open-source datasets indicate that A-TIP achieves state-of- the-art performance compared with all baselines.- Anthology ID:
- 2022.coling-1.511
- Volume:
- Proceedings of the 29th International Conference on Computational Linguistics
- Month:
- October
- Year:
- 2022
- Address:
- Gyeongju, Republic of Korea
- Venue:
- COLING
- SIG:
- Publisher:
- International Committee on Computational Linguistics
- Note:
- Pages:
- 5857–5869
- Language:
- URL:
- https://aclanthology.org/2022.coling-1.511
- DOI:
- Cite (ACL):
- Dongyuan Li, Jingyi You, Kotaro Funakoshi, and Manabu Okumura. 2022. A-TIP: Attribute-aware Text Infilling via Pre-trained Language Model. In Proceedings of the 29th International Conference on Computational Linguistics, pages 5857–5869, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
- Cite (Informal):
- A-TIP: Attribute-aware Text Infilling via Pre-trained Language Model (Li et al., COLING 2022)
- PDF:
- https://preview.aclanthology.org/remove-xml-comments/2022.coling-1.511.pdf
- Data
- ROCStories