TIGS: An Inference Algorithm for Text Infilling with Gradient Search

Dayiheng Liu, Jie Fu, Pengfei Liu, Jiancheng Lv


Abstract
Text infilling aims at filling in the missing part of a sentence or paragraph, which has been applied to a variety of real-world natural language generation scenarios. Given a well-trained sequential generative model, it is challenging for its unidirectional decoder to generate missing symbols conditioned on the past and future information around the missing part. In this paper, we propose an iterative inference algorithm based on gradient search, which could be the first inference algorithm that can be broadly applied to any neural sequence generative models for text infilling tasks. Extensive experimental comparisons show the effectiveness and efficiency of the proposed method on three different text infilling tasks with various mask ratios and different mask strategies, comparing with five state-of-the-art methods.
Anthology ID:
P19-1406
Volume:
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2019
Address:
Florence, Italy
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4146–4156
Language:
URL:
https://aclanthology.org/P19-1406
DOI:
10.18653/v1/P19-1406
Bibkey:
Cite (ACL):
Dayiheng Liu, Jie Fu, Pengfei Liu, and Jiancheng Lv. 2019. TIGS: An Inference Algorithm for Text Infilling with Gradient Search. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4146–4156, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
TIGS: An Inference Algorithm for Text Infilling with Gradient Search (Liu et al., ACL 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/auto-file-uploads/P19-1406.pdf
Code
 dayihengliu/Text-Infilling-Gradient-Search