Syntax-Infused Variational Autoencoder for Text Generation
Xinyuan Zhang, Yi Yang, Siyang Yuan, Dinghan Shen, Lawrence Carin
Abstract
We present a syntax-infused variational autoencoder (SIVAE), that integrates sentences with their syntactic trees to improve the grammar of generated sentences. Distinct from existing VAE-based text generative models, SIVAE contains two separate latent spaces, for sentences and syntactic trees. The evidence lower bound objective is redesigned correspondingly, by optimizing a joint distribution that accommodates two encoders and two decoders. SIVAE works with long short-term memory architectures to simultaneously generate sentences and syntactic trees. Two versions of SIVAE are proposed: one captures the dependencies between the latent variables through a conditional prior network, and the other treats the latent variables independently such that syntactically-controlled sentence generation can be performed. Experimental results demonstrate the generative superiority of SIVAE on both reconstruction and targeted syntactic evaluations. Finally, we show that the proposed models can be used for unsupervised paraphrasing given different syntactic tree templates.- Anthology ID:
- P19-1199
- Volume:
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Month:
- July
- Year:
- 2019
- Address:
- Florence, Italy
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 2069–2078
- Language:
- URL:
- https://aclanthology.org/P19-1199
- DOI:
- 10.18653/v1/P19-1199
- Cite (ACL):
- Xinyuan Zhang, Yi Yang, Siyang Yuan, Dinghan Shen, and Lawrence Carin. 2019. Syntax-Infused Variational Autoencoder for Text Generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2069–2078, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Syntax-Infused Variational Autoencoder for Text Generation (Zhang et al., ACL 2019)
- PDF:
- https://preview.aclanthology.org/remove-xml-comments/P19-1199.pdf
- Data
- Penn Treebank