Generating Sentences by Editing Prototypes
Kelvin Guu, Tatsunori B. Hashimoto, Yonatan Oren, Percy Liang
Abstract
We propose a new generative language model for sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional language models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation. Furthermore, the model gives rise to a latent edit vector that captures interpretable semantics such as sentence similarity and sentence-level analogies.- Anthology ID:
- Q18-1031
- Volume:
- Transactions of the Association for Computational Linguistics, Volume 6
- Month:
- Year:
- 2018
- Address:
- Cambridge, MA
- Editors:
- Lillian Lee, Mark Johnson, Kristina Toutanova, Brian Roark
- Venue:
- TACL
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 437–450
- Language:
- URL:
- https://aclanthology.org/Q18-1031
- DOI:
- 10.1162/tacl_a_00030
- Cite (ACL):
- Kelvin Guu, Tatsunori B. Hashimoto, Yonatan Oren, and Percy Liang. 2018. Generating Sentences by Editing Prototypes. Transactions of the Association for Computational Linguistics, 6:437–450.
- Cite (Informal):
- Generating Sentences by Editing Prototypes (Guu et al., TACL 2018)
- PDF:
- https://preview.aclanthology.org/ingest-acl-2023-videos/Q18-1031.pdf
- Code
- kelvinguu/neural-editor + additional community code
- Data
- Billion Word Benchmark, One Billion Word Benchmark