Abstract
Generating paragraphs of diverse contents is important in many applications. Existing generation models produce similar contents from homogenized contexts due to the fixed left-to-right sentence order. Our idea is permuting the sentence orders to improve the content diversity of multi-sentence paragraph. We propose a novel framework PermGen whose objective is to maximize the expected log-likelihood of output paragraph distributions with respect to all possible sentence orders. PermGen uses hierarchical positional embedding and designs new procedures for training, and decoding in the sentence-permuted generation. Experiments on three paragraph generation benchmarks demonstrate PermGen generates more diverse outputs with a higher quality than existing models.- Anthology ID:
- 2021.emnlp-main.412
- Volume:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Month:
- November
- Year:
- 2021
- Address:
- Online and Punta Cana, Dominican Republic
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 5051–5062
- Language:
- URL:
- https://aclanthology.org/2021.emnlp-main.412
- DOI:
- 10.18653/v1/2021.emnlp-main.412
- Cite (ACL):
- Wenhao Yu, Chenguang Zhu, Tong Zhao, Zhichun Guo, and Meng Jiang. 2021. Sentence-Permuted Paragraph Generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5051–5062, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
- Cite (Informal):
- Sentence-Permuted Paragraph Generation (Yu et al., EMNLP 2021)
- PDF:
- https://preview.aclanthology.org/nodalida-main-page/2021.emnlp-main.412.pdf
- Code
- wyu97/permgen
- Data
- AGENDA, ROCStories