Towards Coherent and Cohesive Long-form Text Generation
Woon Sang Cho, Pengchuan Zhang, Yizhe Zhang, Xiujun Li, Michel Galley, Chris Brockett, Mengdi Wang, Jianfeng Gao
Abstract
Generating coherent and cohesive long-form texts is a challenging task. Previous works relied on large amounts of human-generated texts to train neural language models. However, few attempted to explicitly improve neural language models from the perspectives of coherence and cohesion. In this work, we propose a new neural language model that is equipped with two neural discriminators which provide feedback signals at the levels of sentence (cohesion) and paragraph (coherence). Our model is trained using a simple yet efficient variant of policy gradient, called ‘negative-critical sequence training’, which is proposed to eliminate the need of training a separate critic for estimating ‘baseline’. Results demonstrate the effectiveness of our approach, showing improvements over the strong baseline – recurrent attention-based bidirectional MLE-trained neural language model.- Anthology ID:
- W19-2401
- Volume:
- Proceedings of the First Workshop on Narrative Understanding
- Month:
- June
- Year:
- 2019
- Address:
- Minneapolis, Minnesota
- Editors:
- David Bamman, Snigdha Chaturvedi, Elizabeth Clark, Madalina Fiterau, Mohit Iyyer
- Venue:
- WNU
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1–11
- Language:
- URL:
- https://aclanthology.org/W19-2401
- DOI:
- 10.18653/v1/W19-2401
- Cite (ACL):
- Woon Sang Cho, Pengchuan Zhang, Yizhe Zhang, Xiujun Li, Michel Galley, Chris Brockett, Mengdi Wang, and Jianfeng Gao. 2019. Towards Coherent and Cohesive Long-form Text Generation. In Proceedings of the First Workshop on Narrative Understanding, pages 1–11, Minneapolis, Minnesota. Association for Computational Linguistics.
- Cite (Informal):
- Towards Coherent and Cohesive Long-form Text Generation (Cho et al., WNU 2019)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/W19-2401.pdf