Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs

Dongyeop Kang, Eduard Hovy

Abstract
Generating a long, coherent text such as a paragraph requires a high-level control of different levels of relations between sentences (e.g., tense, coreference). We call such a logical connection between sentences as a (paragraph) flow. In order to produce a coherent flow of text, we explore two forms of intersentential relations in a paragraph: one is a human-created linguistical relation that forms a structure (e.g., discourse tree) and the other is a relation from latent representation learned from the sentences themselves. Our two proposed models incorporate each form of relations into document-level language models: the former is a supervised model that jointly learns a language model as well as discourse relation prediction, and the latter is an unsupervised model that is hierarchically conditioned by a recurrent neural network (RNN) over the latent information. Our proposed models with both forms of relations outperform the baselines in partially conditioned paragraph generation task. Our codes and data are publicly available.
Anthology ID:
D19-1589
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
5809–5815
Language:
URL:
https://aclanthology.org/D19-1589
DOI:
10.18653/v1/D19-1589
Bibkey:
Cite (ACL):
Dongyeop Kang and Eduard Hovy. 2019. Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5809–5815, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Linguistic Versus Latent Relations for Modeling Coherent Flow in Paragraphs (Kang & Hovy, EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-2023-videos/D19-1589.pdf
Attachment:
 D19-1589.Attachment.pdf
Code
 dykang/flownet
PDF Search Code Attachment