Abstract
Despite the success of attention-based neural models for natural language generation and classification tasks, they are unable to capture the discourse structure of larger documents. We hypothesize that explicit discourse representations have utility for NLP tasks over longer documents or document sequences, which sequence-to-sequence models are unable to capture. For abstractive summarization, for instance, conventional neural models simply match source documents and the summary in a latent space without explicit representation of text structure or relations. In this paper, we propose to use neural discourse representations obtained from a rhetorical structure theory (RST) parser to enhance document representations. Specifically, document representations are generated for discourse spans, known as the elementary discourse units (EDUs). We empirically investigate the benefit of the proposed approach on two different tasks: abstractive summarization and popularity prediction of online petitions. We find that the proposed approach leads to substantial improvements in all cases.- Anthology ID:
- U19-1010
- Volume:
- Proceedings of the 17th Annual Workshop of the Australasian Language Technology Association
- Month:
- 4--6 December
- Year:
- 2019
- Address:
- Sydney, Australia
- Editors:
- Meladel Mistica, Massimo Piccardi, Andrew MacKinlay
- Venue:
- ALTA
- SIG:
- Publisher:
- Australasian Language Technology Association
- Note:
- Pages:
- 67–76
- Language:
- URL:
- https://aclanthology.org/U19-1010
- DOI:
- Cite (ACL):
- Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2019. Improved Document Modelling with a Neural Discourse Parser. In Proceedings of the 17th Annual Workshop of the Australasian Language Technology Association, pages 67–76, Sydney, Australia. Australasian Language Technology Association.
- Cite (Informal):
- Improved Document Modelling with a Neural Discourse Parser (Koto et al., ALTA 2019)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/U19-1010.pdf
- Code
- fajri91/RSTExtractor