Generating Discourse Connectives with Pre-trained Language Models: Conditioning on Discourse Relations Helps Reconstruct the PDTB
Symon Stevens-Guille, Aleksandre Maskharashvili, Xintong Li, Michael White
Abstract
We report results of experiments using BART (Lewis et al., 2019) and the Penn Discourse Tree Bank (Webber et al., 2019) (PDTB) to generate texts with correctly realized discourse relations. We address a question left open by previous research (Yung et al., 2021; Ko and Li, 2020) concerning whether conditioning the model on the intended discourse relation—which corresponds to adding explicit discourse relation information into the input to the model—improves its performance. Our results suggest that including discourse relation information in the input of the model significantly improves the consistency with which it produces a correctly realized discourse relation in the output. We compare our models’ performance to known results concerning the discourse structures found in written text and their possible explanations in terms of discourse interpretation strategies hypothesized in the psycholinguistics literature. Our findings suggest that natural language generation models based on current pre-trained Transformers will benefit from infusion with discourse level information if they aim to construct discourses with the intended relations.- Anthology ID:
- 2022.sigdial-1.48
- Volume:
- Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue
- Month:
- September
- Year:
- 2022
- Address:
- Edinburgh, UK
- Editors:
- Oliver Lemon, Dilek Hakkani-Tur, Junyi Jessy Li, Arash Ashrafzadeh, Daniel Hernández Garcia, Malihe Alikhani, David Vandyke, Ondřej Dušek
- Venue:
- SIGDIAL
- SIG:
- SIGDIAL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 500–515
- Language:
- URL:
- https://aclanthology.org/2022.sigdial-1.48
- DOI:
- 10.18653/v1/2022.sigdial-1.48
- Cite (ACL):
- Symon Stevens-Guille, Aleksandre Maskharashvili, Xintong Li, and Michael White. 2022. Generating Discourse Connectives with Pre-trained Language Models: Conditioning on Discourse Relations Helps Reconstruct the PDTB. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 500–515, Edinburgh, UK. Association for Computational Linguistics.
- Cite (Informal):
- Generating Discourse Connectives with Pre-trained Language Models: Conditioning on Discourse Relations Helps Reconstruct the PDTB (Stevens-Guille et al., SIGDIAL 2022)
- PDF:
- https://preview.aclanthology.org/corrections-2024-05/2022.sigdial-1.48.pdf
- Code
- symonjorystevens-guille/penngen