Abstract
Large pre-trained neural models have achieved remarkable success in natural language process (NLP), inspiring a growing body of research analyzing their ability from different aspects. In this paper, we propose a test suite to evaluate the cohesive ability of pre-trained language models. The test suite contains multiple cohesion phenomena between adjacent and non-adjacent sentences. We try to compare different pre-trained language models on these phenomena and analyze the experimental results,hoping more attention can be given to discourse cohesion in the future. The built discourse cohesion test suite will be publicly available at https://github.com/probe2/discourse_cohesion.- Anthology ID:
- 2022.codi-1.4
- Volume:
- Proceedings of the 3rd Workshop on Computational Approaches to Discourse
- Month:
- October
- Year:
- 2022
- Address:
- Gyeongju, Republic of Korea and Online
- Editors:
- Chloe Braud, Christian Hardmeier, Junyi Jessy Li, Sharid Loaiciga, Michael Strube, Amir Zeldes
- Venue:
- CODI
- SIG:
- Publisher:
- International Conference on Computational Linguistics
- Note:
- Pages:
- 28–34
- Language:
- URL:
- https://aclanthology.org/2022.codi-1.4
- DOI:
- Cite (ACL):
- Jie He, Wanqiu Long, and Deyi Xiong. 2022. Evaluating Discourse Cohesion in Pre-trained Language Models. In Proceedings of the 3rd Workshop on Computational Approaches to Discourse, pages 28–34, Gyeongju, Republic of Korea and Online. International Conference on Computational Linguistics.
- Cite (Informal):
- Evaluating Discourse Cohesion in Pre-trained Language Models (He et al., CODI 2022)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2022.codi-1.4.pdf