Next Sentence Prediction helps Implicit Discourse Relation Classification within and across Domains

Wei Shi, Vera Demberg

Abstract
Implicit discourse relation classification is one of the most difficult tasks in discourse parsing. Previous studies have generally focused on extracting better representations of the relational arguments. In order to solve the task, it is however additionally necessary to capture what events are expected to cause or follow each other. Current discourse relation classifiers fall short in this respect. We here show that this shortcoming can be effectively addressed by using the bidirectional encoder representation from transformers (BERT) proposed by Devlin et al. (2019), which were trained on a next-sentence prediction task, and thus encode a representation of likely next sentences. The BERT-based model outperforms the current state of the art in 11-way classification by 8% points on the standard PDTB dataset. Our experiments also demonstrate that the model can be successfully ported to other domains: on the BioDRB dataset, the model outperforms the state of the art system around 15% points.
Anthology ID:
D19-1586
Volume:
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
Month:
November
Year:
2019
Address:
Hong Kong, China
Editors:
Kentaro Inui, Jing Jiang, Vincent Ng, Xiaojun Wan
Venues:
EMNLP | IJCNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
5790–5796
Language:
URL:
https://aclanthology.org/D19-1586
DOI:
10.18653/v1/D19-1586
Bibkey:
Cite (ACL):
Wei Shi and Vera Demberg. 2019. Next Sentence Prediction helps Implicit Discourse Relation Classification within and across Domains. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5790–5796, Hong Kong, China. Association for Computational Linguistics.
Cite (Informal):
Next Sentence Prediction helps Implicit Discourse Relation Classification within and across Domains (Shi & Demberg, EMNLP-IJCNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/ml4al-ingestion/D19-1586.pdf
PDF Search