Abstract
Multi-party dialogues are more difficult for models to understand than one-to-one two-party dialogues, since they involve multiple interlocutors, resulting in interweaving reply-to relations and information flows. To step over these obstacles, an effective way is to pre-train a model that understands the discourse structure of multi-party dialogues, namely, to whom each utterance is replying. However, due to the lack of explicitly annotated discourse labels in multi-party dialogue corpora, previous works fail to scale up the pre-training process by putting aside the unlabeled multi-party conversational data for nothing. To fully utilize the unlabeled data, we propose to treat the discourse structures as latent variables, then jointly infer them and pre-train the discourse-aware model by unsupervised latent variable inference methods. Experiments on multiple downstream tasks show that our pre-trained model outperforms strong baselines by large margins and achieves state-of-the-art (SOTA) results, justifying the effectiveness of our method. The official implementation of this paper is available at https://github.com/EricLee8/MPD_EMVI.- Anthology ID:
- 2023.acl-long.533
- Volume:
- Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 9584–9599
- Language:
- URL:
- https://aclanthology.org/2023.acl-long.533
- DOI:
- 10.18653/v1/2023.acl-long.533
- Cite (ACL):
- Yiyang Li, Xinting Huang, Wei Bi, and Hai Zhao. 2023. Pre-training Multi-party Dialogue Models with Latent Discourse Inference. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9584–9599, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Pre-training Multi-party Dialogue Models with Latent Discourse Inference (Li et al., ACL 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.acl-long.533.pdf