Abstract
Pre-trained language models have been successful in many scenarios. However, their usefulness in task-oriented dialogues is limited due to the intrinsic linguistic differences between general text and task-oriented dialogues. Current task-oriented dialogue pre-training methods rely on a contrastive framework, which faces challenges such as selecting true positives and hard negatives, as well as lacking diversity. In this paper, we propose a novel dialogue pre-training model called BootTOD. It learns task-oriented dialogue representations via a self-bootstrapping framework. Unlike contrastive counterparts, BootTOD aligns context and context+response representations and dismisses the requirements of contrastive pairs. BootTOD also uses multiple appropriate response targets to model the intrinsic one-to-many diversity of human conversations. Experimental results show that BootTOD outperforms strong TOD baselines on diverse downstream dialogue tasks.- Anthology ID:
- 2024.lrec-main.221
- Volume:
- Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
- Month:
- May
- Year:
- 2024
- Address:
- Torino, Italia
- Editors:
- Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
- Venues:
- LREC | COLING
- SIG:
- Publisher:
- ELRA and ICCL
- Note:
- Pages:
- 2458–2467
- Language:
- URL:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2024.lrec-main.221/
- DOI:
- Cite (ACL):
- Weihao Zeng, Keqing He, Yejie Wang, Dayuan Fu, and Weiran Xu. 2024. BootTOD: Bootstrap Task-oriented Dialogue Representations by Aligning Diverse Responses. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 2458–2467, Torino, Italia. ELRA and ICCL.
- Cite (Informal):
- BootTOD: Bootstrap Task-oriented Dialogue Representations by Aligning Diverse Responses (Zeng et al., LREC-COLING 2024)
- PDF:
- https://preview.aclanthology.org/build-pipeline-with-new-library/2024.lrec-main.221.pdf