Counterfactual Dialog Mixing as Data Augmentation for Task-Oriented Dialog Systems

Sebastian Steindl, Ulrich Schäfer, Bernd Ludwig

Abstract
High-quality training data for Task-Oriented Dialog (TOD) systems is costly to come by if no corpora are available. One method to extend available data is data augmentation. Yet, the research into and adaptation of data augmentation techniques for TOD systems is limited in comparison with other data modalities. We propose a novel, causally-flavored data augmentation technique called Counterfactual Dialog Mixing (CDM) that generates realistic synthetic dialogs via counterfactuals to increase the amount of training data. We demonstrate the method on a benchmark dataset and show that a model trained to classify the counterfactuals from the original data fails to do so, which strengthens the claim of creating realistic synthetic dialogs. To evaluate the effectiveness of CDM, we train a current architecture on a benchmark dataset and compare the performance with and without CDM. By doing so, we achieve state-of-the-art on some metrics. We further investigate the external generalizability and a lower resource setting. To evaluate the models, we adopted an interactive evaluation scheme.
Anthology ID:
2024.lrec-main.363
Volume:
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
Venues:
LREC | COLING
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
4078–4087
Language:
URL:
https://aclanthology.org/2024.lrec-main.363
DOI:
Bibkey:
Cite (ACL):
Sebastian Steindl, Ulrich Schäfer, and Bernd Ludwig. 2024. Counterfactual Dialog Mixing as Data Augmentation for Task-Oriented Dialog Systems. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 4078–4087, Torino, Italia. ELRA and ICCL.
Cite (Informal):
Counterfactual Dialog Mixing as Data Augmentation for Task-Oriented Dialog Systems (Steindl et al., LREC-COLING 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2024.lrec-main.363.pdf
PDF Search