Learning from Mistakes: Combining Ontologies via Self-Training for Dialogue Generation
Lena Reed, Vrindavan Harrison, Shereen Oraby, Dilek Hakkani-Tur, Marilyn Walker
Abstract
Natural language generators (NLGs) for task-oriented dialogue typically take a meaning representation (MR) as input, and are trained end-to-end with a corpus of MR/utterance pairs, where the MRs cover a specific set of dialogue acts and domain attributes. Creation of such datasets is labor intensive and time consuming. Therefore, dialogue systems for new domain ontologies would benefit from using data for pre-existing ontologies. Here we explore, for the first time, whether it is possible to train an NLG for a new larger ontology using existing training sets for the restaurant domain, where each set is based on a different ontology. We create a new, larger combined ontology, and then train an NLG to produce utterances covering it. For example, if one dataset has attributes for family friendly and rating information, and the other has attributes for decor and service, our aim is an NLG for the combined ontology that can produce utterances that realize values for family friendly, rating, decor and service. Initial experiments with a baseline neural sequence-to-sequence model show that this task is surprisingly challenging. We then develop a novel self-training method that identifies (errorful) model outputs, automatically constructs a corrected MR input to form a new (MR, utterance) training pair, and then repeatedly adds these new instances back into the training data. We then test the resulting model on a new test set. The result is a self-trained model whose performance is an absolute 75.4% improvement over the baseline model. We also report a human qualitative evaluation of the final model showing that it achieves high naturalness, semantic coherence and grammaticality.- Anthology ID:
- 2020.sigdial-1.3
- Volume:
- Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue
- Month:
- July
- Year:
- 2020
- Address:
- 1st virtual meeting
- Venue:
- SIGDIAL
- SIG:
- SIGDIAL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 21–34
- Language:
- URL:
- https://aclanthology.org/2020.sigdial-1.3
- DOI:
- Cite (ACL):
- Lena Reed, Vrindavan Harrison, Shereen Oraby, Dilek Hakkani-Tur, and Marilyn Walker. 2020. Learning from Mistakes: Combining Ontologies via Self-Training for Dialogue Generation. In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 21–34, 1st virtual meeting. Association for Computational Linguistics.
- Cite (Informal):
- Learning from Mistakes: Combining Ontologies via Self-Training for Dialogue Generation (Reed et al., SIGDIAL 2020)
- PDF:
- https://preview.aclanthology.org/ingestion-script-update/2020.sigdial-1.3.pdf
- Data
- E2E