Logical Reasoning for Task Oriented Dialogue Systems
Sajjad Beygi, Maryam Fazel-Zarandi, Alessandra Cervone, Prakash Krishnan, Siddhartha Jonnalagadda
Abstract
In recent years, large pretrained models have been used in dialogue systems to improve successful task completion rates. However, lack of reasoning capabilities of dialogue platforms make it difficult to provide relevant and fluent responses, unless the designers of a conversational experience spend a considerable amount of time implementing these capabilities in external rule based modules. In this work, we propose a novel method to fine-tune pretrained transformer models such as Roberta and T5, to reason over a set of facts in a given dialogue context. Our method includes a synthetic data generation mechanism which helps the model learn logical relations, such as comparison between list of numerical values, inverse relations (and negation), inclusion and exclusion for categorical attributes, and application of a combination of attributes over both numerical and categorical values, and spoken form for numerical values, without need for additional training data. We show that the transformer based model can perform logical reasoning to answer questions when the dialogue context contains all the required information, otherwise it is able to extract appropriate constraints to pass to downstream components (e.g. a knowledge base) when partial information is available. We observe that transformer based models such as UnifiedQA-T5 can be fine-tuned to perform logical reasoning (such as numerical and categorical attributes’ comparison) over attributes seen at training time (e.g., accuracy of 90%+ for comparison of smaller than kmax=5 values over heldout test dataset).- Anthology ID:
- 2022.ecnlp-1.10
- Volume:
- Proceedings of the Fifth Workshop on e-Commerce and NLP (ECNLP 5)
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Shervin Malmasi, Oleg Rokhlenko, Nicola Ueffing, Ido Guy, Eugene Agichtein, Surya Kallumadi
- Venue:
- ECNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 68–79
- Language:
- URL:
- https://aclanthology.org/2022.ecnlp-1.10
- DOI:
- 10.18653/v1/2022.ecnlp-1.10
- Cite (ACL):
- Sajjad Beygi, Maryam Fazel-Zarandi, Alessandra Cervone, Prakash Krishnan, and Siddhartha Jonnalagadda. 2022. Logical Reasoning for Task Oriented Dialogue Systems. In Proceedings of the Fifth Workshop on e-Commerce and NLP (ECNLP 5), pages 68–79, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Logical Reasoning for Task Oriented Dialogue Systems (Beygi et al., ECNLP 2022)
- PDF:
- https://preview.aclanthology.org/add_acl24_videos/2022.ecnlp-1.10.pdf