Deep Reinforcement Learning-based Dialogue Policy with Graph Convolutional Q-network

Kai Xu, Zhengyu Wang, Yuxuan Long, Qiaona Zhao

Abstract
Deep Reinforcement learning (DRL) has been successfully applied to the dialogue policy of task-oriented dialogue systems. However, one challenge in the existing DRL-based dialogue policy methods is their unstructured state-action representations without the ability to learn the relationship between dialogue states and actions. To alleviate this problem, we propose a graph-structured dialogue policy framework for task-oriented dialogue systems. More specifically, we use an unsupervised approach to construct two different bipartite graphs. Then, we generate the user-related and knowledge-related subgraphs based on the matching dialogue sub-states with bipartite graph nodes. A variant of graph convolutional network is employed to encode dialogue subgraphs. After that, we use a bidirectional gated cycle unit (BGRU) and self-attention mechanism to obtain the high-level historical state representations and employ a neural network for the high-level current state representations. The two state representations are joined to learn the action value of dialogue policy. Experiments implemented with different DRL algorithms demonstrate that the proposed framework significantly improves the effectiveness and stability of dialogue policies.
Anthology ID:
2024.lrec-main.407
Volume:
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
Venues:
LREC | COLING
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
4555–4565
Language:
URL:
https://aclanthology.org/2024.lrec-main.407
DOI:
Bibkey:
Cite (ACL):
Kai Xu, Zhengyu Wang, Yuxuan Long, and Qiaona Zhao. 2024. Deep Reinforcement Learning-based Dialogue Policy with Graph Convolutional Q-network. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 4555–4565, Torino, Italia. ELRA and ICCL.
Cite (Informal):
Deep Reinforcement Learning-based Dialogue Policy with Graph Convolutional Q-network (Xu et al., LREC-COLING 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-4/2024.lrec-main.407.pdf
PDF Search