Improve Chit-Chat and QA Sentence Classification in User Messages of Dialogue System using Dialogue Act Embedding

Chi Hsiang Chao, Xi Jie Hou, Yu Ching Chiu


Abstract
In recent years, dialogue system is booming and widely used in customer service system, and has achieved good results. Viewing the conversation records between users and real customer service, we can see that the user’s sentences are mixed with questions about products and services, and chat with customer service. According to the experience of professionals, it is helpful in improving the user experience to mix some chats in customer service conversations. However, users’ questions are expected to be answered, while chatting is expected to interact with customer service. In order to produce an appropriate response, the dialogue system must be able to distinguish these two intentions effectively. Dialog act is a classification that linguists define according to its function. We think this information will help distinguishing questioning sentences and chatting sentences. In this paper, we combine a published COVID-19 QA dataset and a COVID-19-topic chat dataset to form our experimental data. Based on the BERT (Bidirectional Encoder Representation from Transformers) model, we build a question-chat classifier model. The experimental results show that the accuracy of the configuration with dialog act embedding is 16% higher than that with only original statement embedding. In addition, it is found that conversation behavior types such as “Statement-non-opinion”, “Signal-non-understanding” and “Appreciation” are more related to question sentences, while “Wh-Question”, “Yes-No-Question” and “Rhetorical-Question” questions are more related to chat sentences.
Anthology ID:
2021.rocling-1.19
Volume:
Proceedings of the 33rd Conference on Computational Linguistics and Speech Processing (ROCLING 2021)
Month:
October
Year:
2021
Address:
Taoyuan, Taiwan
Venue:
ROCLING
SIG:
Publisher:
The Association for Computational Linguistics and Chinese Language Processing (ACLCLP)
Note:
Pages:
138–143
Language:
URL:
https://aclanthology.org/2021.rocling-1.19
DOI:
Bibkey:
Cite (ACL):
Chi Hsiang Chao, Xi Jie Hou, and Yu Ching Chiu. 2021. Improve Chit-Chat and QA Sentence Classification in User Messages of Dialogue System using Dialogue Act Embedding. In Proceedings of the 33rd Conference on Computational Linguistics and Speech Processing (ROCLING 2021), pages 138–143, Taoyuan, Taiwan. The Association for Computational Linguistics and Chinese Language Processing (ACLCLP).
Cite (Informal):
Improve Chit-Chat and QA Sentence Classification in User Messages of Dialogue System using Dialogue Act Embedding (Chao et al., ROCLING 2021)
Copy Citation:
PDF:
https://preview.aclanthology.org/update-css-js/2021.rocling-1.19.pdf
Data
SQuAD