Abstract
We investigate differences in user communication with live chat agents versus a commercial Intelligent Virtual Agent (IVA). This case study compares the two types of interactions in the same domain for the same company filling the same purposes. We compared 16,794 human-to-human conversations and 27,674 conversations with the IVA. Of those IVA conversations, 8,324 escalated to human live chat agents. We then investigated how human-to-human communication strategies change when users first communicate with an IVA in the same conversation thread. We measured quantity, quality, and diversity of language, and analyzed complexity using numerous features. We find that while the complexity of language did not significantly change between modes, the quantity and some quality metrics did vary significantly. This fair comparison provides unique insight into how humans interact with commercial IVAs and how IVA and chatbot designers might better curate training data when automating customer service tasks.- Anthology ID:
- 2020.sigdial-1.11
- Volume:
- Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue
- Month:
- July
- Year:
- 2020
- Address:
- 1st virtual meeting
- Venue:
- SIGDIAL
- SIG:
- SIGDIAL
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 79–85
- Language:
- URL:
- https://aclanthology.org/2020.sigdial-1.11
- DOI:
- Cite (ACL):
- Timothy Hewitt and Ian Beaver. 2020. A Case Study of User Communication Styles with Customer Service Agents versus Intelligent Virtual Agents. In Proceedings of the 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 79–85, 1st virtual meeting. Association for Computational Linguistics.
- Cite (Informal):
- A Case Study of User Communication Styles with Customer Service Agents versus Intelligent Virtual Agents (Hewitt & Beaver, SIGDIAL 2020)
- PDF:
- https://preview.aclanthology.org/paclic-22-ingestion/2020.sigdial-1.11.pdf