Abstract
In a typical call center, only up to 8% of callersleave a Customer Satisfaction (CSAT) surveyresponse at the end of the call, and these tend tobe customers with strongly positive or negativeexperiences. To manage this data sparsity andresponse bias, we outline a predictive CSATdeep learning algorithm that infers CSAT onthe 1-5 scale on inbound calls to the call centerwith minimal latency. The key metric to maximize is the precision for CSAT = 1 (lowestCSAT). We maximize this metric in two ways. First, reframing the problemas a binary class, rather than five-class problem during model fine-tuning, and then mapping binary outcomes back to five classes usingtemperature-scaled model probabilities. Second, using soft labels to represent the classes. Theresult is a production model able to support keycustomer workflows with high accuracy overmillions of calls a month.- Anthology ID:
- 2023.acl-industry.62
- Volume:
- Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track)
- Month:
- July
- Year:
- 2023
- Address:
- Toronto, Canada
- Editors:
- Sunayana Sitaram, Beata Beigman Klebanov, Jason D Williams
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 652–659
- Language:
- URL:
- https://aclanthology.org/2023.acl-industry.62
- DOI:
- 10.18653/v1/2023.acl-industry.62
- Cite (ACL):
- Etienne Manderscheid and Matthias Lee. 2023. Predicting Customer Satisfaction with Soft Labels for Ordinal Classification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track), pages 652–659, Toronto, Canada. Association for Computational Linguistics.
- Cite (Informal):
- Predicting Customer Satisfaction with Soft Labels for Ordinal Classification (Manderscheid & Lee, ACL 2023)
- PDF:
- https://preview.aclanthology.org/landing_page/2023.acl-industry.62.pdf