@inproceedings{xie-etal-2026-building,
    title = "Building Multi-turn Intent Classification with {LLM}-based Labeling",
    author = "Xie, Biancen  and
      Bian, Kaiqi  and
      Gusain, Jai Ranjan Singh  and
      Ramanathan, Manikandarajan  and
      Maragoud, Raj",
    editor = "Mysore, Sheshera  and
      Kumar, Sachin  and
      Balachandran, Vidhisha  and
      Hayati, Shirley Anugrah  and
      Brahman, Faeze  and
      Moussa, Hanane Nour  and
      Salemi, Alireza",
    booktitle = "Proceedings of the Second Workshop on Customizable {NLP}: Progress and Challenges in Customizing {NLP} for a Domain, Application, Group, or Individual ({C}ustom{NLP}4{U})",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.customnlp4u-1.8/",
    pages = "68--83",
    ISBN = "979-8-89176-396-8",
    abstract = "Intent classification is essential for customer service routing, connecting customers to the appropriate agents and reducing handling time and operational cost. Developing a real-world multi-turn intent classification system is challenging due to complex intent taxonomies, dynamic intent switching within conversations, and limited labeled training data. To address these challenges, we propose a scalable multi-turn intent classification framework for ecommerce customer service that models intent along multiple dimensions. We introduce LLMbased labeling strategies to annotate real customer transcripts at scale and augment training with LLM-simulated multi-turn dialogues that expand coverage of topic and intent switches, which are rare in existing transcripts. Through extensive experiments, we find that explanationguided labeling with a self-critique step produces the most accurate training labels. Finetuned models built on a RoBERTa backbone outperform zero-shot LLM prompting while achieving substantially lower inference latency. Finally, we show that a hybrid approach that combines the fine-tuned classifier with LLM prompting further improves accuracy over either component alone. Overall, our results provide practical guidance for building and deploying high-accuracy, low-latency, large-scale multi-turn intent classification systems."
}