Hamvir Dev
2026
Beyond Instruction Optimization: Multi-Agent Error-Driven Class Description Refinement for LLM-Based Classification
Hamvir Dev | Shivam Ratnakant Mhaskar | Sasanka Vutla | Anup Pattnaik
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Hamvir Dev | Shivam Ratnakant Mhaskar | Sasanka Vutla | Anup Pattnaik
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Large Language Models (LLMs) have demonstrated considerable efficacy in classification tasks, yet their performance depends on two critical prompt components: Task Instructions (HOW to classify) and Class Descriptions (WHAT defines each class). While prompt engineering research has extensively explored instruction optimization, class descriptions have received comparatively less attention, often being treated as fixed inputs or simple label names. This represents a critical gap for real-world classification tasks, particularly in contact center domains, where labels often suffer from ambiguous boundaries, overlapping definitions, and incomplete coverage of possible cases—substantially limiting accuracy regardless of instruction quality.We propose a multi-agent framework for iteratively refining class descriptions based on classification errors. By analyzing misclassified instances, language agents automatically generate improved descriptions that better capture class distinctions and resolve ambiguities. Empirical evaluation across contact center and public benchmark datasets demonstrates upto 20.71% accuracy improvements over static class descriptions, addressing an orthogonal dimension to existing instruction optimization techniques.
2025
Scalable and Cost Effective High-Cardinality Classification with LLMs via Multi-View Label Representations and Retrieval Augmentation
Anup Pattnaik | Sasanka Vutla | Hamvir Dev | Jeevesh Nandan | Cijo George
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track
Anup Pattnaik | Sasanka Vutla | Hamvir Dev | Jeevesh Nandan | Cijo George
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track
Classifying contact center interactions into a large number of categories is critical for downstream analytics, but challenging due to high label cardinality, and cost constraints. While Large Language Models (LLMs) offer flexibility for such tasks, existing methods degrade with increasing label space, showing significant inconsistencies and sensitivity to label ordering. We propose a scalable, cost-effective two-step retrieval-augmented classification framework, enhanced with a multi-view representation of labels. Our method significantly improves accuracy and consistency over baseline LLM approaches. Experiments across 4 private and 5 open datasets yield performance improvements of upto 14.6% while reducing inference cost by 60-91% compared to baseline approaches.