Transfer Learning for Generalizable Automated LLM Improvement Pipeline for IVR Navigation

Vishal Sankar Ram, Jason Kushner, Manas Paldhe, Youngseo Son


Abstract
Administrative tasks in the healthcare domain share linguistic commonalities, but it can be time-consuming to manually design LLM prompts for each use case. When calling health insurers, interactive voice response (IVR) systems cause delays in patient care and increase provider burnout due to complex routing and long hold times. Thus, IVR navigation models can offer significant time savings and reduce barriers to care. We propose a production-quality automated LLM pipeline which leverages a small number of human-labeled ground truth datasets to transfer specialized prompts from one task to another; specifically, we perform a cross-task transfer of our IVR navigation logic, adapting the prompt from reaching the claims department to reaching the patient benefit department. Our approach reduces prompt complexity by up to 80% and obtains 82% turn-level accuracy in real-world industrial healthcare settings, surpassing a human-designed prompt at 79%.
Anthology ID:
2026.acl-industry.136
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Yunyao Li, Georg Rehm, Mei Tu
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2012–2030
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.acl-industry.136/
DOI:
Bibkey:
Cite (ACL):
Vishal Sankar Ram, Jason Kushner, Manas Paldhe, and Youngseo Son. 2026. Transfer Learning for Generalizable Automated LLM Improvement Pipeline for IVR Navigation. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), pages 2012–2030, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
Transfer Learning for Generalizable Automated LLM Improvement Pipeline for IVR Navigation (Ram et al., ACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.acl-industry.136.pdf