Abstract
With the rapid development of LLMs, it is natural to ask how to harness their capabilities efficiently. In this paper, we explore whether it is feasible to direct each input query to a single most suitable LLM. To this end, we propose LLM routing for challenging reasoning tasks. Our extensive experiments suggest that such routing shows promise but is not feasible in all scenarios, so more robust approaches should be investigated to fill this gap.- Anthology ID:
- 2024.insights-1.15
- Volume:
- Proceedings of the Fifth Workshop on Insights from Negative Results in NLP
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Shabnam Tafreshi, Arjun Akula, João Sedoc, Aleksandr Drozd, Anna Rogers, Anna Rumshisky
- Venues:
- insights | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 124–134
- Language:
- URL:
- https://aclanthology.org/2024.insights-1.15
- DOI:
- Cite (ACL):
- Kv Aditya Srivatsa, Kaushal Maurya, and Ekaterina Kochmar. 2024. Harnessing the Power of Multiple Minds: Lessons Learned from LLM Routing. In Proceedings of the Fifth Workshop on Insights from Negative Results in NLP, pages 124–134, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- Harnessing the Power of Multiple Minds: Lessons Learned from LLM Routing (Srivatsa et al., insights-WS 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.insights-1.15.pdf