JCT at SemEval-2026 Task 8: Resource-Efficient Multi-Turn RAG via Nano-LLM Rewriting and Hybrid Reranking

Tal Farhan, Chaya Liebeskind


Abstract
This paper describes our system submission for SemEval-2026 Task A (MTRAGEval), focusing on multi-turn Retrieval-Augmented Generation (RAG). Conversational queries often suffer from contextual ambiguity, rendering standard retrieval methods ineffective. We propose a highly resource-efficient pipeline that decouples query understanding from retrieval using a 1.5B parameter Nano-LLM (Qwen) for query rewriting, followed by parallel hybrid retrieval (Qdrant) and Cross-Encoder reranking. During internal development, our optimized system achieved an nDCG@5 score of 0.1991 on answerable queries, outperforming the official BM25 baseline. On the official blind test set, the system achieved a score of 0.1744. While our absolute performance trails behind baselines utilizing massive 20B parameter models, our work establishes a crucial baseline for extreme resource efficiency in conversational RAG. We provide a comprehensive error analysis detailing the impact of domain shifts, retrieval funnels, and we conduct a qualitative analysis on the organizers’ surprise “Underspecified” class to highlight the vulnerabilities of generative query rewriting.
Anthology ID:
2026.semeval-1.173
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1326–1331
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.173/
DOI:
Bibkey:
Cite (ACL):
Tal Farhan and Chaya Liebeskind. 2026. JCT at SemEval-2026 Task 8: Resource-Efficient Multi-Turn RAG via Nano-LLM Rewriting and Hybrid Reranking. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 1326–1331, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
JCT at SemEval-2026 Task 8: Resource-Efficient Multi-Turn RAG via Nano-LLM Rewriting and Hybrid Reranking (Farhan & Liebeskind, SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.173.pdf
Supplementarymaterial:
 2026.semeval-1.173.SupplementaryMaterial.zip