Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG

Sifei Meng, Dmitry Ilvovsky


Abstract
Multi-turn retrieval-augmented generation (RAG) is challenging due to evolving user intent, conversational noise, and strict context limits. We propose a training-free hybrid retrieval pipeline for SemEval-2026 Task 8 that combines dense and sparse retrieval with controlled query rewriting and cross-encoder reranking. Our system achieves 0.5453 nDCG@5 on the official test set of Task A, ranking 3rd out of 38 teams and outperforming the strongest baseline (0.4795). For Task C, we reuse the Task A retrieved documents in a lightweight generation pipeline based on the official prompt, achieving 0.5312 (harmonic mean of quality and faithfulness) and ranking 15th out of 29 teams. All retrieval components are open-source, while rewriting and generation use LLM APIs. Code and scripts are available on GitHub (https://github.com/mengsifei/MultiturnRAG).
Anthology ID:
2026.semeval-1.32
Volume:
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Month:
July
Year:
2026
Address:
San Diego, California, USA
Editors:
Ekaterina Kochmar, Debanjan Ghosh, Kai North, Mamoru Komachi
Venues:
SemEval | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
221–227
Language:
URL:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.32/
DOI:
Bibkey:
Cite (ACL):
Sifei Meng and Dmitry Ilvovsky. 2026. Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG. In Proceedings of the 20th International Workshop on Semantic Evaluation (2026), pages 221–227, San Diego, California, USA. Association for Computational Linguistics.
Cite (Informal):
Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG (Meng & Ilvovsky, SemEval 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.32.pdf
Supplementarymaterial:
 2026.semeval-1.32.SupplementaryMaterial.zip