Rithika S
2026
RAGonauts at SemEval-2026 Task 8: BM25 Retrieval with Last-Turn Query Formulation for Multi-Turn RAG Conversations
Rajalakshmi Sivanaiah | Angel Deborah S | Karthik Raja C | Rithika S
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
Rajalakshmi Sivanaiah | Angel Deborah S | Karthik Raja C | Rithika S
Proceedings of the 20th International Workshop on Semantic Evaluation (2026)
This paper describes the submission to Task~A of SemEval-2026 Task~8: MTRAGEval, which evaluates passage retrieval for multi-turn Retrieval-Augmented Generation (RAG) conversations across multiple knowledge domains. The task requires retrieving relevant supporting passages given conversational history, where user queries often contain implicit references and incomplete contextual information. This paper proposes a lightweight and training-free retrieval framework based on BM25 ranking combined with conversational query formulation. Queries are derived from dialogue turns and retrieval is performed using domain-specific indices to preserve corpus relevance. Without neural retrievers or fine-tuning, our system achieves an nDCG@5 score of 0.2836 on the official evaluation set, ranking 33\textsuperscript{rd} on the leaderboard. This result demonstrates that sparse lexical retrieval remains an efficient and reproducible baseline for conversational RAG systems.