@inproceedings{ray-gupta-2026-iitkanbdone,
    title = "{IITK}an{BD}one at {S}em{E}val-2026 Task 8: {MTRAGE}val - Evaluating Multi-Turn {RAG} Conversations",
    author = "Ray, Soumendra  and
      Gupta, Garima",
    editor = "Kochmar, Ekaterina  and
      Ghosh, Debanjan  and
      North, Kai  and
      Komachi, Mamoru",
    booktitle = "Proceedings of the 20th {I}nternational {W}orkshop on {S}emantic {E}valuation (2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.145/",
    pages = "1063--1067",
    ISBN = "979-8-89176-414-9",
    abstract = "This paper describes our system for the MT-RAG (Multi-Turn Retrieval-Augmented Generation) shared task, which addresses the challenge of multi-turn conversational question answering using retrieval-augmented generation. We participated in three sub-tasks of Task 8: Task A (retrieval), Task B (generation with reference passages), and Task C (end-to-end RAG). For Task A, we evaluated multiple retrieval approaches including BM25, BGE, and hybrid methods, achieving best performance with ELSER (Elastic Learned Sparse EncodeR) with nDCG@5 of 0.4018 (Rank 24/38). For Task B, we employed the Mistral-7B-Instruct-v0.2 model via HuggingFace for response generation using gold reference passages, achieving a harmonic mean score of 0.6976 (Rank 13/26). For Task C, we combined ELSER retrieval with Mistral-7B generation, using top-5 retrieved passages as context, achieving a score of 0.4289 (Rank 23/29). Our system demonstrates the effectiveness of learned sparse retrieval methods and instruction-tuned models for multi-turn conversational RAG scenarios."
}