@inproceedings{athanasiou-etal-2026-ails,
    title = "{AILS}-{NTUA} at {S}em{E}val-2026 Task 8: Evaluating Multi-Turn {RAG} Conversations",
    author = "Athanasiou, Dimosthenis  and
      Lymperaiou, Maria  and
      Filandrianos, Giorgos  and
      Voulodimos, Athanasios  and
      Stamou, Giorgos",
    editor = "Kochmar, Ekaterina  and
      Ghosh, Debanjan  and
      North, Kai  and
      Komachi, Mamoru",
    booktitle = "Proceedings of the 20th {I}nternational {W}orkshop on {S}emantic {E}valuation (2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.175/",
    pages = "1340--1365",
    ISBN = "979-8-89176-414-9",
    abstract = "We describe the AILS-NTUA system for SemEval-2026 Task 8 (MTRAGEval), addressing all three subtasks of multi-turn retrieval-augmented generation: passage retrieval (A), reference-grounded response generation (B), and end-to-end RAG (C).Our approach is based on two main design principles. First, we adopt a query-diversity-over-retriever-diversity strategy, where multiple complementary LLM-based query reformulations are issued to a single corpus-aligned sparse retriever and combined using a variance-aware nested Reciprocal Rank Fusion scheme. Second, we employ an agentic generation pipeline that decomposes grounded response generation into evidence span extraction, dual-candidate drafting, and calibrated multi-judge selection.The proposed system achieves strong performance across subtasks, ranking first in Task A and second in Task B in the official evaluation. Our empirical findings indicate that query diversity over a well-aligned retriever is more effective than heterogeneous retriever ensembling, and that answerability calibration{---}rather than retrieval coverage{---}emerges as the primary bottleneck in end-to-end performance."
}