@inproceedings{derunets-etal-2026-raguteam,
    title = "{R}agu{T}eam at {S}em{E}val-2026 Task 8: Meno and {F}riends in a Judge-Orchestrated {LLM} Ensemble for Faithful Multi-Turn Response Generation",
    author = "Derunets, Roman  and
      Bondarenko, Ivan  and
      Sedukhin, Oleg  and
      Komarov, Mikhail  and
      Chernov, Ivan  and
      Kulakov, Mikhail",
    editor = "Kochmar, Ekaterina  and
      Ghosh, Debanjan  and
      North, Kai  and
      Komachi, Mamoru",
    booktitle = "Proceedings of the 20th {I}nternational {W}orkshop on {S}emantic {E}valuation (2026)",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.semeval-1.216/",
    pages = "1678--1694",
    ISBN = "979-8-89176-414-9",
    abstract = "This paper describes our first-place submission to Task B (generation with reference passages) of the SemEval-2026 Task 8 MTRAGEval shared task on multi-turn retrieval-augmented generation. We propose a heterogeneous ensemble of seven LLMs organised into two groups with distinct prompting strategies, and use a GPT-4o-mini judge to select the best candidate response for each instance. Our system ranked first among 26 teams, achieving a conditioned harmonic mean score of 0.78 and substantially outperforming the strongest organiser baseline (0.64). Ablation experiments show that diversity across model families, scales, and prompting strategies is critical: the ensemble consistently outperforms any individual model. We also include Meno-Lite-0.1, a 7B domain-adapted model with a favourable cost{--}performance trade-off, and present an analysis of MTRAGEval that highlights annotation limitations and directions for benchmark improvement."
}