@inproceedings{cuconasu-etal-2026-tale,
    title = "A Tale of Trust and Accuracy: Base vs. Instruct {LLM}s in {RAG} Systems",
    author = "Cuconasu, Florin  and
      Trappolini, Giovanni  and
      Tonellotto, Nicola  and
      Silvestri, Fabrizio",
    editor = "Yang, Eugene  and
      Lawrie, Dawn  and
      MacAvaney, Sean  and
      Mayfield, James  and
      Soldaini, Luca  and
      Yates, Andrew",
    booktitle = "Proceedings of the 1st Workshop on Multilingual Report Generation via Retrieval Augmented Generation ({RAG}4{R}eports 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, CA, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.rag4reports-1.3/",
    pages = "1--23",
    ISBN = "979-8-89176-417-0",
    abstract = "Retrieval-Augmented Generation (RAG) represents a significant advancement in artificial intelligence combining a retrieval phase with a generative phase, with the latter typically being powered by Large Language Models (LLMs). Common wisdom and practices in RAG involve using ``instructed'' LLMs, which are fine-tuned with supervised training to enhance their ability to follow instructions and are aligned with human preferences using state-of-the-art techniques.However, contrary to this popular belief, our study demonstrates that base models outperform their instructed counterparts in RAG tasks by 20{\%} on average under our experimental settings. This finding challenges the prevailing assumptions about the superiority of instructed LLMs in RAG applications. Further investigations reveal a more complex situation, questioning fundamental aspects of RAG and suggesting the need for broader discussions on the topic; or, as Fromm would have it, ``Seldom is a glance at the statistics enough to understand the meaning of the figures''."
}