@inproceedings{strich-etal-2026-encourage,
    title = "{E}ncou{RAG}e: Evaluating {RAG} Local, Reliable, and Efficient",
    author = "Strich, Jan  and
      Semmann, Martin  and
      Biemann, Chris",
    editor = "Yang, Eugene  and
      Lawrie, Dawn  and
      MacAvaney, Sean  and
      Mayfield, James  and
      Soldaini, Luca  and
      Yates, Andrew",
    booktitle = "Proceedings of the 1st Workshop on Multilingual Report Generation via Retrieval Augmented Generation ({RAG}4{R}eports 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, CA, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.rag4reports-1.5/",
    pages = "36--46",
    ISBN = "979-8-89176-417-0",
    abstract = "We introduce $\textbf{EncouRAGe}$, a comprehensive Python library designed to streamline the development and evaluation of Retrieval-Augmented Generation (RAG) systems using Large Language Models (LLMs) and Embedding Models. EncouRAGe comprises five modular and extensible components: $\textit{Type Manifest}$, $\textit{RAG Factory}$, $\textit{Inference}$, $\textit{Vector Store}$, and $\textit{Metrics}$, facilitating flexible experimentation and extensible development. Each component helps to make development RAG evaluation and emphasizes $\textbf{scientific reproducibility}$, $\textbf{diverse evaluation metrics}$, and $\textbf{local deployment}$, enabling researchers to efficiently assess datasets within RAG workflows. This paper presents implementation details and an extensive evaluation across multiple benchmark datasets, including $\textit{25k QA pairs}$ and $\textit{over 51k documents}$. Our results show that RAG still underperforms compared to the $\textit{Oracle Context}$, while $\textit{Hybrid BM25}$ consistently achieves the best results across all four datasets. $\textbf{Code}$: https://github.com/uhh-hcds/encourage"
}