@inproceedings{achkar-etal-2026-pipeline,
    title = "A Pipeline to Bootstrap the Evaluation of Retrieval-Augmented Generation for the Automation of Systematic Reviews in Computer Science",
    author = {Achkar, Pierre  and
      Gollub, Tim  and
      Simons, Arno  and
      Scells, Harrisen  and
      Fr{\"o}be, Maik  and
      Potthast, Martin},
    editor = "Yang, Eugene  and
      Lawrie, Dawn  and
      MacAvaney, Sean  and
      Mayfield, James  and
      Soldaini, Luca  and
      Yates, Andrew",
    booktitle = "Proceedings of the 1st Workshop on Multilingual Report Generation via Retrieval Augmented Generation ({RAG}4{R}eports 2026)",
    month = jul,
    year = "2026",
    address = "San Diego, CA, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.rag4reports-1.8/",
    pages = "65--70",
    ISBN = "979-8-89176-417-0",
    abstract = "Automating systematic reviews (SRs), i.e., evidence-driven analyses under explicit protocol constraints, is a natural target for retrieval-augmented generation and deep research agents, yet existing benchmarks evaluate isolated subtasks or assume fixed evidence inputs. We introduce RAG4SR-CS-200, a benchmark of 200 computer science systematic reviews designed for protocol-driven systematic review automation. Each instance comprises review objectives, research questions, eligibility criteria, cleaned full-text review structure, references, and extracted tables. These elements support evaluation across key tasks in systematic review creation such as literature retrieval, eligibility screening, citation-grounded review generation, and structured table generation, in both stage-wise and end-to-end settings. RAG4SR-CS-200 provides a foundation for developing more reliable and diagnosable deep research agents for scientific evidence synthesis. Code and data are publicly available (https://github.com/webis-de/rag4sr-cs-200)."
}