@inproceedings{belz-thomson-2024-2024,
    title = "The 2024 {R}epro{NLP} Shared Task on Reproducibility of Evaluations in {NLP}: Overview and Results",
    author = "Belz, Anya  and
      Thomson, Craig",
    editor = "Balloccu, Simone  and
      Belz, Anya  and
      Huidrom, Rudali  and
      Reiter, Ehud  and
      Sedoc, Joao  and
      Thomson, Craig",
    booktitle = "Proceedings of the Fourth Workshop on Human Evaluation of NLP Systems (HumEval) @ LREC-COLING 2024",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://preview.aclanthology.org/jlcl-multiple-ingestion/2024.humeval-1.9/",
    pages = "91--105",
    abstract = "This paper presents an overview of, and the results from, the 2024 Shared Task on Reproducibility of Evaluations in NLP (ReproNLP`24), following on from three previous shared tasks on reproducibility of evaluations in NLP, ReproNLP`23, ReproGen`22 and ReproGen`21. This shared task series forms part of an ongoing research programme designed to develop theory and practice of reproducibility assessment in NLP and machine learning, against a backdrop of increasing recognition of the importance of reproducibility across the two fields. We describe the ReproNLP`24 shared task, summarise results from the reproduction studies submitted, and provide additional comparative analysis of their results."
}