@inproceedings{mahamood-2026-reprohum,
    title = "{R}epro{H}um {\#}0669-08: Reproducing a Recipe for Arbitrary Text Style Transfer with {LLM}s",
    author = "Mahamood, Saad",
    editor = "Mille, Simon  and
      Gehrmann, Sebastian  and
      Schmidtov{\'a}, Patr{\'i}cia  and
      Du{\v{s}}ek, Ond{\v{r}}ej  and
      Fadaee, Marzieh  and
      Lo, Kyle  and
      Santus, Enrico  and
      Stanovsky, Gabriel",
    booktitle = "Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics ({GEM})",
    month = jul,
    year = "2026",
    address = "San Diego, California, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-acl-workshops/2026.gem-main.90/",
    pages = "1127--1132",
    ISBN = "979-8-89176-423-1",
    abstract = "We describe our attempt to reproduce a single human evaluation quality criterion that was conducted in the paper ``Reproducing a Recipe for Arbitrary Text Style Transfer with LLMs''. This paper describes the approach and challenges involved in reproducing the human evaluation as done by the original authors. In particular, we describe negative results obtained during the reproduction, and we compare our results with an earlier reproduction for the same experiment. Finally, we describe the insights we gained from attempting this particular reproduction and the barriers that remain in attempting successful reproductions. The results and insights presented will hopefully enable the broader NLP research community to improve both how human evaluations are conducted and enable better reproducibility of NLP experiments in the future."
}