@inproceedings{same-etal-2026-comparative,
    title = "A Comparative Evaluation of End-to-End and Pipeline Approaches for Summarisation",
    author = "Same, Fahime  and
      Mahamood, Saad  and
      Kamath, Srinivas Ramesh",
    editor = "Mahamood, Saad  and
      Howcroft, David M.  and
      van Deemter, Kees  and
      Balloccu, Simone  and
      Sivaprasad, Adarsa  and
      Sundararajan, Barkavi  and
      Bugar{\'i}n Diz, Alberto  and
      Alonso-Moral, Jose Mar{\'i}a",
    booktitle = "Proceedings of the 1st Symposium on Natural Language Generation Evaluations",
    month = jun,
    year = "2026",
    address = "Aberdeen, United Kingdom",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-retroeval/2026.retroeval-main.6/",
    pages = "39--52",
    ISBN = "979-8-89176-436-1",
    abstract = "We describe and evaluate two different architectures for creating book highlights from unstructured data. Given the prevalence of large language models, we examine whether a pipeline-based approach with intermediate steps for text generation is still necessary and whether it continues to offer any benefits over an end-to-end approach. Our comparative evaluations using LLM-as-a-judge across multiple models with different parameter sizes and generation scenarios show that highlights generated by the end-to-end approach are preferred. However, there is a slight but consistent increase in faithfulness for the pipeline-generated highlights when generating at a thematic level. Additionally, our analysis across multiple models shows that while larger models are more faithful, the degree of faithfulness increases when they are used with a pipeline architecture. The findings from our work indicate that whilst there is comparability between the two approaches, the greater faithfulness, controllability, and observability of pipeline-based approaches offer tangible benefits in applied settings."
}