@inproceedings{moll-etal-2025-structuring,
    title = "Structuring Radiology Reports: Challenging {LLM}s with Lightweight Models",
    author = "Moll, Johannes  and
      Fay, Louisa  and
      Azhar, Asfandyar  and
      Ostmeier, Sophie  and
      Gatidis, Sergios  and
      Lueth, Tim C.  and
      Langlotz, Curtis  and
      Delbrouck, Jean-Benoit",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.392/",
    pages = "7718--7735",
    ISBN = "979-8-89176-332-6",
    abstract = "Radiology reports are critical for clinical decision-making but often lack a standardized format, limiting both human interpretability and machine learning (ML) applications. While large language models (LLMs) have shown strong capabilities in reformatting clinical text, their high computational requirements, lack of transparency, and data privacy concerns hinder practical deployment. To address these challenges, we explore lightweight encoder-decoder models ({\ensuremath{<}}300M parameters){---}specifically T5 and BERT2BERT{---}for structuring radiology reports from the MIMIC-CXR and CheXpert Plus datasets. We benchmark these models against eight open-source LLMs (1B{--}70B parameters), adapted using prefix prompting, in-context learning (ICL), and low-rank adaptation (LoRA) finetuning. Our best-performing lightweight model outperforms all LLMs adapted using prompt-based techniques on a human-annotated test set. While some LoRA-finetuned LLMs achieve modest gains over the lightweight model on the Findings section (BLEU 6.4{\%}, ROUGE-L 4.8{\%}, BERTScore 3.6{\%}, F1-RadGraph 1.1{\%}, GREEN 3.6{\%}, and F1-SRR-BERT 4.3{\%}), these improvements come at the cost of substantially greater computational resources. For example, LLaMA-3-70B incurred more than 400 times the inference time, cost, and carbon emissions compared to the lightweight model. These results underscore the potential of lightweight, task-specific models as sustainable and privacy-preserving solutions for structuring clinical text in resource-constrained healthcare settings."
}