@inproceedings{ren-etal-2025-beyond,
    title = "Beyond Reconstruction: Generating Privacy-Preserving Clinical Letters",
    author = "Ren, Libo  and
      Belkadi, Samuel  and
      Han, Lifeng  and
      Del-Pinto, Warren  and
      Nenadic, Goran",
    editor = "Habernal, Ivan  and
      Ghanavati, Sepideh  and
      Jain, Vijayanta  and
      Igamberdiev, Timour  and
      Wilson, Shomir",
    booktitle = "Proceedings of the Sixth Workshop on Privacy in Natural Language Processing",
    month = apr,
    year = "2025",
    address = "Albuquerque, New Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2025.privatenlp-main.6/",
    pages = "60--74",
    ISBN = "979-8-89176-246-6",
    abstract = "Due to the sensitive nature of clinical letters, their use in model training, medical research, and education is limited. This work aims to generate diverse, de-identified, and high-quality synthetic clinical letters to enhance privacy protection. This study explores various pre-trained language models (PLMs) for text masking and generation, employing various masking strategies with a focus on Bio{\_}ClinicalBERT. Both qualitative and quantitative methods are used for evaluation, supplemented by a downstream Named Entity Recognition (NER) task. Our results indicate that encoder-only models outperform encoder-decoder models. General-domain and clinical-domain PLMs exhibit comparable performance when clinical information is preserved. Preserving clinical entities and document structure yields better performance than fine-tuning alone. Masking stopwords enhances text quality, whereas masking nouns or verbs has a negative impact. BERTScore proves to be the most reliable quantitative evaluation metric in our task. Contextual information has minimal impact, indicating that synthetic letters can effectively replace original ones in downstream tasks. Unlike previous studies that focus primarily on reconstructing original letters or training a privacy-detection and substitution model, this project provides a framework for generating diverse clinical letters while embedding privacy detection, enabling sensitive dataset expansion and facilitating the use of real-world clinical data. Our codes and trained models will be publicly available at https://github.com/HECTA-UoM/Synthetic4Health."
}