@inproceedings{wang-etal-2025-process,
    title = "Process-Supervised Reward Models for Verifying Clinical Note Generation: A Scalable Approach Guided by Domain Expertise",
    author = "Wang, Hanyin  and
      Gao, Chufan  and
      Xu, Qiping  and
      Liu, Bolun  and
      Hussein, Guleid  and
      Korsapati, Hariprasad Reddy  and
      El Labban, Mohamad  and
      Iheasirim, Kingsley  and
      Hassan, Mohamed  and
      Anil, Gokhan  and
      Bartlett, Brian  and
      Sun, Jimeng",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.967/",
    pages = "19138--19158",
    ISBN = "979-8-89176-332-6",
    abstract = "Process-supervised reward models (PRMs) excel at providing step-by-step verification for large language model (LLM) outputs in domains like mathematics and coding. However, their application to fields lacking ground-truth answers, such as clinical note generation, poses significant challenges. We introduce a novel framework for training PRMs to deliver step-level reward signals for LLM-generated clinical notes. By precisely defining meaningful ``steps,'' injecting realistic ``errors'' informed by domain expertise, and leveraging LLMs to generate process supervision data at scale, we overcome previous limitations. Our PRM, built on LLaMA-3.1 8B, consistently outperforms proprietary reasoning and non-reasoning models, achieving state-of-the-art performance on two key evaluations: (1) distinguishing gold-standard from error-containing samples with 98.8{\%} accuracy, and (2) selecting physician-preferred clinical notes with 56.2{\%} accuracy. We investigate critical components for effective PRM training, including optimal loss functions and data selection strategies, and present a comprehensive physician reader study identifying predictors of downstream Best-of-N performance. Our study sheds light on unlocking the potential of PRMs for diverse generative tasks across domains."
}