Emma McKibbin
2026
Response Content Units: Evaluating Completeness and Proactiveness in Medical Open-Response Question Answering
Yongsin Park | Wen-wai Yim | Emma McKibbin | Asma Ben Abacha | Fei Xia
Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM)
Yongsin Park | Wen-wai Yim | Emma McKibbin | Asma Ben Abacha | Fei Xia
Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM)
Remote clinical care has significantly increased the workload for healthcare professionals managing digital inquiries. While automated systems aim to alleviate this burden, consumer health questions present unique challenges due to their linguistic complexity and the need for proactive clinical guidance, which traditional question-answering models often overlook. We introduce the medical Response Content Units (RCU) schema, a framework that facilitates automatic analysis to identify question-answer completeness and critical answer subparts, which can then be used as tools for supporting clinician response or for automatic metric evaluation. Our analysis using this schema reveals a 16.4% gap in response completeness in professional replies and demonstrates that essential medical directives are provided 2.4 to 12.1 times as frequently as direct answers. We provide baseline results and publicly release our annotations and source code to offer an evaluation framework that is more closely aligned with real-world clinical requirements.