CogStack-KCL-UCL at ArchEHR-QA 2025: Investigating Hybrid LLM Approaches for Grounded Clinical Question Answering

Shubham Agarwal, Thomas Searle, Kawsar Noor, Richard Dobson


Abstract
We present our system for the ArchEHR shared task, which focuses on answering clinical and patient-facing questions grounded in real-world EHR data. Our core contribution is a 2-Stage prompting pipeline that separates evidence selection from answer generation while employing in-context learning strategies. Our experimentation leveraged the open-weight Gemma-v3 family of models, with our best submission using the Gemma-12B model securing 5th place overall on the unseen test set. Through systematic experimentation, we demonstrate the effectiveness of task decomposition in improving both factual accuracy and answer relevance in grounded clinical question answering.
Anthology ID:
2025.bionlp-share.16
Volume:
Proceedings of the 24th Workshop on Biomedical Language Processing (Shared Tasks)
Month:
August
Year:
2025
Address:
Vienna, Austria
Editors:
Sarvesh Soni, Dina Demner-Fushman
Venues:
BioNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
126–135
Language:
URL:
https://preview.aclanthology.org/landing_page/2025.bionlp-share.16/
DOI:
Bibkey:
Cite (ACL):
Shubham Agarwal, Thomas Searle, Kawsar Noor, and Richard Dobson. 2025. CogStack-KCL-UCL at ArchEHR-QA 2025: Investigating Hybrid LLM Approaches for Grounded Clinical Question Answering. In Proceedings of the 24th Workshop on Biomedical Language Processing (Shared Tasks), pages 126–135, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
CogStack-KCL-UCL at ArchEHR-QA 2025: Investigating Hybrid LLM Approaches for Grounded Clinical Question Answering (Agarwal et al., BioNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2025.bionlp-share.16.pdf