CogStack-KCL-UCL at ArchEHR-QA 2025: Investigating Hybrid LLM Approaches for Grounded Clinical Question Answering

Shubham Agarwal, Thomas Searle, Kawsar Noor, Richard Dobson


Abstract
We present our system for the ArchEHR shared task, which focuses on answering clinical and patient-facing questions grounded in real-world EHR data. Our core contribution is a 2-Stage prompting pipeline that separates evidence selection from answer generation while employing in-context learning strategies. Our experimentation leveraged the open-weight Gemma-v3 family of models, with our best submission using the Gemma-12B model securing 5th place overall on the unseen test set. Through systematic experimentation, we demonstrate the effectiveness of task decomposition in improving both factual accuracy and answer relevance in grounded clinical question answering.
Anthology ID:
2025.bionlp-share.16
Volume:
BioNLP 2025 Shared Tasks
Month:
August
Year:
2025
Address:
Vienna, Austria
Editors:
Sarvesh Soni, Dina Demner-Fushman
Venues:
BioNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
126–135
Language:
URL:
https://preview.aclanthology.org/acl25-workshop-ingestion/2025.bionlp-share.16/
DOI:
Bibkey:
Cite (ACL):
Shubham Agarwal, Thomas Searle, Kawsar Noor, and Richard Dobson. 2025. CogStack-KCL-UCL at ArchEHR-QA 2025: Investigating Hybrid LLM Approaches for Grounded Clinical Question Answering. In BioNLP 2025 Shared Tasks, pages 126–135, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
CogStack-KCL-UCL at ArchEHR-QA 2025: Investigating Hybrid LLM Approaches for Grounded Clinical Question Answering (Agarwal et al., BioNLP 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/acl25-workshop-ingestion/2025.bionlp-share.16.pdf