Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges
Sanjana Ramprasad, Jered Mcinerney, Iain Marshall, Byron Wallace
Abstract
In this work we present TrialsSummarizer, a system that aims to automatically summarize evidence presented in the set of randomized controlled trials most relevant to a given query. Building on prior work, the system retrieves trial publications matching a query specifying a combination of condition, intervention(s), and outcome(s), and ranks these according to sample size and estimated study quality. The top-k such studies are passed through a neural multi-document summarization system, yielding a synopsis of these trials. We consider two architectures: A standard sequence-to-sequence model based on BART, and a multi-headed architecture intended to provide greater transparency and controllability to end-users. Both models produce fluent and relevant summaries of evidence retrieved for queries, but their tendency to introduce unsupported statements render them inappropriate for use in this domain at present. The proposed architecture may help users verify outputs allowing users to trace generated tokens back to inputs. The demonstration video can be found at https://vimeo.com/735605060The prototype, source code, and model weights are available at: https://sanjanaramprasad.github.io/trials-summarizer/- Anthology ID:
- 2023.eacl-demo.27
- Volume:
- Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations
- Month:
- May
- Year:
- 2023
- Address:
- Dubrovnik, Croatia
- Editors:
- Danilo Croce, Luca Soldaini
- Venue:
- EACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 236–247
- Language:
- URL:
- https://aclanthology.org/2023.eacl-demo.27
- DOI:
- 10.18653/v1/2023.eacl-demo.27
- Cite (ACL):
- Sanjana Ramprasad, Jered Mcinerney, Iain Marshall, and Byron Wallace. 2023. Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 236–247, Dubrovnik, Croatia. Association for Computational Linguistics.
- Cite (Informal):
- Automatically Summarizing Evidence from Clinical Trials: A Prototype Highlighting Current Challenges (Ramprasad et al., EACL 2023)
- PDF:
- https://preview.aclanthology.org/naacl24-info/2023.eacl-demo.27.pdf