Abstract
We present a new approach to generating the ‘Findings’ and ‘Impression’ sections in the chest X-rays radiology reports, developed as part of the shared radiology task at BioNLP 2024. By integrating a DINOv2 vision encoder trained on medical data with specialized biomedical large language model using the LLaVA framework, our method addresses complex medical semantics and diverse findings in imaging. We use datasets from PadChest, BIMCV-COVID19, CheXpert, OpenI, and MIMIC-CXR. The evaluation metrics demonstrate our method’s effectiveness and the potential for automating the generation of radiology reports.- Anthology ID:
- 2024.bionlp-1.51
- Volume:
- Proceedings of the 23rd Workshop on Biomedical Natural Language Processing
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Dina Demner-Fushman, Sophia Ananiadou, Makoto Miwa, Kirk Roberts, Junichi Tsujii
- Venues:
- BioNLP | WS
- SIG:
- SIGBIOMED
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 603–607
- Language:
- URL:
- https://aclanthology.org/2024.bionlp-1.51
- DOI:
- Cite (ACL):
- Marina Munkhoeva, Dmitry Umerenkov, and Valentin Samokhin. 2024. AIRI at RRG24: LLaVa with specialised encoder and decoder. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 603–607, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- AIRI at RRG24: LLaVa with specialised encoder and decoder (Munkhoeva et al., BioNLP-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.bionlp-1.51.pdf