Diego Campanini
2024
iHealth-Chile-1 at RRG24: In-context Learning and Finetuning of a Large Multimodal Model for Radiology Report Generation
Diego Campanini
|
Oscar Loch
|
Pablo Messina
|
Rafael Elberg
|
Denis Parra
Proceedings of the 23rd Workshop on Biomedical Natural Language Processing
This paper presents the approach of the iHealth-Chile-1 team for the shared task of Large-Scale Radiology Report Generation at the BioNLP workshop, inspired by progress in large multimodal models for processing images and text. In this work, we leverage LLaVA, a Visual-Language Model (VLM), composed of a vision-encoder, a vision-language connector or adapter, and a large language model able to process text and visual embeddings. We achieve our best result by enriching the input prompt of LLaVA with the text output of a simpler report generation model. With this enriched-prompt technique, we improve our results in 4 of 5 metrics (BLEU-4, Rouge-L, BertScore and F1-RadGraph,), only doing in-context learning. Moreover, we provide details about different architecture settings, fine-tuning strategies, and dataset configurations.
iHealth-Chile-3&2 at RRG24: Template Based Report Generation
Oscar Loch
|
Pablo Messina
|
Rafael Elberg
|
Diego Campanini
|
Álvaro Soto
|
René Vidal
|
Denis Parra
Proceedings of the 23rd Workshop on Biomedical Natural Language Processing
This paper presents the approaches of the iHealth-Chile-3 and iHealth-Chile-2 teams for the shared task of Large-Scale Radiology Report Generation at the BioNLP workshop. Inspired by prior work on template-based report generation, both teams focused on exploring various template-based strategies, using predictions from multi-label image classifiers as input. Our best approach achieved a modest F1-RadGraph score of 19.42 on the findings hidden test set, ranking 7th on the leaderboard. Notably, we consistently observed a discrepancy between our classification metrics and the F1-CheXbert metric reported on the leaderboard, which always showed lower scores. This suggests that the F1-CheXbert metric may be missing some of the labels mentioned by the templates.
Search
Co-authors
- Oscar Loch 2
- Pablo Messina 2
- Rafael Elberg 2
- Denis Parra 2
- Álvaro Soto 1
- show all...