@inproceedings{bauer-etal-2024-ikim,
    title = "{IKIM} at {MEDIQA}-{M}3{G} 2024: Multilingual Visual Question-Answering for Dermatology through {VLM} Fine-tuning and {LLM} Translations",
    author = "Bauer, Marie  and
      Seibold, Constantin  and
      Kleesiek, Jens  and
      Dada, Amin",
    editor = "Naumann, Tristan  and
      Ben Abacha, Asma  and
      Bethard, Steven  and
      Roberts, Kirk  and
      Bitterman, Danielle",
    booktitle = "Proceedings of the 6th Clinical Natural Language Processing Workshop",
    month = jun,
    year = "2024",
    address = "Mexico City, Mexico",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/fix-sig-urls/2024.clinicalnlp-1.44/",
    doi = "10.18653/v1/2024.clinicalnlp-1.44",
    pages = "439--447",
    abstract = "This paper presents our solution to the MEDIQA-M3G Challenge at NAACL-ClinicalNLP 2024. We participated in all three languages, ranking first in Chinese and Spanish and third in English. Our approach utilizes LLaVA-med, an open-source, medical vision-language model (VLM) for visual question-answering in Chinese, and Mixtral-8x7B-instruct, a Large Language Model (LLM) for a subsequent translation into English and Spanish. In addition to our final method, we experiment with alternative approaches: Training three different models for each language instead of translating the results from one model, using different combinations and numbers of input images, and additional training on publicly available data that was not part of the original challenge training set."
}