LlamaMTS: Optimizing Metastasis Detection with Llama Instruction Tuning and BERT-Based Ensemble in Italian Clinical Reports
Livia Lilli, Stefano Patarnello, Carlotta Masciocchi, Valeria Masiello, Fabio Marazzi, Tagliaferri Luca, Nikola Capocchiano
Abstract
Information extraction from Electronic Health Records (EHRs) is a crucial task in healthcare, and the lack of resources and language specificity pose significant challenges. This study addresses the limited availability of Italian Natural Language Processing (NLP) tools for clinical applications and the computational demand of large language models (LLMs) for training. We present LlamaMTS, an instruction-tuned Llama for the Italian language, leveraging the LoRA technique. It is ensembled with a BERT-based model to classify EHRs based on the presence or absence of metastasis in patients affected by Breast cancer. Through our evaluation analysis, we discovered that LlamaMTS exhibits superior performance compared to both zero-shot LLMs and other Italian BERT-based models specifically fine-tuned on the same metastatic task. LlamaMTS demonstrates promising results in resource-constrained environments, offering a practical solution for information extraction from Italian EHRs in oncology, potentially improving patient care and outcomes.- Anthology ID:
- 2024.clinicalnlp-1.13
- Volume:
- Proceedings of the 6th Clinical Natural Language Processing Workshop
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Tristan Naumann, Asma Ben Abacha, Steven Bethard, Kirk Roberts, Danielle Bitterman
- Venues:
- ClinicalNLP | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 162–171
- Language:
- URL:
- https://aclanthology.org/2024.clinicalnlp-1.13
- DOI:
- Cite (ACL):
- Livia Lilli, Stefano Patarnello, Carlotta Masciocchi, Valeria Masiello, Fabio Marazzi, Tagliaferri Luca, and Nikola Capocchiano. 2024. LlamaMTS: Optimizing Metastasis Detection with Llama Instruction Tuning and BERT-Based Ensemble in Italian Clinical Reports. In Proceedings of the 6th Clinical Natural Language Processing Workshop, pages 162–171, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- LlamaMTS: Optimizing Metastasis Detection with Llama Instruction Tuning and BERT-Based Ensemble in Italian Clinical Reports (Lilli et al., ClinicalNLP-WS 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.clinicalnlp-1.13.pdf