UniBuc at SemEval-2024 Task 2: Tailored Prompting with Solar for Clinical NLI
Marius Micluta-Campeanu, Claudiu Creanga, Ana-maria Bucur, Ana Sabina Uban, Liviu P. Dinu
Abstract
This paper describes the approach of the UniBuc team in tackling the SemEval 2024 Task 2: Safe Biomedical Natural Language Inference for Clinical Trials. We used SOLAR Instruct, without any fine-tuning, while focusing on input manipulation and tailored prompting. By customizing prompts for individual CTR sections, in both zero-shot and few-shots settings, we managed to achieve a consistency score of 0.72, ranking 14th in the leaderboard. Our thorough error analysis revealed that our model has a tendency to take shortcuts and rely on simple heuristics, especially when dealing with semantic-preserving changes.- Anthology ID:
- 2024.semeval-1.88
- Volume:
- Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)
- Month:
- June
- Year:
- 2024
- Address:
- Mexico City, Mexico
- Editors:
- Atul Kr. Ojha, A. Seza Doğruöz, Harish Tayyar Madabushi, Giovanni Da San Martino, Sara Rosenthal, Aiala Rosá
- Venue:
- SemEval
- SIG:
- SIGLEX
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 586–595
- Language:
- URL:
- https://aclanthology.org/2024.semeval-1.88
- DOI:
- Cite (ACL):
- Marius Micluta-Campeanu, Claudiu Creanga, Ana-maria Bucur, Ana Sabina Uban, and Liviu P. Dinu. 2024. UniBuc at SemEval-2024 Task 2: Tailored Prompting with Solar for Clinical NLI. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 586–595, Mexico City, Mexico. Association for Computational Linguistics.
- Cite (Informal):
- UniBuc at SemEval-2024 Task 2: Tailored Prompting with Solar for Clinical NLI (Micluta-Campeanu et al., SemEval 2024)
- PDF:
- https://preview.aclanthology.org/jeptaln-2024-ingestion/2024.semeval-1.88.pdf