TLab at #SMM4H 2024: Retrieval-Augmented Generation for ADE Extraction and Normalization
Jacob Berkowitz, Apoorva Srinivasan, Jose Cortina, Nicholas Tatonetti1
Abstract
SMM4H 2024 Task 1 is focused on the identification of standardized Adverse Drug Events (ADEs) in tweets. We introduce a novel Retrieval-Augmented Generation (RAG) method, leveraging the capabilities of Llama 3, GPT-4, and the SFR-embedding-mistral model, along with few-shot prompting techniques, to map colloquial tweet language to MedDRA Preferred Terms (PTs) without relying on extensive training datasets. Our method achieved competitive performance, with an F1 score of 0.359 in the normalization task and 0.392 in the named entity recognition (NER) task. Notably, our model demonstrated robustness in identifying previously unseen MedDRA PTs (F1=0.363) greatly surpassing the median task score of 0.141 for such terms.- Anthology ID:
- 2024.smm4h-1.36
- Volume:
- Proceedings of The 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Dongfang Xu, Graciela Gonzalez-Hernandez
- Venues:
- SMM4H | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 153–157
- Language:
- URL:
- https://aclanthology.org/2024.smm4h-1.36
- DOI:
- Cite (ACL):
- Jacob Berkowitz, Apoorva Srinivasan, Jose Cortina, and Nicholas Tatonetti1. 2024. TLab at #SMM4H 2024: Retrieval-Augmented Generation for ADE Extraction and Normalization. In Proceedings of The 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks, pages 153–157, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- TLab at #SMM4H 2024: Retrieval-Augmented Generation for ADE Extraction and Normalization (Berkowitz et al., SMM4H-WS 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2024.smm4h-1.36.pdf