Intervention extraction in preclinical animal studies of Alzheimer’s Disease: Enhancing regex performance with language model-based filtering
Yiyuan Pu, Kaitlyn Hair, Daniel Beck, Mike Conway, Malcolm MacLeod, Karin Verspoor
Abstract
We explore different information extraction tools for annotation of interventions to support automated systematic reviews of preclinical AD animal studies. We compare two PICO (Population, Intervention, Comparison, and Outcome) extraction tools and two prompting-based learning strategies based on Large Language Models (LLMs). Motivated by the high recall of a dictionary-based approach, we define a two-stage method, removing false positives obtained from regexes with a pre-trained LM. With ChatGPT-based filtering using three-shot prompting, our approach reduces almost two-thirds of False Positives compared to the dictionary approach alone, while outperforming knowledge-free instructional prompting.- Anthology ID:
- 2024.bionlp-1.39
- Volume:
- Proceedings of the 23rd Workshop on Biomedical Natural Language Processing
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Dina Demner-Fushman, Sophia Ananiadou, Makoto Miwa, Kirk Roberts, Junichi Tsujii
- Venue:
- BioNLP
- SIG:
- SIGBIOMED
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 486–492
- Language:
- URL:
- https://aclanthology.org/2024.bionlp-1.39
- DOI:
- Cite (ACL):
- Yiyuan Pu, Kaitlyn Hair, Daniel Beck, Mike Conway, Malcolm MacLeod, and Karin Verspoor. 2024. Intervention extraction in preclinical animal studies of Alzheimer’s Disease: Enhancing regex performance with language model-based filtering. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 486–492, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Intervention extraction in preclinical animal studies of Alzheimer’s Disease: Enhancing regex performance with language model-based filtering (Pu et al., BioNLP 2024)
- PDF:
- https://preview.aclanthology.org/bionlp-24-ingestion/2024.bionlp-1.39.pdf