Abstract
Biomedical relation extraction, aiming to automatically discover high-quality and semantic relations between the entities from free text, is becoming a vital step for automated knowledge discovery. Pretrained language models have achieved impressive performance on various natural language processing tasks, including relation extraction. In this paper, we perform extensive empirical comparisons of encoder-only transformers with the encoder-decoder transformer, specifically T5, on ten public biomedical relation extraction datasets. We study the relation extraction task from four major biomedical tasks, namely chemical-protein relation extraction, disease-protein relation extraction, drug-drug interaction, and protein-protein interaction. We also explore the use of multi-task fine-tuning to investigate the correlation among major biomedical relation extraction tasks. We report performance (micro F-score) using T5, BioBERT and PubMedBERT, demonstrating that T5 and multi-task learning can improve the performance of the biomedical relation extraction task.- Anthology ID:
- 2022.bionlp-1.37
- Volume:
- Proceedings of the 21st Workshop on Biomedical Language Processing
- Month:
- May
- Year:
- 2022
- Address:
- Dublin, Ireland
- Editors:
- Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Junichi Tsujii
- Venue:
- BioNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 376–382
- Language:
- URL:
- https://aclanthology.org/2022.bionlp-1.37
- DOI:
- 10.18653/v1/2022.bionlp-1.37
- Cite (ACL):
- Mourad Sarrouti, Carson Tao, and Yoann Mamy Randriamihaja. 2022. Comparing Encoder-Only and Encoder-Decoder Transformers for Relation Extraction from Biomedical Texts: An Empirical Study on Ten Benchmark Datasets. In Proceedings of the 21st Workshop on Biomedical Language Processing, pages 376–382, Dublin, Ireland. Association for Computational Linguistics.
- Cite (Informal):
- Comparing Encoder-Only and Encoder-Decoder Transformers for Relation Extraction from Biomedical Texts: An Empirical Study on Ten Benchmark Datasets (Sarrouti et al., BioNLP 2022)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2022.bionlp-1.37.pdf
- Data
- DDI