A distantly supervised dataset for automated data extraction from diagnostic studies

Christopher Norman, Mariska Leeflang, René Spijker, Evangelos Kanoulas, Aurélie Névéol

Abstract
Systematic reviews are important in evidence based medicine, but are expensive to produce. Automating or semi-automating the data extraction of index test, target condition, and reference standard from articles has the potential to decrease the cost of conducting systematic reviews of diagnostic test accuracy, but relevant training data is not available. We create a distantly supervised dataset of approximately 90,000 sentences, and let two experts manually annotate a small subset of around 1,000 sentences for evaluation. We evaluate the performance of BioBERT and logistic regression for ranking the sentences, and compare the performance for distant and direct supervision. Our results suggest that distant supervision can work as well as, or better than direct supervision on this problem, and that distantly trained models can perform as well as, or better than human annotators.
Anthology ID:
W19-5012
Volume:
Proceedings of the 18th BioNLP Workshop and Shared Task
Month:
August
Year:
2019
Address:
Florence, Italy
Editors:
Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Junichi Tsujii
Venue:
BioNLP
SIG:
SIGBIOMED
Publisher:
Association for Computational Linguistics
Note:
Pages:
105–114
Language:
URL:
https://aclanthology.org/W19-5012
DOI:
10.18653/v1/W19-5012
Bibkey:
Cite (ACL):
Christopher Norman, Mariska Leeflang, René Spijker, Evangelos Kanoulas, and Aurélie Névéol. 2019. A distantly supervised dataset for automated data extraction from diagnostic studies. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 105–114, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
A distantly supervised dataset for automated data extraction from diagnostic studies (Norman et al., BioNLP 2019)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-bitext-workshop/W19-5012.pdf
PDF Search