Biomedical Concept Relatedness – A large EHR-based benchmark

Claudia Schulz, Josh Levy-Kramer, Camille Van Assel, Miklos Kepes, Nils Hammerla


Abstract
A promising application of AI to healthcare is the retrieval of information from electronic health records (EHRs), e.g. to aid clinicians in finding relevant information for a consultation or to recruit suitable patients for a study. This requires search capabilities far beyond simple string matching, including the retrieval of concepts (diagnoses, symptoms, medications, etc.) related to the one in question. The suitability of AI methods for such applications is tested by predicting the relatedness of concepts with known relatedness scores. However, all existing biomedical concept relatedness datasets are notoriously small and consist of hand-picked concept pairs. We open-source a novel concept relatedness benchmark overcoming these issues: it is six times larger than existing datasets and concept pairs are chosen based on co-occurrence in EHRs, ensuring their relevance for the application of interest. We present an in-depth analysis of our new dataset and compare it to existing ones, highlighting that it is not only larger but also complements existing datasets in terms of the types of concepts included. Initial experiments with state-of-the-art embedding methods show that our dataset is a challenging new benchmark for testing concept relatedness models.
Anthology ID:
2020.coling-main.577
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
6565–6575
Language:
URL:
https://aclanthology.org/2020.coling-main.577
DOI:
10.18653/v1/2020.coling-main.577
Bibkey:
Cite (ACL):
Claudia Schulz, Josh Levy-Kramer, Camille Van Assel, Miklos Kepes, and Nils Hammerla. 2020. Biomedical Concept Relatedness – A large EHR-based benchmark. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6565–6575, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
Biomedical Concept Relatedness – A large EHR-based benchmark (Schulz et al., COLING 2020)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingestion-script-update/2020.coling-main.577.pdf
Code
 babylonhealth/EHR-Rel
Data
EHR-Rel