Analysing Representations of Memory Impairment in a Clinical Notes Classification Model
Mark Ormerod, Jesús Martínez-del-Rincón, Neil Robertson, Bernadette McGuinness, Barry Devereux
Abstract
Despite recent advances in the application of deep neural networks to various kinds of medical data, extracting information from unstructured textual sources remains a challenging task. The challenges of training and interpreting document classification models are amplified when dealing with small and highly technical datasets, as are common in the clinical domain. Using a dataset of de-identified clinical letters gathered at a memory clinic, we construct several recurrent neural network models for letter classification, and evaluate them on their ability to build meaningful representations of the documents and predict patients’ diagnoses. Additionally, we probe sentence embedding models in order to build a human-interpretable representation of the neural network’s features, using a simple and intuitive technique based on perturbative approaches to sentence importance. In addition to showing which sentences in a document are most informative about the patient’s condition, this method reveals the types of sentences that lead the model to make incorrect diagnoses. Furthermore, we identify clusters of sentences in the embedding space that correlate strongly with importance scores for each clinical diagnosis class.- Anthology ID:
- W19-5005
- Volume:
- Proceedings of the 18th BioNLP Workshop and Shared Task
- Month:
- August
- Year:
- 2019
- Address:
- Florence, Italy
- Editors:
- Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Junichi Tsujii
- Venue:
- BioNLP
- SIG:
- SIGBIOMED
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 48–57
- Language:
- URL:
- https://aclanthology.org/W19-5005
- DOI:
- 10.18653/v1/W19-5005
- Cite (ACL):
- Mark Ormerod, Jesús Martínez-del-Rincón, Neil Robertson, Bernadette McGuinness, and Barry Devereux. 2019. Analysing Representations of Memory Impairment in a Clinical Notes Classification Model. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 48–57, Florence, Italy. Association for Computational Linguistics.
- Cite (Informal):
- Analysing Representations of Memory Impairment in a Clinical Notes Classification Model (Ormerod et al., BioNLP 2019)
- PDF:
- https://preview.aclanthology.org/teach-a-man-to-fish/W19-5005.pdf