PLM-ICD: Automatic ICD Coding with Pretrained Language Models

Chao-Wei Huang, Shang-Chi Tsai, Yun-Nung Chen


Abstract
Automatically classifying electronic health records (EHRs) into diagnostic codes has been challenging to the NLP community. State-of-the-art methods treated this problem as a multi-label classification problem and proposed various architectures to model this problem. However, these systems did not leverage the superb performance of pretrained language models, which achieved superb performance on natural language understanding tasks. Prior work has shown that pretrained language models underperformed on this task with the regular fine-tuning scheme. Therefore, this paper aims at analyzing the causes of the underperformance and developing a framework for automatic ICD coding with pretrained language models. We spotted three main issues through the experiments: 1) large label space, 2) long input sequences, and 3) domain mismatch between pretraining and fine-tuning. We propose PLM-ICD, a framework that tackles the challenges with various strategies. The experimental results show that our proposed framework can overcome the challenges and achieves state-of-the-art performance in terms of multiple metrics on the benchmark MIMIC data. Our source code is available at https://github.com/MiuLab/PLM-ICD.
Anthology ID:
2022.clinicalnlp-1.2
Volume:
Proceedings of the 4th Clinical Natural Language Processing Workshop
Month:
July
Year:
2022
Address:
Seattle, WA
Editors:
Tristan Naumann, Steven Bethard, Kirk Roberts, Anna Rumshisky
Venue:
ClinicalNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10–20
Language:
URL:
https://aclanthology.org/2022.clinicalnlp-1.2
DOI:
10.18653/v1/2022.clinicalnlp-1.2
Bibkey:
Cite (ACL):
Chao-Wei Huang, Shang-Chi Tsai, and Yun-Nung Chen. 2022. PLM-ICD: Automatic ICD Coding with Pretrained Language Models. In Proceedings of the 4th Clinical Natural Language Processing Workshop, pages 10–20, Seattle, WA. Association for Computational Linguistics.
Cite (Informal):
PLM-ICD: Automatic ICD Coding with Pretrained Language Models (Huang et al., ClinicalNLP 2022)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-3/2022.clinicalnlp-1.2.pdf
Video:
 https://preview.aclanthology.org/nschneid-patch-3/2022.clinicalnlp-1.2.mp4
Code
 miulab/plm-icd +  additional community code