%
% File acl2020.tex
%
%% Based on the style files for ACL 2020, which were
%% Based on the style files for ACL 2018, NAACL 2018/19, which were
%% Based on the style files for ACL-2015, with some improvements
%%  taken from the NAACL-2016 style
%% Based on the style files for ACL-2014, which were, in turn,
%% based on ACL-2013, ACL-2012, ACL-2011, ACL-2010, ACL-IJCNLP-2009,
%% EACL-2009, IJCNLP-2008...
%% Based on the style files for EACL 2006 by 
%%e.agirre@ehu.es or Sergi.Balari@uab.es
%% and that of ACL 08 by Joakim Nivre and Noah Smith

\documentclass[11pt,a4paper]{article}
\usepackage[hyperref]{acl2020}
\usepackage{times}
\usepackage{latexsym}
\usepackage{graphicx}
\usepackage{CJK}
\usepackage{multirow}
\usepackage{booktabs}
\usepackage{amssymb}


\renewcommand{\UrlFont}{\ttfamily\small}

% This is not strictly necessary, and may be commented out,
% but it will improve the layout of the manuscript,
% and will typically save some space.
\usepackage{microtype}

\aclfinalcopy % Uncomment this line for the final submission
%\def\aclpaperid{***} %  Enter the acl Paper ID here

\setlength\titlebox{6.5cm}
% You can expand the titlebox if you need extra space
% to show all the authors. Please do not make the titlebox
% smaller than 5cm (the original size); we will check this
% in the camera-ready version and ask you to change it back.

\newcommand\BibTeX{B\textsc{ib}\TeX}


\title{Clinical-Coder: Assigning Interpretable ICD-10 Codes to Chinese Clinical Notes}


\author{
\textbf{Pengfei Cao}\footnotemark[1]  \textsuperscript{1,2},
\textbf{Chenwei Yan}\footnotemark[1]  \textsuperscript{3},\\ 
\textbf{Xiangling Fu}\textsuperscript{3}, 
\textbf{Yubo Chen}\textsuperscript{1,2}, 
\textbf{Kang Liu}\textsuperscript{1,2}, 
\textbf{Jun Zhao}\textsuperscript{1,2},
\textbf{Shengping Liu}\textsuperscript{4}, 
\textbf{Weifeng Chong}\textsuperscript{4},
\\
$^1$ National Laboratory of Pattern Recognition, Institute of Automation,\\ 
Chinese Academy of Sciences, Beijing, 100190, China\\
$^2$ School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, 100049, China\\
$^3$ Key Laboratory of Trustworthy Distributed Computing and Service(BUPT),\\ 
Ministry of Education, Beijing University of Posts and Telecommunications, Beijing, China\\
$^4$ Beijing Unisound Information Technology Co., Ltd, Beijing, 100028, China\\
\{pengfei.cao, yubo.chen, kliu, jzhao\}@nlpr.ia.ac.cn, \{2013213347, fuxiangling\}@bupt.edu.cn,\\
\{liushengping, chongweifeng\}@unisound.com
}



\date{}

\begin{document}
\begin{CJK}{UTF8}{gbsn}
\maketitle

\renewcommand{\thefootnote}{\fnsymbol{footnote}}
\footnotetext[1]{co-first authors, they contributed equally to this work}

\begin{abstract}
	In this paper, we introduce Clinical-Coder, an online system aiming to assign ICD codes to Chinese clinical notes. ICD coding has been a research hotspot of clinical medicine, but the interpretability of prediction hinders its practical application. We exploit a \textbf{D}ilated \textbf{C}onvolutional \textbf{A}ttention network with \textbf{N}-gram \textbf{M}atching Mechanism (DCANM) to capture semantic features for non-continuous words and continuous n-gram words, concentrating on explaining the reason why each ICD code to be predicted. The experiments demonstrate that our approach is effective and that our system is able to provide supporting information in clinical decision making.
\end{abstract}

\section{Introduction}

	International Classification of Disease (ICD) is the diagnostic classification standard in the field of clinical medicine, which assigns unique code to each disease. The popularization of ICD codes immensely promotes the information sharing and clinical research of disease worldwide and has a positive influence on health condition research, insurance claims, morbidity and mortality statistics \citep{Shi2017}. Therefore, ICD coding -- which assigns proper ICD codes to a clinical note -- has drawn much attention. 


	It is always that ICD coding relies on the manual work of professional staff. The manual coding is very error-prone and time-consuming since the continuous updating version of ICD codes results in a substantial increase in code numbers. The number of ICD-10 codes is up to 72,184, more than five times the previous version (i.e., ICD-9). It allows for more detailed classifications of patients' conditions, injuries, and diseases. However, there is no doubt that the increased granularity increases the difficulty of manual coding. 


	Existing studies came up with several approaches of automatic coding prediction to replace the repetitive manual work, from the traditional machine learning methods \citep{Adler2013, Koopman2015}, to neural network methods \citep{Shi2017, Yu2019}. Although these methods achieve great success, they are still confronted with a critical challenge, which is the interpretability of predicted codes. Explainable model and results are essential for clinical medicine decision making \citep{Mullenbach2018}. Thus, the practical approach is supposed to predict correct codes and simultaneously give the reason why each code is predicted. 


	\begin{figure}[ht]
	        \centering
	        \includegraphics[scale=0.25]{pictures/fig0.png}
	        \caption{Two kind of semantic phenomenon: explicit semantic features and implicit semantic features.}
	        \label{fig0}
	\end{figure}

	\begin{figure*}[ht]
	        \centering
	        \includegraphics[scale=0.375]{pictures/fig1.png}
	        \caption{The screenshot of Clinical-Coder system, the English version can be found in the appendix \ref{appendix1}. (a) gives the predicted diseases after users enter the clinical notes which contains four parts, admission situation, admission diagnosis, discharge situation and discharge diagnosis. (b1) and (b2) are the visualization of supporting information for predictions.}
	        \label{fig1}
	\end{figure*}

	In this paper, we try to provide the interpretability of predictions from a semantic perspective. It is a phenomenon that the exact disease names or similar expressions of disease names often appear in the discharge summary. For example, as shown in Figure \ref{fig0}, the exact matching with disease name such as ``fatty liver'' is a direct evidence of inference. We call the continuous consistent words as explicit semantic features. Moreover, the inexact matching such as ``rheumatoid multisite arthritis'' is also very useful to predict the codes and should be taken into consideration. We refer to the non-continuous words as implicit semantic features. The two kinds of semantic features are both clues to explain the reason why to assign each code, which is also the basis of experts in manual coding process. To capture the two semantic phenomena, we exploit dilated convolution and n-gram matching mechanism to extract implicit semantic features and explicit semantic features, respectively. Furthermore, we develop a system to assist the professional coders in assigning the correct codes. In summary, the main contributions are as follows:
	
	% Since the semantic infomation is crutial when understanding natural language\cite{Zuo2019EventCR}, t

	\begin{itemize}

		\item We collect a large-scale Chinese Clinical notes dataset, making up for the lack of Chinese ICD coding corpus.
		\item We propose a novel method to simultaneously capture implicit and explicit semantic features, which enables to give interpretability for each predicted code. 
		\item We develop an open-access online system, called clinical-coder, that automatically assigns codes to the free-text clinical notes with an indication of the supporting information for each code to be predicted. It uses vivid visualization to provide interpretability of prediction for each ICD code. The site can be accessed by \url{http://159.226.21.226/disease-prediction}, and instructions video is provided at \url{https://youtu.be/U4TImTwEysE}. 
		% 添加视频的链接
	\end{itemize}


	Figure \ref{fig1} illustrates an example of the automatic coding for a Chinese Clinical note in our system (For the convenience of readers, the English version is included in the appendix \ref{appendix1}). The left of Figure \ref{fig1} (a) is the free-text notes user entered, and the right of Figure \ref{fig1} (a) is predicted codes and corresponding disease names. Figure \ref{fig1} (b1) and Figure \ref{fig1} (b2) are the visualization of supporting information for predictions. The detailed description is presented in the section \ref{userinterface}. 



\section{Related Work}

	\begin{figure*}[ht]
	        \centering
	        \includegraphics[scale=0.43]{pictures/fig2.png}
	        \caption{The whole architecture of the model. The input is the clinical text, and output is the ICD codes. The yellow dotted box indicates how to use attention-based dilated convolution to capture the implicit semantic of non-continuous words. The green dotted box indicates how to use n-gram matching mechanism to capture the explicit semantic of continuous n-gram words.}
	        \label{fig2}

	\end{figure*}

	\subsection{Automatic ICD coding}

		Automatic ICD coding has recently been a research hotspot in the field of clinical medicine, where neural network architecture methods show promising results than traditional machine learning methods.

		Most studies treat automatic ICD coding as a multi-label classification problem and use only the free-text in summaries to predict codes \cite{Subotin2015, Kavuluru2015, Yu2019}, while many methods benefit from extra information. \citeauthor{Shi2017} \shortcite{Shi2017} encode label description with character-level and word-level long short-term memory network. \newcite{Rios2018} encode label description with averaging words embedding. Furthermore, adversarial learning is employed to unify writing styles of diagnosis descriptions and ICD code descriptions \cite{Pengtao2018A}. Besides code descriptions, Wikipedia comes to be regarded as an external knowledge source \cite{Prakash2017, Bai2019}.

		Additionally, inferring interpretability is a crucial challenge and obstacle for practical automatic coding, since professionals are willing to be convinced by the model insights of vital supporting information or decision-making process \cite{Vani2017, Mullenbach2018}. \newcite{Baumel2018} employ bidirectional Gated Recurrent Unit with sentence-level attention to obtain relevant sentences for each code. \newcite{Mullenbach2018} use attention at the word level, which is more fine-grained. Our work is inspired by \cite{Mullenbach2018}, assigning the importance value for each label to the discharge summaries to assists in explaining the model prediction process.

	\subsection{Dilated Convolution}
		Dilated convolution is designed for image classification to aggregate multi-scale contextual information without losing resolution in computer vision \cite{Yu2016}. It inserts ``holes'' in the standard convolution map to increase the reception field. The hole-structure brings a breakthrough improvement to the semantic segmentation task.

		Similarly, several hole-structured convolution neural networks (CNNs) \cite{Lei2015, Guo2017} are designed to handle natural language processing tasks. In the text, there exists non-continuous semantic where useless information may be interspersed among the sentences. Holes in the dilated convolution can ignore the extra word between the non-continuous words and well adapt to match non-continuous semantic. Since the semantic infomation is crutial when understanding natural language\cite{Zuo2019EventCR}, we apply the dilated convolution to encode the text, capturing the non-continuous semantic information. 


		% Similarly, some researchers explored several variants of CNNs to process continuous-word semantic. \citep{Lei2015} improve traditional convolution by the decay weight between continuous 3-grams to capture the variable spacing and recognize non-consecutive phrases. \citep{Guo2017} propose Skip Convolution which skip one gram in feature map to obtain more effective semantic. 


\section{Clinical Coder System}

	\subsection{Method}
		We propose a \textbf{D}ilated \textbf{C}onvolutional \textbf{A}ttention network with \textbf{N}-gram \textbf{M}atching Mechanism (DCANM) for ICD coding task. Figure \ref{fig2} describes the architecture of the model. The input of model is all sentences in clinical notes, which are spliced together. The input sentences interact with ICD code names to capture explicit semantic features and generate an n-gram matrix. At the same time, the input sentences are transformed into vector and processed by dilated CNN to capture implicit semantic features. Attention mechanism is used to improve the performance. Then all features are concatenated to form the final features. Finally, we use a sigmoid classifier to predict the probability of each code. Next, we give the detailed descriptions. 
			

		\paragraph{Word Embedding.} Word embedding is a low-dimensional vector representation of a word. We use the pre-trained embedded matrix $W^{wrd} \in \mathbb{R}^{d^w \times |V|}$, where $d^w$ is the dimension of word embedding and $|V|$ is the size of vocabulary. Given a sentence, $S = [w_1, w_2, ..., w_N]$, where $N$ is the number of words in the sentence, we can get the word embedding by: 

			\begin{equation}
				w_e=W^{wrd}v_i,  
			\end{equation}
		where $v_i$ is the one-hot representation of the current word in the corresponding column of $W^{wrd}$. 

		\paragraph{Explicit Semantic Features.} N-gram matching mechanism is applied to capture explicit semantic features. We use disease names (D) to sampling on the text (T). First, move the sliding window on the disease name $d_l \in D$ to get a n-gram substring. Then, calculate the frequency of each n-gram sub-string in the free-text. The sum of frequencies of gram with same length $n$ (denoted as $gram_n$) has reflected the emergence of disease names in the text, nevertheless some grams have their unique particularity. For example, given a 2-gram string, ``糖尿'' (Diabetes) is more representative than ``慢性''(Chronic) though they have the same length. To represent the degree of importance of different n-gram, each n-gram is given a term frequency-inverse document frequency (tf-idf) weight. Finally, for each free-text clinical note, we calculate an explicit semantic n-gram matrix ($M$) with size of $L \times W$, where $L$ is numbers of labels and $W$ is the numbers of sliding windows. For example, we have four sliding windows which lengths are 2, 3, 4, 5, so $W$ is 4. For the $l$-th row the $w$-th column item in the feature map, we have:
			\begin{equation}
				m_{l,w}=\sum_{i=1}^{L_{gram_{ln}}} count_{gram_{lni}} * tf\_idf_{gram_{lni}}
			\end{equation}
			\begin{equation}
				tf\_idf_{gram_i}= \frac{n}{L_{n_l}} *  \frac{L}{freq_{gram_{lni}}},
			\end{equation}
		where $w$ is the index of $n$-length sliding window, $gram_{ln}$ is all $n$-length substrings of the $l$-th disease name, $gram_{lni}$ is the $i$-th $gram_{ln}$, $L_{gram_{ln}}$ is the number of $gram_{ln}$, $count_{gram_{lni}}$ is the frequencies of $gram_{lni}$ in the text, $L_{n_l}$ is the length of the $l$-th disease name, $freq_{gram_{lni}}$ is the frequencies of $gram_{lni}$ in all disease names. 

		In this calculation, we can distinguish the importance degree of n-gram substring. It also works on English clinical notes, for instance, in a specific case from MIMIC-III \citep{Johnson2016}, the tf-idf value of ``history of'' is 1.79 while ``atrial fibrillation'' is 9.32 because ``history of'' appears 249 times in all ICD disease names and ``atrial fibrillation'' only appears two times. The higher the value is, the more representative the word is. Therefore ``atrial fibrillation'' is more likely to indicate a disease than ``history of''. 

		\paragraph{Implicit Semantic Features.} Dilated convolution is applied to capture implicit semantic features. For a long clinical text, dilated convolution extends the reception field in the situation of not using pooling operation so that every kernel has a wider range of information. More importantly, it has ``holes'' in convolution map, which means it can be adapted to match the non-continuous semantic information. For example, ``类风湿性多部位关节炎''(Rheumatoid multisite arthritis) in the clinical notes refers to ``类风湿性关节炎''(Rheumatoid arthritis) in ICD, the convolution map with holes can tolerate the redundant parts, as shown in Figure \ref{fig3}. It is a distinct advantage of dilated convolution for processing texts.

			 \begin{figure}[ht]
		        \centering
		        \includegraphics[scale=0.3]{pictures/fig3.png}
		        \caption{An example of the dilated convolution in processing text.}
		        \label{fig3}
			\end{figure}

		Formally, the actual filter width of dilated convolutional neural network is computed as,
			\begin{equation}
				k_d=r(k-1)+1,
			\end{equation}
		where $r \in [1,2,3,...]$ is the dilated rate, $k$ is the origin filter width.

		For each step $n$, the typical convolution is computed as formula \ref{cnn} and dilated convolution is computed as formula \ref{dcnn}. The dilated CNN is same as typical CNN when the dilated rate is 1, since $k_d$ equals to $k$ when $r=1$:
			\begin{equation}
				h_n=tanh(W_c*  x_{n:n+k-1}  +b_c)
				\label{cnn}
			\end{equation}
			\begin{equation}
				h_n^\prime=tanh(W_c*  x_{n:n+k_d-1}  +b_c),
				\label{dcnn}
			\end{equation}
where $W_c \in \mathbb{R}^{k_d \times d_e \times d_c}$ is the convolutional filter map, $k_d$ is the actual filter width, $d_e$ is the size of the word embedding, an $d_c$ the size of the filter output and $b_c \in \mathbb{R}^{d_c}$ is the bias.


		\paragraph{Attention.} After convolution, the sentence is represented as $H \in \mathbb{R}^{d_c \times N}$. We employ the per-label attention mechanism \citep{Mullenbach2018} to find the most contributed characters for each label. 

		For each label $l$, the distributed attention weight is computed as:
			\begin{equation}
				\alpha_l = SoftMax(H^{T} u_l),
			\end{equation}
where $u_l \in \mathbb{R}^{d_c}$ is the vector representation of label $l$. Finally, the sentence is represented as:
			\begin{equation}
				m_l=H\alpha_l
			\end{equation}

		We employ attention both for typical CNN and dilated CNN, for convenience of distinction, we denote them as $m_l$ and $m_l^\prime$, respectively.


		\paragraph{Classification.} $m_l$ and $m_l^\prime$ is concatenated with the linear transformed n-gram matrix horizontally. The aim of this step is to combining all the features together. Then we exploit sigmoid classifier and the prediction of label $i$ is computed as,
			\begin{equation}
				\hat{y_i} =  \sigma(W^{T}[m_l; m_l^\prime; m_l^{\prime\prime}] + b),
			\end{equation}
		where $i \in \left[1, 2, ..., L\right]$, $W \in \mathbb{R}^{3d_c}$, $b$ is the bias, $m_l^{\prime\prime}$ is the linear projection of n-gram matrix($M$).

		The loss function is the multi-label binary cross-entropy \citep{Nam2013}. 
			\begin{equation}
				\mathcal{L} =  \sum_{i=1}^{L} [ -y_ilog (\hat{y_i}) - (1-y_i) log(1-\hat{y_i}) ],
			\end{equation}
		where $y_i \in \{0, 1\}$ is the ground truth for the $i$-th label and $\hat{y_i}$ is the sigmoid score for the $i$-th label. 


	\subsection{User Interface}\label{userinterface}
		Figure \ref{fig1} illustrates the user interface of our system. 

		\paragraph{User Input.}
		The left of Figure \ref{fig1}(a) displays the user input. The user enters the whole free clinical note, which includes at least one from admission situation, admission diagnosis, discharge situation, and discharge diagnosis into the input box. 

		\paragraph{Predicted Labels.}
		The predicted labels are presented in the list of Figure \ref{fig1}(a), including disease name and homologous ICD codes. The number of predicted codes are not always the same as the diseases in discharge diagnosis, because clinicians may leave out certain diseases and several diagnoses should be combined into one ICD code\citep{Shi2017}. Our model can list all these diseases, and give the reason why they should be predicted.

		\paragraph{Interpretability.}
		Interpretability is a critical aspect of the decision-making system, especially in the clinical medicine domain. In our system, we give two ways, n-gram matching mechanism and attention, to assist users in understanding why each code is predicted. A user can know why the model predicted the labels, and what the key information in its decision was:
		
		(1) \textbf{N-gram Matching Mechanism.} When a patient suffering from a disease, the corresponding text span related to disease names often appear in the discharge summary. As shown in Figure \ref{fig1} (b1), the gram in disease name is highlighted to give a hint to users if it appears in the clinical text. Highlighting not only tells users why we predict each code but also prompts the place of the important information.
		
		(2) \textbf{Attention.} As shown in Figure \ref{fig1} (b2), the red background is attention distribution, and the darker the color is, the more useful the word is to predict the current label. The darker color is also helpful and attractive for human-being to double-check the correction of labels.


\section{Experiments}

	\subsection{Dataset}

		We evaluate our model on both Chinese and English datasets. The Chinese dataset, collected by us, contains 50,678 Chinese clinical notes and 6,200 unique ICD-10 codes. For each clinical note, it contains five parts: admission situation, admission diagnosis, discharge situation, discharge diagnosis and annotated ICD-10 codes. Admission situation involves chief complaints, past medical history, etc. Discharge situation involves the results of general examination. Admission diagnosis and discharge diagnosis involve disease names, which may not be totally consistent with standard names in ICD-10. The manually annotated codes are based on ICD-10, which are tagged by professional coders after reading through the whole clinical note. 

		\begin{table}[ht]
			\centering
			\scalebox{0.7}{
				\begin{tabular}{cccc}
				\toprule 
				\textbf{} & \textbf{CN-Full} & \textbf{CN-50} & \textbf{MIMIC-III-50}\\ \hline
				\# Samples & 50,678 & 36,758 & 9,795 \\
				\# Labels & 6200 & 50 & 50 \\
				Vocabulary size & 3,957 & 3,957 & 51,917 \\
				\# Average tokens per sample & 621 & 655 & 1,530 \\
				\# Average labels per sample & 4.3 & 2.6 & 5.7 \\
				\bottomrule
				\end{tabular}}
			\caption{Detailed information for three datasets.}
			\label{tab_DetailedDataset}
		\end{table}

		\begin{table*}[htb]
			\centering
			\scalebox{0.69}{
			\begin{tabular}{@{}ccccccccccccc@{}}

				\toprule
				\textbf{Dataset} & \multicolumn{6}{c}{\textbf{CN-Full}} & \multicolumn{6}{c}{\textbf{CN-50}} \\ \midrule
				\multirow{2}{*}{Model} & \multicolumn{2}{c}{\textbf{F1}} & \multicolumn{2}{c}{\textbf{AUC}} & \multicolumn{2}{c}{\textbf{R@k}} & \multicolumn{2}{c}{\textbf{F1}} & \multicolumn{2}{c}{\textbf{AUC}} & \multicolumn{2}{c}{\textbf{R@k}} \\ 
                & Macro      & Micro     & Macro      & Micro      & k=5        & k=10       & Macro      & Micro     & Macro      & Micro      & k=5        & k=8        \\ \cmidrule(r){1-13}
                
				CAML\cite{Mullenbach2018}        &0.0600& 	0.6755 &  	 0.8832     &  	0.9808 &0.6099     &    0.7651   &											0.8305  &  0.8458 &  0.9846 &  0.9902 &  0.8796  & 0.9579       \\\hline
				Dilated CNN    & 	0.1017  &  0.6997 &  0.8637 &  0.9772 &  0.6268  & 0.7864 &   											0.8399  &  0.8523 &  0.9849 &  0.9904 &  0.8807  & 0.9550       \\
				N-gram Matching & 	\textbf{0.1200}  &  0.7050 &  \textbf{0.9574} &  \textbf{0.9915} &  0.6393  & 0.8036 &   											0.8385  &  0.8543 &  0.9867 &  0.9922 &  0.8900  & 0.9640       \\\hline
				\textbf{DACNM} & 0.1116	&  \textbf{0.7127} &  0.9520 &  0.9909 &  \textbf{0.6430}  & \textbf{0.8043} &   			\textbf{0.8452}  &  \textbf{0.8602} &  \textbf{0.9878} &  \textbf{0.9932} &  \textbf{0.8895}  & \textbf{0.9657}       \\ \bottomrule
			\end{tabular}}
		\caption{Evaluation on Chinese dataset CN-Full and CN-50.}
		\label{tab_chinese}
		\end{table*}

		The dataset (CN-Full) is formed with full labels mentioned above, and it is divided into train set and test set with the radio of 9:1. In addition, due to the phenomenon that massive codes are infrequent, and a small amount of codes are high-frequent, we reconstructed a sub-dataset (CN-50) with the most frequent 50 codes from the original dataset. The specific process is that filtering the origin train set and test set, and maintain the data which has at least one of the top 50 most frequent codes.

		% MIMIC-III is an open-access English dataset. 
		% \citep{Mullenbach2018, Baumel2018, Prakash2017, Shi2017}
		To better compare with the previous works, we also evaluate our method on the MIMIC-III dataset \citep{Johnson2016}, which is the most authoritative English dataset for evaluating the performance of automatic ICD coding approaches. The detailed description for these datasets is listed in Table \ref{tab_DetailedDataset}.

		
	\subsection{Data Preprocess and Parameters}
		We splice the admission situation, admission diagnosis, discharge situation and discharge diagnosis together, which is the input of the model. The max length of the input is 1000. The word embedding is pre-trained using Word2Vec \citep{Mikolov2013} with the dimensions of 100. The text is from all clinical notes. The batch size is 16. The dropout rate is 0.5. The optimizer is Adam \citep{adam} with a learning rate of 0.0001. 

		We use Micro-F1, Macro-F1, area under the ROC (Receiver Operating Characteristic) curve (AUC) and P@k as the metrics. P@k (Precision at k) is the fraction of the $k$ highest-scored labels that are present in the ground truth.

	\subsection{Results}
		First, for the Chinese dataset (CN-Full and CN-50), CAML \citep{Mullenbach2018} is set as our baseline, which use traditional convolutional attention network. Moreover, we test the dilated CNN and n-gram matching mechanism separately. The results in Table \ref{tab_chinese} indicate that dilated CNN and n-gram matching mechanism both have a positive effect on improving performance from baseline, and the best results are obtained when they combined.

		We also evaluate our method on English dataset (MIMIC-III-50). The results are shown in Table \ref{tab_compare_50}. The CNN and Bi-GRU are the classic methods and the results are the same as \citep{Mullenbach2018}. Our proposed model achieves the Micro-F1 score of 0.641, which outperforms all previous works, more importantly providing interpretability. 


		\begin{table}[ht]
			\centering
			\scalebox{0.57}{
				\begin{tabular}{cccccc}
					\toprule 
		            \multirow{2}{*}{\textbf{Model}} & \multicolumn{2}{c}{\textbf{F1}} & \multicolumn{2}{c}{\textbf{AUC}} & \textbf{P@k} \\
		            \cmidrule(lr){2-6} 
		            & Macro & Micro & Macro & Micro & k=5      \\
		            \hline 
					CNN    		   &       0.576    &     0.625     &    0.876   &    0.907   &    \textbf{0.620}     \\
					Bi-GRU         &       0.484    &     0.549     &    0.828   &    0.868   &    0.591             \\
					C-MemNN\citep{Prakash2017} &  - &       -       &    0.833   &      -     &    0.42             \\
					\citep{Shi2017}&  -             &     0.532     &    -       &    0.900   &      -            \\
					HA-GRU\citep{Baumel2018}&  -    &     0.366     &    -       &      -     &      -            \\
					CAML\citep{Mullenbach2018}   &   0.532    &    0.614     &    0.875   &    0.909   &  0.609        \\
					DR-CAML\citep{Mullenbach2018}&   0.576    &    0.633     &    0.884   &    0.916   &  0.618       \\
					\hline
					\textbf{DACNM} (Proposed model) & \textbf{0.579}   &\textbf{0.641}    &\textbf{0.890}  &\textbf{0.916}  &    0.616         \\
					\bottomrule
				\end{tabular}}
			\caption{Evaluation on MIMIC-III-50 dataset}
			\label{tab_compare_50}
		\end{table}


		Besides, we notice that macro-F1 measure is always lower than micro-F1, especially in the full labels datasets. It means the smaller classes have poorer performance than larger classes, which is consistent with the facts. Either MIMIC-III or the Chinese dataset, the sample distributions are extremely imbalanced. Minority of codes are highly frequent, while most codes are infrequent. N-gram matching mechanism helps improve macro-F1 on CN-Full dataset obviously, reaching two times than baseline. It can be inferred that utilizing grams in disease names is useful for the smaller class.


\section{Conclusion}
	In this paper, we propose a \textbf{D}ilated \textbf{C}onvolutional \textbf{A}ttention network with \textbf{N}-gram \textbf{M}atching Mechanism (DCANM) for automatic ICD coding. The dilated CNN, which is first applied to the ICD coding task, aims to capture semantic information for non-continuous words, and the n-gram matching mechanism aims to capture the continuous semantic. They both provide a pretty good interpretability for prediction. Moreover, we develop an open-access system to help users assign ICD codes. We will try to utilize external resources to solve the few-shot and zero-shot problem in the future. 

\section*{Acknowledgments}

	This work is supported by the National Natural Science Foundation of China (No.61922085, No.61533018, No.61976211) and the Key Research Program of the Chinese Academy of Sciences (Grant NO.ZDBS-SSW-JSC006). This work is also supported by Beijing Academy of Artificial Intelligence (BAAI2019QN0301) and a grant from Ant Financial Services Group.

\bibliography{anthology,acl2020}
\bibliographystyle{acl_natbib}

\appendix

\section{Appendix: English Version of Figure 2(a)}
\label{appendix1}
Since the online system is for Chinese clinical notes, we provide corresponding English version for reading convenience.


\paragraph{Admissions situation:}

\paragraph{}The middle-aged man was admitted to the hospital because of a ``blood glucose increase of 5 years and poor glycemic control in June''. The patient's routine physical examination five years ago revealed an increase in blood glucose, about 9 mmol / L on an empty stomach, and the local diagnosis was: ``type 2 diabetes''. No obvious dry mouth, frequent drinking, polyuria and weight loss, no dizziness, no increased urine foam, no numbness in hands, feet, and occasionally blurred vision. Diet and exercise were used to control blood glucose. Normally, blood glucose was monitored irregularly, and fasting blood glucose fluctuated between 8-9mmol / L. During the course of the disease, the general condition of the patient is OK, the diet and sleep are OK, and there is no obvious abnormality in the stool. The weight loss in the last 2 months is about 3Kg. Auxiliary examination B: Ultralow hypoechoic nodules in the right lobe of the thyroid gland, considering glial cysts; bilateral carotid atherosclerosis with right plaque formation; fatty liver; right renal cyst with calcium milk; enlarged prostate; Atherosclerosis.

\paragraph{Admission diagnosis:}


\paragraph{}\textbf{1.} Type 2 diabetes \textbf{2.} Kidney stones \textbf{3.} Thyroid nodules \textbf{4.} Atherosclerosis \textbf{5.} Fatty liver


\paragraph{Discharge situation:}

\paragraph{}The patient had no dry mouth and frequent drinking, no polyuria, diet and sleep were OK, physical examination: clear mind, good spirits, slightly thicker breathing sounds in both lungs, and no wet and dry rales. Heart rhythm is uniform, and no noise is heard. The abdomen is flat, the whole abdomen is soft, no tenderness, no tenderness, no edema in both lower limbs.

\paragraph{Discharge diagnosis:}

\paragraph{}\textbf{1.}Type 2 diabetes \textbf{2.} Right renal cyst \textbf{3.} Thyroid nodule \textbf{4.} Atherosclerosis \textbf{5.} Fatty liver \textbf{6.} Hyperlipidemia

\paragraph{Predicted diseases and codes:}


\paragraph{}
\quad   \textbf{1.} Thyroid nodule    	E04.101

\quad    \textbf{2.} Type 2 diabetes   	E11.9022

\quad    \textbf{3.} Hyperlipidemia    	E78.501

\quad    \textbf{4.} Atherosclerosis   	I70.904

\quad    \textbf{5.} Fatty liver       	K76.001

\quad    \textbf{6.} Kidney stones     	N20.000

\quad    \textbf{7.} Acquired renal cysts N28.100



\end{CJK}
\end{document}
