\section{Related Work}
\label{rel}
\subsection{Patent Classification}
\label{patent cls}
Most of the previous patent classification systems focus on developing more features derived from patent structures with traditional learning algorithms \cite{Verberne2011}. Going beyond document-level features, \citet{Cai2007} and \citet{Qiu2011} capture the label hierarchy in model representations. It is flexible for  traditional learning algorithms to integrate prior knowledge; however, it is difficult for them to generalize to unseen data.  

Deep Neural Networks make efficient use of large-scale training data and  generalize well to unseen data. There are a few NN-based patent classification methods. \citet{Grawe2017} uses Long Short Term Memory (LSTM) on 50 IPC subgroups and \citet{Li2018} applies Convolutional Neural Networks (CNN) to IPC subclasses classification. 

There are some efforts on mapping papers to IPC in the context of patent retrieval combined with K nearest neighbours (KNN) classification. \citet{cao:2008} adopt a query-expansion approach to retrieve relevant patents and use a KNN classifier to label the research paper.  \citet{Xiao2008} combine different scoring methods to rerank the retrieved IPC and achieved the best performance in the NTCIR-7 workshop for classify research papers to IPC system \cite{ntcir:report}. To our best knowledge, there has been no attempt tackling both tasks in one model.

%\cite{Gomez2014} uses Principle Component Analysis to map patents to low-rank space. 

%which lends itself to many NLP applications, such as named entity typing \cite{Yogatama2015} and sentiment analysis \cite{Severyn2015}.

%Most of the traditional text classification methods are built upon bag-of-words features. \cite{Yang1997, Bakeryz1998} select the most salient words or statistics from the document-end, and \cite{Scott1998,Garla2012} derive class-dependent similarity measures from the label-end. Although it is flexible to integrate prior knowledge via such feature engineering, it is difficult for the traditional learning methods to generalize.

%Text Classification is one of the most fundamental tasks in Natural Language Processing. Most of the traditional systems are built upon bag-of-words features \cite{McCallum1998,Lodhi2002}. Most of the traditional text classification systems are built upon bag-of-words features, via models ranging from Naive Bayes \cite{McCallum1998},  to Support Vector Machines \cite{Lodhi2002}. To reduce the dimension of the document representation, various feature selection methods have been proposed to pick the most salient words or statistics from the original set \cite{Yang1997, Bakeryz1998}. Beyond the document features, \cite{Scott1998,Garla2012} utilize the domain knowledge encoded in taxonomies to derive classification-target-dependent similarity measures between words. Feature engineering enables traditional learning algorithms to flexibly integrate prior knowledge, however, it is difficult for them to generalize to unseen data. %

%Recently, Artificial Neural Networks (ANN) give the state-of-the-art performance on text classification \cite{Kim2014,Lai2015}. It leverages distributed representations of characters/words (e.g. word embeddings \cite{Mikolov2013}). Word embeddings capture the semantic relations between words. Deep Neural Networks (DNN) make efficient use of large-scale training data and  generalize well to unseen data. 


%There were shared tasks on patent classification evaluating on the subclass level \cite{Piroi2011} and the section level \cite{Molla2018}. \cite{Verberne2011}  

% In this paper, we aim to classify both patents and papers to CPC \textit{subclasses} with more than 600 labels. We discover that the label descriptions can provide precise cues to classify a document which contains multiple semantic aspects.  Figure \ref{fig:label} provides an example on the necessity of integrating label cues in our task. Inspired by Wide \& Deep (WnD) framework \cite{Cheng2016}, we apply joint training of CNN and Regression models to enjoy the merits from both sides. CNN learns the semantic similarity between texts and classes. The Regression model integrates the retrieval features to take into consideration the prior knowledge on classes.  When adapting to a different domain, the Regression model can extend easily with domain-specific features.  


\subsection{Label Information}
Leveraging label information is not new and is mostly accomplished by embedding labels and texts in the same space to measure their correlations \cite{Yogatama2015,honglun:2018}. \citet{Ma2016} adds prototypes to the label representation. \citet{honglun:2018} proposes to transform classification to a matching target between texts and labels for multi-task learning. \citet{wang2018joint} further weights text features by the compatibilities between text and label embeddings via attention mechanism. Our model differs from previous studies in that we use the string-level similarity between label descriptions and input text instead of label embeddings. Based on the analysis of CPC classification system, we believe that string-level similarity can compensate for what semantic-level similarity cannot captures for patent classification tasks.

% \cite{Xiao2008} treats the task as a retrieval problem, and we jointly learn the retrieval features together with the document features.  

% Google Patent provides the largest commercial system which labels CPC to papers, but it does not offer the classification service for arbitrary patent submissions. 
% Our model follows the Wide \& Deep framework \cite{Cheng2016}, which is proposed for recommender systems. 

%We encode label side information as retrieval features, which emphasizes on the label cues in texts and avoids the possible over-generalization of the label embedding based models. 



% Only a few work on patent classification are based on NN framework. Long Short Term Memory (LSTM)\cite{Grawe2017} and CNN \cite{shaobo:2018} are most popular models. 


