\noindent \textbf{Sentence embedding models.} 
Semantic text embedding models \cite{ramesh-kashyap-etal-2024-comprehensive} are a significant component of many NLP applications, most notably text retrieval and question answering. Text embeddings are used to capture semantic representations of text that go beyond surface level word and character matching methods such as TF-IDF. Currently, practitioners are using pretrained transformer models such as BERT \cite{reimersSentenceBERTSentenceEmbeddings2019}, either by aggregating word-level representations with a pooling operation, or by using specialized training for text similarity \cite{khattabColBERTEfficientEffective2020}. Currently, the best performing models are aggregated in the MTEB leaderboard \cite{muennighoff-etal-2023-mteb}, a benchmark of several text embedding tasks, including several non-English datasets. For the medical and scientific domain \cite{lewis2020pretrained}, several models have been developed. Models such as SciBERT \cite{beltagy-etal-2019-scibert}, BioBERT \cite{alsentzer2019clincalbert}, ClinicalBERT \cite{alsentzer2019clincalbert} and MedBERT \cite{rasmy2021med} offer domain-specific embeddings by training on either specialized biomedical corpora or task-specific datasets.

However, most contextualized text representation models for the medical domain are focused on the English language, with under-represented languages severely lacking in resources such as specialized models or training datasets. In our setup, medical procedure names are written in Romanian, a low resource language, with only a few pretrained language models \cite{masala2024vorbecstiromanecsterecipetrain,masala-etal-2020-robert}. Currently, for Romanian, only a pretrained RoBERT model \cite{masala-etal-2020-robert} is available for direct contextualized text representations, but no such variant exists for the medical domain. Currently, multilingual models such as E5 \cite{wangMultilingualE5Text2024} and MiniLM \cite{wang2020minilm} are ubiquitously used for non-English tasks.

\noindent \textbf{Medical Procedure Matching.} 
The task of medical procedure matching has been performed in the context of assigning medical notes or pathology reports to a predefined set of medical procedures \cite{tavabiSystematicEvaluationCommon2024,levyComparisonMachineLearningAlgorithms2022,zaidatCanNovelNatural2024}, with a focus on the US medical system.

\input{tables/examples}

\citet{tavabiSystematicEvaluationCommon2024} investigated the problem of mapping unstructured operative notes to Current Procedural Terminology (CPT) codes. The CPT code set is a system used to describe medical, surgical and diagnostic services, that are used for billing and insurance reimbursement processes in healthcare. The authors apply common NLP techniques to assign 44,002 notes to 100 most prevalent CPT codes, treating this problem as a classification task. Using TF-IDF, Doc2Vec \cite{leDistributedRepresentationsSentences2014a} and Clinical Bio-BERT \cite{alsentzer2019clincalbert} embeddings as input they train a support vector machine classifier, for each embedding type. In their experiments, TF-IDF outperformed both BERT and Doc2Vec.


\citet{levyComparisonMachineLearningAlgorithms2022} used machine-learning models for predicting CPT codes from pathology reports. Their study analyzed 93,039 pathology reports from the Dartmouth-Hitchcock Department of Pathology and Laboratory Medicine, classifying 42 CPT codes. They evaluated the performance of XGBoost and BERT—using both diagnostic text alone and all report subfields. Their findings indicated that while BERT outperformed XGBoost when trained only on diagnostic text, but using all report subfields resulted in XGBoost achieving the best performance. 

\citet{zaidatCanNovelNatural2024} have also explored assigning CPT codes to spine surgery operative notes, using XLNet \cite{yang2019xlnet}, a bidirectional LSTM \cite{hochreiterLongShortTermMemory1997} model. They fine-tune the model to their operative note dataset, containing 922 entries. 


Previous studies have evaluated the performance of statistical, machine learning and deep learning models on classification of a large number of samples to a relatively small subset of CPT codes. In contrast, we formulate our problem as a retrieval problem, since our dataset is severely imbalanced, and contains two orders of magnitude more CPT codes (38,814 entries). Furthermore, another advantage of this formulation is that by avoiding a fixed set of classes, the addition of more procedures does not require modifying the architecture or retraining the model. Unique to our work, we are the first to tackle this problem in Romanian, a severely low-resource language in terms of specialized models for the medical domain.