In this section, we provide an overview of the problem description, our dataset of medical procedure names and we describe the architecture for performing mapping between clinic descriptions and a set of standardized procedure names.

\subsection{Problem Description}

\begin{figure*}[hbt!]
    % Draw.io: https://drive.google.com/file/d/11BKJUj00ysBIuxILRDrVS5__cvuNQKBD/view?usp=sharing
    \centering
    \includesvg[width=0.75\linewidth]{images/medmatch-retrieval.drawio.svg}
    \caption{Overall diagram of our method. We formulate medical procedure matching as a retrieval problem: entries in the masterlist are embedded and stored in a vector store and the most similar entry is retrieved based on the similarity with a procedure name from a clinic.}
    \label{fig:inference}
\end{figure*}

The problem of matching medical procedure names to a standardized masterlist is non-trivial. Simple text matching is insufficient, as we will demonstrate in Section \ref{results}. Our dataset is comprised of medical procedures and tests from 528 Romanian private clinics, containing 145,298 unique procedure names mapped to their corresponding masterlist entries. Through manual filtering of incorrect mappings, we reduced the dataset to 139,210 entries. Healthcare providers frequently use varying terms, abbreviations, and phrasing for the same procedure, which creates inconsistencies. To illustrate the difficulty, Table \ref{tab:examples} shows some relevant examples of mappings. Healthcare providers may omit obvious terms, such as "polyp resection" being synonymous with "polypectomy". Similarly, entries such as "frontal or lateral X-ray" must be mapped to "1 view X-ray", as they represent a single test being performed. Terminology variations, like vitamin B12 also being called Cyanocobalamin, can add complexity, especially when descriptions include irrelevant details that mislead text-based matching. Several other relevant examples are presented in Table \ref{tab:bm25-misses}: matching a single medical procedure, when the description actually describes two procedures, not recognizing the semantic meaning of descriptions, ignoring important numerical thresholds, retrieving specific procedures instead of general ones (or vice versa), and prioritizing less important terms.

We chose to model our problem as a retrieval problem, and not as a classification problem, since 50\% of elements from the masterlist have only 1 unique procedure assigned.  Figure \ref{fig:histogram} shows the distribution of clinic descriptions assigned to masterlist entries. If we frame our task as a classification problem, we have 39,097 distinct classes, with 19,493 containing only a single sample. Given the severe class imbalance per procedure, a classification model would be inappropriate and would generalize poorly. However, a retrieval-based method can be effectively used by leveraging semantic text embeddings and metric-learning approaches to capture the similarity between clinic descriptions and masterlist entries.

\begin{figure}
    \centering
    \includesvg[width=\linewidth]{images/histogram.svg}
    \caption{Distribution of number of unique clinic descriptions per masterlist procedure. There is a severe data imbalance: 19,493 (~50\%) out of 39,097 entries contain only a single example.}
    \label{fig:histogram}
\end{figure}

% Greu de explicat asta sa se inteleaga. Am lasat-o simplu ca am filtrat de mana intrarile gresite.
% {\color{orange} During the initial evaluation, we analyze cases where the recommendation model fails to match entries correctly. We find that the model often suggests plausible mappings, whereas the given ground-truth mappings appear to be incorrect. As a result, we fine-tune the embeddings model using the entire dataset and eliminate the misses@100, reducing the dataset to 139,210 entries.}

\subsection{Procedure Matching as Retrieval}
\label{sec:architecture}

In our retrieval setup, we used the provided masterlist procedure names to build a retrieval index \cite{wangMilvusPurposeBuiltVector2021} and clinic descriptions as the references for the queries. We embed the descriptions using dense \cite{masala-etal-2020-robert,wangMultilingualE5Text2024,alsentzer2019clincalbert} and sparse models \cite{bm25}. At inference time, we embed the query clinic descriptions that require a masterlist description and perform a similarity search. The vector DB returns the top-k most similar results for each of our clinic description. Figure \ref{fig:inference} showcases this approach.

The vector index includes two types of entries: masterlist entries and clinic description $\leftrightarrow$ masterlist pairs. In the first scenario, the similarity score is calculated between the query and the masterlist entries, with the index returning the most similar masterlist entries. In the second scenario, the similarity score is computed between the query and the clinic descriptions stored in the index, and the masterlist entry associated with the most similar clinic description is returned. We build our search and evaluation architecture over Milvus \cite{wangMilvusPurposeBuiltVector2021}, a high-performance vector database.

For our setup, we used three types of text embeddings: \textit{(i)} sparse text embeddings using BM25 \cite{bm25}, \textit{(ii)} dense semantic embeddings with several pretrained transformer models \cite{masala-etal-2020-robert,wangMultilingualE5Text2024,alsentzer2019clincalbert}, both zero-shot and fine-tuned with metric learning, and \textit{(iii)} a hybrid ranking approach using RRF \cite{cormackReciprocalRankFusion2009}. 

\subsection{Sparse Embeddings with BM25}
\label{sec:sparse-embeddings}

For computing the sparse embeddings, we use BM25 \cite{bm25} to identify the most relevant word-level features from the training set descriptions and masterlist entries. Text is preprocessed by removing diacritics, punctuation, and Romanian stopwords, followed by stemming the remaining words. At inference time, we compute the inner product between the query descriptions and the masterlist descriptions, as well as the clinic description pairs.

\begin{figure*}[hbt!]
    % Draw.io: https://drive.google.com/file/d/1nxlDirFDHTUeuvX48gcXN8br562So7qc/view?usp=sharing
    \centering
    \includesvg[width=0.75\linewidth]{images/medmatch-metric-learning.drawio.svg}
    \caption{Fine-tuning approach for dense sentence embeddings. A pretrained text embedding model is trained to minimize the distance between representations of masterlist entries and associated clinic procedure names while maximising the distance between every other entry.}
    \label{fig:metric-learning}
\end{figure*}

\subsection{Dense Embeddings with Pretrained Transformer Models}
\label{sec:dense-embeddings-fine-tuning}

Recent studies have highlighted the challenges of selecting optimal sentence embedding models for domain-specific retrieval tasks \cite{wornow2023shaky}. Generic benchmarks do not always align with real-world performance, necessitating task-specific evaluations. The MTEB Leaderboard \citep{muennighoffMTEBMassiveText2023} ranks top-performing embedding models based on retrieval performance across various datasets.

We experiment with three models for dense embeddings: mE5-large \citep{wangMultilingualE5Text2024}, RoBERT-large \citep{masala-etal-2020-robert}, and BioClinicalBERT \citep{alsentzer2019clincalbert}. We select mE5 due to its strong performance on multilingual retrieval tasks, RoBERT as a strong language-specific baseline model pre-trained using only Romanian text, and BioBERT as a domain-specific model, pretrained on biomedical text, which may capture medical terminology better than general-purpose models.

\noindent \textbf{Fine-tuning with Metric Learning.} We fine-tune the pretrained text embedding models using the MultipleNegativesRankingLoss objective \cite{henderson2017efficient}, as shown in Figure \ref{fig:metric-learning}. We consider the clinic descriptions as anchors $(a_i)$ and the corresponding masterlist descriptions $(p_i)$ as positive pairs - $(a_i, p_i)$. The negative pair consists of every combination $(a_i, p_j)$, where $p_j, j \ne i$ are all other masterlist descriptions. In this way, our embedding model learns to increase the cosine similarity between the clinic descriptions and their mapped masterlist description, while decreasing the similarity between the clinic description and all other masterlist items. The model is fine-tuned on 80,911 pairs for 20 epochs, using a batch size of 4096. We use a learning rate of 2e-5, with a cosine scheduler and a warmup ratio of 0.1. All experiments are run on an NVIDIA A100 80GB GPU.

\subsection{Hybrid search}
\citet{cormackReciprocalRankFusion2009} proposed Reciprocal Rank Fusion (RRF) as a method of aggregating the ranking results of multiple information retrieval systems. It is calculated using the formula:

\begin{equation}
    \text{RRFscore}(d \in D) = \sum_{r \in R} \frac{1}{k + r(d)}
\end{equation}

where $D$ is the set of results to be ranked, $R$ represents the multiple returned rankings of these results, $k$ is a constant, and $r(d)$ is the rank of a result $d$. We combine the results of dense and sparse embeddings using RRF and analyze its effect on retrieval accuracy.