@inproceedings{jorgensen-breitung-2025-margins,
    title = "Margins in Contrastive Learning: {Evaluating} Multi-task Retrieval for Sentence Embeddings",
    author = "J{\o}rgensen, Tollef Emil  and
      Breitung, Jens",
    editor = "Johansson, Richard  and
      Stymne, Sara",
    booktitle = "Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025)",
    month = mar,
    year = "2025",
    address = "Tallinn, Estonia",
    publisher = "University of Tartu Library",
    url = "https://preview.aclanthology.org/Ingest-2025-COMPUTEL/2025.nodalida-1.28/",
    pages = "269--278",
    ISBN = "978-9908-53-109-0",
    abstract = "This paper explores retrieval with sentence embeddings by fine-tuning sentence-transformer models for classification while preserving their ability to capture semantic similarity. To evaluate this balance, we introduce two opposing metrics {--} polarity score and semantic similarity score {--} that measure the model`s capacity to separate classes and retain semantic relationships between sentences. We propose a system that augments supervised datasets with contrastive pairs and triplets, training models under various configurations and evaluating their performance on top-$k$ sentence retrieval. Experiments on two binary classification tasks demonstrate that reducing the margin parameter of loss functions greatly mitigates the trade-off between the metrics. These findings suggest that a single fine-tuned model can effectively handle joint classification and retrieval tasks, particularly in low-resource settings, without relying on multiple specialized models."
}