@inproceedings{zhou-etal-2025-rank,
    title = "Rank-Awareness and Angular Constraints: A New Perspective on Learning Sentence Embeddings from {NLI} Data",
    author = "Zhou, Zicheng  and
      Huang, Min  and
      Miao, Qinghai",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1129/",
    pages = "22206--22220",
    ISBN = "979-8-89176-332-6",
    abstract = "Learning high-quality sentence embeddings from Natural Language Inference (NLI) data is often challenged by a critical signal conflict between discrete labels and the continuous spectrum of semantic similarity, as well as information loss from discarded neutral sentence pairs during training. To address this, we introduce Rank-Awareness and Angular Optimization Embeddings (RAOE), a framework that leverages the full NLI dataset (Entailment, Neutral, Contradiction) augmented with pre-computed continuous similarity scores ($S$). RAOE employs a novel composite objective which features: (1) a Rank Margin objective that enforces rank consistency against $S$ using an explicit margin, and (2) a Gated Angular objective that conditionally refines embedding geometry based on NLI label ($L$) and $S$ score agreement. Extensive evaluations on STS tasks and the MTEB benchmark demonstrate RAOE{'}s effectiveness. Our general-purpose RAOE-S1 model (BERT-base) significantly outperforms strong baselines, achieving an average Spearman{'}s correlation of 85.11 (vs. SimCSE{'}s 81.57 and AnglE{'}s 82.43), and shows consistent improvements on MTEB. Further STS-specialized fine-tuning (RAOE-S2) establishes new state-of-the-art performance on STS (88.17 with BERT-base). These results confirm RAOE{'}s ability to efficiently learn robust and nuanced sentence representations through the synergy of rank-awareness and conditional angular constraints. Code is available at https://github.com/Shengjingwa/RAOE."
}