@inproceedings{takeshita-etal-2025-randomly,
    title = "Randomly Removing 50{\%} of Dimensions in Text Embeddings has Minimal Impact on Retrieval and Classification Tasks",
    author = "Takeshita, Sotaro  and
      Takeshita, Yurina  and
      Ruffinelli, Daniel  and
      Ponzetto, Simone Paolo",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-emnlp/2025.emnlp-main.1410/",
    pages = "27693--27714",
    ISBN = "979-8-89176-332-6",
    abstract = "In this paper, we study the surprising impact that truncating text embeddings has on downstream performance. We consistently observe across 6 state-of-the-art text encoders and 26 downstream tasks, that randomly removing up to 50{\%} of embedding dimensions results in only a minor drop in performance, less than 10{\%}, in retrieval and classification tasks. Given the benefits of using smaller-sized embeddings, as well as the potential insights about text encoding, we study this phenomenon and find that, contrary to what is suggested in prior work, this is not the result of an ineffective use of representation space. Instead, we find that a large number of uniformly distributed dimensions actually cause an increase in performance when removed. This would explain why, on average, removing a large number of embedding dimensions results in a marginal drop in performance. We make similar observations when truncating the embeddings used by large language models to make next-token predictions on generative tasks, suggesting that this phenomenon is not isolated to classification or retrieval tasks."
}