@inproceedings{feng-etal-2025-dont,
    title = "Don{'}t Reinvent the Wheel: Efficient Instruction-Following Text Embedding based on Guided Space Transformation",
    author = "Feng, Yingchaojie  and
      Sun, Yiqun  and
      Sun, Yandong  and
      Zhu, Minfeng  and
      Huang, Qiang  and
      Tung, Anthony Kum Hoe  and
      Chen, Wei",
    editor = "Che, Wanxiang  and
      Nabende, Joyce  and
      Shutova, Ekaterina  and
      Pilehvar, Mohammad Taher",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingestion-acl-25/2025.acl-long.1196/",
    pages = "24511--24525",
    ISBN = "979-8-89176-251-0",
    abstract = "In this work, we investigate an important task named instruction-following text embedding, which generates dynamic text embeddings that adapt to user instructions, highlighting specific attributes of text. Despite recent advancements, existing approaches suffer from significant computational overhead, as they require re-encoding the entire corpus for each new instruction. To address this challenge, we propose GSTransform, a novel instruction-following text embedding framework based on Guided Space Transformation. Our key observation is that instruction-relevant information is inherently encoded in generic embeddings but remains underutilized. Instead of repeatedly encoding the corpus for each instruction, GSTransform is a lightweight transformation mechanism that adapts pre-computed embeddings in real time to align with user instructions, guided by a small amount of text data with instruction-focused label annotation. We conduct extensive experiments on three instruction-awareness downstream tasks across nine real-world datasets, demonstrating that GSTransform improves instruction-following text embedding quality over state-of-the-art methods while achieving dramatic speedups of 6{\textasciitilde}300$\times$ in real-time processing on large-scale datasets. The source code is available at https://github.com/YingchaojieFeng/GSTransform."
}