Rajeev Ranjan
2026
One Word Is Not Enough: Simple Prompts Improve Word Embeddings
Rajeev Ranjan
Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)
Rajeev Ranjan
Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)
Text embedding models are designed for sentence-level applications like retrieval and semantic similarity, and are primarily evaluated on sentence-level benchmarks. Their behavior on isolated words is less understood. We show that simply prepending semantic prompts to words before embedding substantially improves word similarity correlations. Testing 7 text embedding models, including text-embedding-3-large (OpenAI), embed-english-v3.0 (Cohere), voyage-3 (Voyage AI), all-mpnet-base-v2, and Qwen3-Embedding-8B, on 3 standard benchmarks (SimLex-999, WordSim-353, MEN-3000), we find that prompts like "meaning: word" or "Represent the semantic concept: word" improve Spearman correlations by up to +0.28 on SimLex-999. Some models fail completely on bare words (ρ ≈ 0) but recover with prompts (+0.73 improvement). Our best results achieve ρ=0.692 on SimLex-999 with embed-english-v3.0 (Cohere), ρ=0.811 on WordSim-353, and ρ=0.855 on MEN-3000 with text-embedding-3-large (OpenAI). These results outperform classic static embeddings like Word2Vec (ρ=0.40) and even the best static method LexVec (ρ=0.48) on SimLex-999, establishing a new state-of-the-art for pure embedding methods. This zero-shot technique requires no training and works with any text embedding model.