@inproceedings{lippmann-yang-2025-zero,
    title = "Zero-Shot Contextual Embeddings via Offline Synthetic Corpus Generation",
    author = "Lippmann, Philip  and
      Yang, Jie",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/author-page-yu-wang-polytechnic/2025.findings-emnlp.111/",
    doi = "10.18653/v1/2025.findings-emnlp.111",
    pages = "2089--2104",
    ISBN = "979-8-89176-335-7",
    abstract = "Context-aware embedding methods boost retrieval accuracy by conditioning on corpus statistics (e.g., term co-occurrence and topical patterns) extracted from neighboring documents. However, this context-aware approach requires access to the target corpus or requires domain-specific finetuning, posing practical barriers in privacy-sensitive or resource-constrained settings. We present ZEST, a zero-shot contextual adaptation framework that replaces real corpus access with a one-time offline synthesis of a compact proxy. Given only a handful of exemplar documents representative of the general target domain, we use a multi-step hierarchical procedure to generate a synthetic context corpus of several hundred documents that aims to emulate key domain-specific distributions. At inference, the frozen context-aware encoder uses this proxy corpus {--} without any finetuning or target corpus access {--} to produce domain-adapted embeddings. Across the MTEB benchmark, ZEST{'}s zero-shot synthetic context adaptation using only five example documents performs within 0.5{\%} of models leveraging full target corpus access {--} demonstrating remarkable efficacy without any retraining. ZEST thus provides a practical method for deploying high-performance, adaptable embeddings in constrained environments."
}