Abstract
Embeddings extracted by pre-trained Large Language Models (LLMs) have significant potential to improve information retrieval and search. Beyond the zero-shot setup in which they are being conventionally used, being able to take advantage of the information from the relevant query-corpus paired data can further boost the LLM capabilities. In this paper, we propose a novel method, Search-Adaptor, for customizing LLMs for information retrieval in an efficient and robust way. Search-Adaptor modifies the embeddings generated by pre-trained LLMs, and can be integrated with any LLM, including those only available via prediction APIs. On multiple English, multilingual, and multimodal retrieval datasets, we show consistent and significant performance benefits for Search-Adaptor – e.g., more than 5% improvements for Google Embedding APIs in nDCG@10 averaged over 14 BEIR datasets.- Anthology ID:
- 2024.acl-long.661
- Volume:
- Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Month:
- August
- Year:
- 2024
- Address:
- Bangkok, Thailand
- Editors:
- Lun-Wei Ku, Andre Martins, Vivek Srikumar
- Venue:
- ACL
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 12230–12247
- Language:
- URL:
- https://aclanthology.org/2024.acl-long.661
- DOI:
- 10.18653/v1/2024.acl-long.661
- Cite (ACL):
- Jinsung Yoon, Yanfei Chen, Sercan Arik, and Tomas Pfister. 2024. Search-Adaptor: Embedding Customization for Information Retrieval. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12230–12247, Bangkok, Thailand. Association for Computational Linguistics.
- Cite (Informal):
- Search-Adaptor: Embedding Customization for Information Retrieval (Yoon et al., ACL 2024)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-5/2024.acl-long.661.pdf