Efficient k-NN Search with Cross-Encoders using Adaptive Multi-Round CUR Decomposition
Nishant Yadav, Nicholas Monath, Manzil Zaheer, Andrew McCallum
Abstract
Cross-encoder models, which jointly encode and score a query-item pair, are prohibitively expensive for direct k-nearest neighbor (k-NN) search. Consequently, k-NN search typically employs a fast approximate retrieval (e.g. using BM25 or dual-encoder vectors), followed by reranking with a cross-encoder; however, the retrieval approximation often has detrimental recall regret. This problem is tackled by ANNCUR (Yadav et al., 2022), a recent work that employs a cross-encoder only, making search efficient using a relatively small number of anchor items, and a CUR matrix factorization. While ANNCUR’s one-time selection of anchors tends to approximate the cross-encoder distances on average, doing so forfeits the capacity to accurately estimate distances to items near the query, leading to regret in the crucial end-task: recall of top-k items. In this paper, we propose ADACUR, a method that adaptively, iteratively, and efficiently minimizes the approximation error for the practically important top-k neighbors. It does so by iteratively performing k-NN search using the anchors available so far, then adding these retrieved nearest neighbors to the anchor set for the next round. Empirically, on multiple datasets, in comparison to previous traditional and state-of-the-art methods such as ANNCUR and dual-encoder-based retrieve-and-rerank, our proposed approach ADACUR consistently reduces recall error—by up to 70% on the important k = 1 setting—while using no more compute than its competitors.- Anthology ID:
- 2023.findings-emnlp.544
- Volume:
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- Findings
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 8088–8103
- Language:
- URL:
- https://aclanthology.org/2023.findings-emnlp.544
- DOI:
- 10.18653/v1/2023.findings-emnlp.544
- Cite (ACL):
- Nishant Yadav, Nicholas Monath, Manzil Zaheer, and Andrew McCallum. 2023. Efficient k-NN Search with Cross-Encoders using Adaptive Multi-Round CUR Decomposition. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8088–8103, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Efficient k-NN Search with Cross-Encoders using Adaptive Multi-Round CUR Decomposition (Yadav et al., Findings 2023)
- PDF:
- https://preview.aclanthology.org/emnlp22-frontmatter/2023.findings-emnlp.544.pdf