Understanding Differential Search Index for Text Retrieval

Xiaoyang Chen, Yanjiang Liu, Ben He, Le Sun, Yingfei Sun

Abstract
The Differentiable Search Index (DSI) is a novel information retrieval (IR) framework that utilizes a differentiable function to generate a sorted list of document identifiers in response to a given query. However, due to the black-box nature of the end-to-end neural architecture, it remains to be understood to what extent DSI possesses the basic indexing and retrieval abilities. To mitigate this gap, in this study, we define and examine three important abilities that a functioning IR framework should possess, namely, exclusivity, completeness, and relevance ordering. Our analytical experimentation shows that while DSI demonstrates proficiency in memorizing the unidirectional mapping from pseudo queries to document identifiers, it falls short in distinguishing relevant documents from random ones, thereby negatively impacting its retrieval effectiveness. To address this issue, we propose a multi-task distillation approach to enhance the retrieval quality without altering the structure of the model and successfully endow it with improved indexing abilities. Through experiments conducted on various datasets, we demonstrate that our proposed method outperforms previous DSI baselinesThe code and data for this work can be found at https://github.com/VerdureChen/Understang_DSI.
Anthology ID:
2023.findings-acl.681
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10701–10717
Language:
URL:
https://aclanthology.org/2023.findings-acl.681
DOI:
10.18653/v1/2023.findings-acl.681
Bibkey:
Cite (ACL):
Xiaoyang Chen, Yanjiang Liu, Ben He, Le Sun, and Yingfei Sun. 2023. Understanding Differential Search Index for Text Retrieval. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10701–10717, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Understanding Differential Search Index for Text Retrieval (Chen et al., Findings 2023)
Copy Citation:
PDF:
https://preview.aclanthology.org/landing_page/2023.findings-acl.681.pdf
PDF Search