A Counterfactual Explanation Framework for Retrieval Models

Bhavik Chandna, Procheta Sen


Abstract
Explainability has become a crucial concern in today’s world, aiming to enhance transparency in machine learning and deep learning models. Information retrieval is no exception to this trend. In existing literature on explainability of information retrieval, the emphasis has predominantly been on illustrating the concept of relevance concerning a retrieval model. The questions addressed include why a document is relevant to a query, why one document exhibits higher relevance than another, or why a specific set of documents is deemed relevant for a query. However, limited attention has been given to understanding why a particular document is not favored (e.g., not within top-K) with respect to a query and a retrieval model. In an effort to address this gap, our work focuses on the question of what terms need to be added within a document to improve its ranking. This, in turn, answers the question of which words in the document played a role in not being favored by a retrieval model for a particular query. We use a counterfactual framework to solve the above-mentioned research problem. Our experiments show the effectiveness of our proposed approach in predicting counterfactuals for both statistical (e.g. BM25) and deep-learning-based models (e.g. DRMM, DSSM, ColBERT, MonoT5).
Anthology ID:
2026.findings-acl.1917
Volume:
Findings of the Association for Computational Linguistics: ACL 2026
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
38477–38490
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1917/
DOI:
Bibkey:
Cite (ACL):
Bhavik Chandna and Procheta Sen. 2026. A Counterfactual Explanation Framework for Retrieval Models. In Findings of the Association for Computational Linguistics: ACL 2026, pages 38477–38490, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
A Counterfactual Explanation Framework for Retrieval Models (Chandna & Sen, Findings 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.findings-acl.1917.pdf
Checklist:
 2026.findings-acl.1917.checklist.pdf