Relevance Scores Calibration for Ranked List Truncation via TMP Adapter

Pavel Posokhov, Sergei Masliukhin, Skrylnikov Stepan, Danil Tirskikh, Olesia Makhnytkina


Abstract
The ranked list truncation task involves determining a truncation point to retrieve the relevant items from a ranked list. Despite current advancements, truncation methods struggle with limited capacity, unstable training and inconsistency of selected threshold. To address these problems we introduce TMP Adapter, a novel approach that builds upon the improved adapter model and incorporates the Threshold Margin Penalty (TMP) as an additive loss function to calibrate ranking model relevance scores for ranked list truncation. We evaluate TMP Adapter’s performance on various retrieval datasets and observe that TMP Adapter is a promising advancement in the calibration methods, which offers both theoretical and practical benefits for ranked list truncation.
Anthology ID:
2025.findings-acl.402
Volume:
Findings of the Association for Computational Linguistics: ACL 2025
Month:
July
Year:
2025
Address:
Vienna, Austria
Editors:
Wanxiang Che, Joyce Nabende, Ekaterina Shutova, Mohammad Taher Pilehvar
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7728–7734
Language:
URL:
https://preview.aclanthology.org/mtsummit-25-ingestion/2025.findings-acl.402/
DOI:
10.18653/v1/2025.findings-acl.402
Bibkey:
Cite (ACL):
Pavel Posokhov, Sergei Masliukhin, Skrylnikov Stepan, Danil Tirskikh, and Olesia Makhnytkina. 2025. Relevance Scores Calibration for Ranked List Truncation via TMP Adapter. In Findings of the Association for Computational Linguistics: ACL 2025, pages 7728–7734, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):
Relevance Scores Calibration for Ranked List Truncation via TMP Adapter (Posokhov et al., Findings 2025)
Copy Citation:
PDF:
https://preview.aclanthology.org/mtsummit-25-ingestion/2025.findings-acl.402.pdf