BracketRank: Large Language Model Document Ranking via Reasoning-based Competitive Elimination

Abdelrahman Abdallah, Mohammed Ali, Bhawna Piryani, Adam Jatowt


Abstract
Although Large Language Models (LLMs) show strong potential for zero-shot document ranking, current listwise approaches face three critical limitations. First, context length constraints prevent processing many documents simultaneously. Second, sequential generation creates bottlenecks that cannot run in parallel. Third, ranking results depend heavily on initial document order, leading to inconsistent performance. We introduce BracketRank, a reasoning-driven competitive elimination framework that addresses these challenges through systematic group competition. Our method uses adaptive grouping to automatically optimise group sizes based on LLM context limits, reasoning-enhanced prompts that require explicit relevance explanations, and bracket-style elimination where documents compete through winner and loser brackets. This structure ensures every document has fair advancement opportunities regardless of initial positioning while allowing for parallel processing across competition stages. We evaluate BracketRank on TREC DL 19, TREC DL 20, and eight BEIR benchmark datasets. Results show that BracketRank achieves 77.90 NDCG@5 on TREC DL 19 and 75.85 NDCG@5 on TREC DL 20, outperforming RankGPT and other state-of-the-art methods. On BEIR datasets, BracketRank reaches 54.66 average NDCG@10, demonstrating robust performance across diverse domains.
Anthology ID:
2026.acl-long.153
Volume:
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
July
Year:
2026
Address:
San Diego, California, United States
Editors:
Maria Liakata, Viviane P. Moreira, Jiajun Zhang, David Jurgens
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3381–3397
Language:
URL:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.153/
DOI:
Bibkey:
Cite (ACL):
Abdelrahman Abdallah, Mohammed Ali, Bhawna Piryani, and Adam Jatowt. 2026. BracketRank: Large Language Model Document Ranking via Reasoning-based Competitive Elimination. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3381–3397, San Diego, California, United States. Association for Computational Linguistics.
Cite (Informal):
BracketRank: Large Language Model Document Ranking via Reasoning-based Competitive Elimination (Abdallah et al., ACL 2026)
Copy Citation:
PDF:
https://preview.aclanthology.org/ingest-acl/2026.acl-long.153.pdf
Checklist:
 2026.acl-long.153.checklist.pdf