BlitzRank: Principled Zero-Shot Ranking Agents with Tournament Graphs

Paper

Code

⚡ BlitzRank uses tournament graphs to extract maximal information from each LLM call—a principled framework achieving Pareto optimality across 14 benchmarks and 5 LLMs with 25–40% fewer queries.

Accurate & Efficient #

Algorithm Visualization #

Abstract #

        Large language models have emerged as powerful zero-shot rerankers for retrieval-augmented generation,
        offering strong generalization without task-specific training. However, existing LLM reranking methods
        either rely on heuristics that fail to fully exploit the information revealed by each ranking decision
        or are inefficient when they do. We introduce a tournament graph framework that provides a principled
        foundation for k-wise reranking. Our key observation is that each k-document comparison reveals a
        complete tournament of *k choose 2* pairwise preferences. These tournaments are aggregated into
        a global preference graph, whose transitive closure yields many additional orderings without further
        model invocations. We formalize when a candidate's rank is certifiably determined and design a query
        schedule that greedily maximizes information gain towards identifying the top-m items. Our framework
        also gracefully handles non-transitive preferences—cycles induced by LLM judgments—by
        collapsing them into equivalence classes that yield principled tiered rankings. Empirically, across
        14 benchmarks and 5 LLMs, our approach achieves Pareto dominance over existing methods: matching or
        exceeding accuracy while requiring 25–40% fewer tokens than comparable approaches, and
        7× fewer than pairwise methods at near-identical quality.

Quick Start #

uv pip install blitzrank
python
from blitzrank import BlitzRank

ranker = BlitzRank(model="openai/gpt-4.1")

query = "capital of France"
docs = ["Berlin is in Germany", "Paris is in France", "Tokyo is in Japan"]

indices = ranker(query, docs, topk=2)  # -> [1, 0]
top_docs = [docs[i] for i in indices]

Citation #

@article{blitzrank2026,
  title={BlitzRank: Principled Zero-shot Ranking Agents with Tournament Graphs},
  author={Agrawal, Sheshansh and Nguyen, Thien Hang and Kiela, Douwe},
  journal={arXiv preprint arXiv:2602.05448},
  year={2026}
}