Rerankers are the quiet layer of every RAG stack. The retriever fetches candidates; the reranker decides which ones actually deserved it. Cohere just shipped Rerank 5 — and did something the reranker market never had before: it made the quality-vs-latency tradeoff an explicit menu with two tiers.
Every number below is Cohere-reported or vendor-sourced, not measured by us. Read it as the spec sheet, with the caveats where they matter.
Two models, one API:
Both support 32K context, 100+ languages, JSON/YAML output, and a drop-in upgrade path: change the model parameter and you're on v5. No open weights — this is API-only, same as previous Cohere rerankers.
Two genuinely new features ship with the family:
Quality (Cohere's own evals — independent replication pending):
Latency (the tier decision):
That's roughly a 3x latency gap between tiers. In a RAG pipeline, the rerank step usually runs last, over a candidate set like that 96-document default. Add it to your retriever latency, your embedding latency, and your generation latency, and the tier choice shows up directly in your p99.
Pick Pro when the query is precision-critical and the request rate is low: research assistants, legal/medical retrieval, anything where a bad top-3 result costs real money. Eleven seconds of rerank is invisible inside a 30-second agentic workflow.
Pick Fast when the user is waiting: chatbots, search bars, anything with a p99 budget under ~10 seconds end to end. If your total pipeline is 6 seconds and the reranker eats 11.7 of them, you don't have a latency problem — you have a tier problem.
The shape of the code is the standard rerank call. Upgrade path per Cohere: change the model parameter.
import cohere
co = cohere.Client("YOUR_API_KEY")
resp = co.rerank(
model="rerank-v5-pro", # or "rerank-v5-fast" for the 4.1s tier
query="what is the latency difference between the two tiers?",
documents=["...", "..."], # your retriever's candidate set
top_n=3,
return_documents=True,
)
for r in resp.results:
print(r.relevance_score, r.document.text[:120])
If you're experimenting, benchmark both tiers on your queries before committing: the +5.8 BRIGHT delta is a composite average, and averages are where real workloads go to be lied to. Run 50 representative queries, measure top-1 and top-3 accuracy per tier, and price the latency against your p99.
The pattern is the story: retrieval quality stopped being one-size-fits-all. The reranker now comes with a menu, and the right answer depends on whose latency budget you're spending.
Decoding AI — AI news + tools, tested with hard numbers. What shipped, what it costs, what actually works: https://decodingaitools.substack.com
This article was produced by an AI agent (Decoding AI's autopilot) from published reports. Benchmarks are vendor-reported unless labeled otherwise; no lab runs were performed for this piece.