A few years ago I did a vector database evaluation, right when embeddings were getting relevant and semantic search was the leading topic in the industry. Back then I compared Weaviate, Pinecone, Qdrant and Milvus. All of them worked, but Qdrant was technologically ahead. It was very fast and it had far better filtering than the others.
A few months ago I did another evaluation, this time mainly focused on if it is worth to replace Elasticsearch in a recommender system. I remembered Elastic as absolutely unusable for vector search in 2023, mostly because of the limit on vector size and bad performance.
Surprisingly, the result was that the gap is mostly gone. There are still differences between Qdrant and Elastic, and some of them matter a lot depending on your workload. But Elastic has put a lot of work into vector search and is now on par with Qdrant on performance. This article shows the differences I found and hopefully helps others pick the right vector database for their specific use case.
There are two kinds of benchmarks you find when you search for this topic, and neither of them answered my question.
Vendor benchmarks: Every vendor benchmark makes the vendor win. Qdrant publishes one where Qdrant wins. Elastic publishes one where Elastic wins. Pinecone and Turbopuffer publish numbers where their offering looks best. Same for Weaviate, Milvus and everybody else. The dataset, the hardware, the client configuration and the competitor’s settings are all chosen by the party that wants to win. I do not think these are dishonest. I think they are useless for making a decision for your case.
Pure Performance benchmarks like ann-benchmarks. These are fine and I respect the work. But they are narrow. They measure recall against queries per second on a single machine, on a static dataset, with no writes happening, no filters, no concurrent traffic and no cost attached. That is a scientific question: how good is this index algorithm. My question was an engineering one: what happens when I put 1.2 million vectors from production into this thing, replay a week of real traffic against it, reindex at the same time, filter by tenant, and pay the bill at the end of the month. Nobody publishes that, so I had to measure it and you probably too.
The workload is 1.2 million items, each with two 768-dimensional CLIP embeddings, one for the image and one for the text. Recall is recall@10 against exact brute-force ground truth. Traffic scenarios come from seven days of production telemetry: a read-heavy scenario at 330 qps, a mixed scenario with 100 online plus 370 batch qps, a peak at 470 qps and a cluster peak at 750 qps. Every scenario runs for five minutes with a re-index stream writing concurrently.
The providers:
Read that list again, because the four were not evaluated on equal terms. Elastic and Qdrant ran on dedicated clusters, and Qdrant was evaluated with presales and support engineers from the vendor available. Pinecone ran on the starter credits, Turbopuffer on a paid serverless plan, and neither had any vendor contact. More importantly, neither of the two was connected to the benchmark cluster through AWS PrivateLink or VPC peering. Their traffic went over the public internet, while Elastic and Qdrant sat in the same region, one hop away from the benchmark client. Keep this in mind for every table below. The Pinecone and Turbopuffer numbers are what you get by signing up, reading the docs and pointing a client at the public endpoint. They are a floor for those products, not a ceiling. I kept them in the post because this is exactly how a lot of teams evaluate vector databases, and the results show why that is a bad idea.
Same index configuration everywhere: HNSW with m=16 and ef_construction=100, binary quantization on. Same write batch size of 256 for every provider. Each client on the fastest path the vendor recommends: gRPC where offered, base64-packed vectors, or json. The setup ran inside the same Kubernetes cluster in the same region as every endpoint, so client round trips are not in the numbers.
Recall and latency on a freshly built index, quantization on:
Elastic is faster and slightly more accurate on a fresh index. It needs 400 candidates to hit 0.95 on text vectors, Qdrant needs 50. Turbopuffer has the best recall of the four, at a latency that is fine for many workloads and not for this one.
The candidate count matters because of quantization. Binary quantization is what makes both of them fast, and it costs recall. From the first campaign, at a fixed 100 candidates, quantization on and off side by side:
At 100 candidates neither reaches 0.95 on text vectors with quantization on. Qdrant loses less recall to quantization, Elastic loses less latency when you switch it off. That is why the final numbers above use a candidate sweep instead of a fixed count.
Ingest, batch size 256 everywhere:
Elastic ingests fastest. Qdrant has the gentler upsert tail and uses less disk. A full rebuild of 1.2M items takes under five minutes on both. Pinecone and Turbopuffer want bigger batches than 256, so their rows are handicapped on top of the network handicap.
Production scenarios with a concurrent re-index, search p99 in ms:
Elastic and Qdrant pass everything up to 750 qps. Pinecone’s free tier throttled to around 135 qps with multi-second tails. Turbopuffer missed 200 ms in every scenario. Neither of those two rows tells you much about the engines. They tell you what a free serverless tier does under sustained production load, which is throttle.
So I decided not to contact the presales teams of Pinecone and Turbopuffer to set up a proper benchmark with dedicated capacity and private networking, but to spend the remaining time on Elastic and Qdrant. Both passed every production scenario on the first serious run. The other two would have needed a sales process, a dedicated tier and a PrivateLink setup before they were even at the same starting line. That is a real gap in this evaluation and I say so again at the end. It is also how most evaluations go in practice: the providers that work out of the box get the deep look, the others get a footnote.
Freshness: The time from a write to that item showing up in search results, while the cluster is serving reads. Qdrant sat at p50 36 to 62 ms and p99 around 130 ms across all scenarios. Elastic sat at p50 0.4 to 0.9 seconds and p99 between 1 and 1.7 seconds. That is the refresh interval, segments, and the whole Lucene write model. You can tune it down, but you are tuning against the architecture.
Tail latency under re-index: In the saturation run Elastic’s p99 climbed to 297 ms during the re-index phase. Qdrant stayed at 167 ms. In the paced scenarios both hold the SLO, so this only matters when you are near the ceiling.
Filtered recall: This one surprised me, because filtering was Qdrant’s strength years ago and it still is. Both providers returned zero cross-tenant leaks over 2,000 checks and passed every filter correctness check. But filtered recall@10 was 0.956 on Qdrant and 0.52 on Elastic. Elastic’s filtered kNN returns correct results, it just loses recall at k=10 when the filter is selective.
→ Lower latency at low and moderate load. Better upsert tails. Less disk. A smaller cluster.
The moderate-load latency is worth a chart, because it is consistent from p50 to p99 and not a tail effect. From the first campaign, 125 and 160 qps with no writes:
Raw query latency on a fresh index: 8 ms p50 and 12 ms p99 against Qdrant’s 10 and 20. Small margin, but consistent.
Ingest rate: About 13% more ready docs per second on a smaller cluster.
Pooled query vectors: This was the biggest surprise of the whole exercise. I tested user-to-item retrieval with pooled user vectors, 1,000 users against the 1.2M items, the shape you get from a two-tower recommender. Pooled vectors are centroid-like. They sit in the middle of the embedding space, far from any single item.
Qdrant’s binary quantization collapses on these vectors. Even with 400 candidates it does not reach 0.95. Turn quantization off and it recovers, at three times the latency. Elastic’s BBQ holds 0.985 with quantization on. My reading is that the difference is the rescoring step. Elastic searches the quantized index and then rescores the candidates against the full-precision vectors, and that step saves it on hard queries.
If you plan to run a recommender on top of your vector store, test this before you pick. A store that is excellent on item-to-item can fail quietly on user-to-item. This is the part I actually wanted to write, because it is the part the vendor benchmarks never show you.
My first contention numbers were garbage: The first campaign used a 20 second idle settle between seeding the collection and starting the measurement. Qdrant’s background optimizer was still churning through segments from the seed when I started measuring. Tail latencies came out 2 to 10 times higher than reality. The fix was a 240 second settle. Every contention number from before that fix went in the bin. If your benchmark shows one provider failing exactly one scenario and passing the rest, suspect the benchmark before you suspect the provider:
Serverless over the public internet is not a fair test: I said it above and I will say it again because it is the easiest mistake to make. Pinecone ran on starter credits, and its 135 qps ceiling is throttling at the serverless tier. It says nothing about the engine on dedicated read capacity. Turbopuffer was a paid plan, so money was not the problem there, but it sits on object storage and has cold-start characteristics that a serverless plan does not let you tune away. And both were reached over the public internet, with no PrivateLink and no VPC peering, while Elastic and Qdrant were one hop away from the benchmark client. Some part of every Pinecone and Turbopuffer latency number is network, and I cannot say how much. I knew all of this going in and still caught myself reading those columns as if they meant something about the products. A fair comparison needs dedicated capacity, private connectivity and the same level of vendor support for everyone. I had that for two of the four. Anything less and the remaining columns are noise with a vendor name on top.
Recall decays under churn: Qdrant’s text recall was 0.960 at 50 candidates on a fresh index. After a campaign of update churn it was 0.925 at 400 candidates. The 0.95 target was unreachable without a rebuild. This is how HNSW behaves when you keep deleting and reinserting, and I would expect it from any HNSW-based store. If you write continuously, budget for periodic rebuilds or put candidate headroom into your recall SLA. I did not measure the same thing on Elastic. Its segment merging model probably behaves differently, but I would not assume it behaves better. This is also something ann-benchmarks will never show you.
The cluster you run today is not the baseline: The incumbent Elastic cluster was on an older version, heavy on RAM and light on CPU. An earlier, less careful test on that cluster had shown CPU saturating around 180 qps on three nodes, with p99 spikes of several seconds during segment merges. That number was the one everyone remembered, and it was the reason the whole evaluation happened. A fresh 9.4 deployment with the same data passed 750 qps. Half of what people think they know about Elastic vector search, including what I thought, is about an old version on the wrong hardware.
Nobody sustained the batch stream: One scenario had a 370 qps batch load hitting the same collection as the online traffic. No provider sustained it on a single collection. Elastic got to about 214 qps, Qdrant to about 167. Moving the batch consumer in-process or onto its own collection fixes that (read-only copy, load isolation).
List price for the six-node Qdrant cluster was around $3,000 a month, with high availability off and backups billed separately. A right-sized Elastic deployment with 60 GB of hot memory came in around $4,800 a month, Kibana included. Add HA and backups to the Qdrant side, add migration engineering and a dual-running period, and the difference is mostly gone.
There is no clear winner.
Qdrant is better at freshness, filtered recall, upsert tails and latency at moderate load, and it does that on a smaller cluster with less memory. If your workload writes continuously and reads with selective filters, that is your answer.
Elasticsearch is better at raw fresh-index latency, ingest rate, and pooled query vectors under quantization. If you plan a recommender on top of the same store, or you already run Elastic and your problem is that vector search is slow, upgrade and resize before you migrate. There is a good chance the problem goes away, and you keep BM25 and aggregations in the same place.
On the scenarios that come from real traffic, both pass with room to spare. So the choice comes down to which of the two lists above describes your workload. A few years ago that was not the case. Qdrant was ahead on everything that mattered and Elastic was not an option. That is over.
Pinecone and Turbopuffer, on free tiers, are not in this race. On dedicated capacity with vendor support they might well be. I did not test that, so I will not guess.
And whatever you do, distrust any benchmarks. The vendor benchmarks will not tell you anythink they are pure PR, and ann-benchmarks can not tell you anything more than algorithmic performance. Even my result here is just one more datapoint, test your workload yourself and compare.
I offer consulting for ML/AI Architecture & ML Production Systems: https://digital-thinking.de/ Originally published at https://ixeption.substack.com on September 7, 2026.
Every Vector Database Benchmark is Wrong was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.