AI vector search database supplier Qdrant has a TurboQuant-based compression scheme. It tells us that, for many organizations, the real limit on AI systems isn't model size anymore. It's how much GPU memory they can afford. Every time a team scales up RAG, semantic search, or recommendation workloads, the vectors and model "working memory" behind them quickly eat through GPU and RAM budgets.
For the past few years, the only real answer has been a mix of familiar quantization schemes and more hardware, with teams accepting that aggressive compression usually means noticeably worse recall and less predictable behavior. TurboQuant, developed by Google Research, is part of a shift away from "buy more hardware" toward "make existing memory work harder." It evolves quantization in a way that directly targets the growing memory bottleneck. On the LLM side, TurboQuant compresses the KV cache, the model's short-term memory in GPU VRAM, by multiples with near-zero accuracy loss, which can translate directly into either fewer GPUs per deployment or longer contexts and more concurrent users on the same hardware.
In vector search, Qdrant has taken the same idea and applied it to embeddings, where the footprint lives in host RAM and on disk: reshaping vectors so they compress more cleanly, then layering in extra corrections that hold recall higher than the existing options at the same footprint. At 8x compression, it is competitive with scalar quantization, and it beats binary quantization at every storage class.
We asked Qdrant some questions about this and they were answered by core team engineer Ivan Pleshkov.
Blocks & Files: How do you diagnose when memory, not compute, is the bottleneck?
Ivan Pleshkov: In vector search, memory is usually the first place to look. A billion openai embeddings is around 6TB, so the pressure tends to show up there before it shows up on the CPU. A useful signal is latency that grows with the size of the collection rather than with the query load, while CPU utilisation stays low. From there the quickest check is to switch quantization on and measure again: if search gets noticeably faster, the constraint was the number of bytes you have to pull from memory rather than the number of calculations you have to do.
Blocks & Files: Why do you need more than multi-tiered KV Cache schemes which effectively extend HBM?
Ivan Pleshkov: This sounds like it is about TurboQuant in inference. The original work from Google is about compressing the KV cache that lives inside an inference engine, in HBM. We took the algorithm from that work and applied it to a different problem: the persistent index behind vector search. There is no KV cache and no HBM on our side. Search in Qdrant runs on CPU (we currently use GPUs for index construction only, not for search), so the memory hierarchy we deal with is DRAM, NVMe and object storage. What the two have in common is the mats — both compress high-dimensional vectors, and a random rotation helps in either case.
In Qdrant, this plays out concretely. Even when quantization is enabled, we persist the original float32 vectors by default. That lets us keep the compressed vectors in RAM and run the fast part of the search entirely there, then rescore the top candidates against the originals to bring recall back up. So there is a tier boundary in our system, too – quantized in memory, original on disk – and it is a balance between memory footprint and recall. What compression quality changes is where that balance sits: the better the quantization, the fewer candidates need rescoring, and the less you go to slow storage. That is why we look at recall per bit rather than at the compression ratio alone.
Blocks & Files: If vector embeddings compression is needed to more effectively use KV Cache, what are the alternatives?
Ivan Pleshkov: For Qdrant, this is about the memory the vector search index needs, not the KV cache.
The most obvious alternative to quantization is the same one you have in any database: grow the cluster. Add nodes, shard the data, and each node holds less. It works, and it needs nothing specific to vectors. It is simply more expensive, because you are paying for RAM instead of saving it. For many teams that is exactly the point where quantization starts to pay off.
The second option is specific to vector search: Matryoshka embeddings. If the model is trained so that the first N dimensions already carry a meaningful representation on their own, you can truncate the vector instead of compressing it. A smaller dimensionality gives you the same kind of memory saving, and you control accuracy by choosing N. The difference is where rescoring lives. With quantization it happens inside Qdrant. With truncation you need both the short and the full vectors, and part of the two-stage search logic moves to the application side. That is not worse, but it is a different split of responsibility.
Blocks & Files: How do you decide among scalar, binary, product quantization, and TurboQuant-based compression in real deployments?
Ivan Pleshkov: It starts from your search requirements and from where you are willing to make the trade-off. There are three parameters – cost, speed and accuracy – and as usual you get to pick two of them.
Let's take the balance between cost and accuracy first. If you can afford more RAM, use the more accurate methods: 4-bit TurboQuant or scalar quantization. TurboQuant at 4 bits takes half the memory of scalar quantization with comparable recall — but comparable does not mean equal. On some datasets, scalar quantization gives better recall, and for some customers that matters more than a 2x difference in memory.
If memory is tight, you move into the 1–2 bit league: TurboQuant or binary quantization. Accuracy is noticeably lower there, and you are choosing again between two options — either you do more rescoring, or you accept the drop in recall. So the balance between cost and accuracy turns into a balance between speed and accuracy. There is no universal answer here, only your own set of requirements. What is worth doing in any case is testing on your own data: behaviour under aggressive compression depends on the embedding model more than on anything else.
More detail here. Blocks & Files: Is the compression reversed when the compressed vector embeddings hit GPU HBM?
Ivan Pleshkov: Vector search is not inference, so we do not use GPUs for search at all. We can build the index on a GPU, but querying it is always done on a CPU. So the compressed vectors never reach HBM in the first place.
The more interesting part of the answer is that we do not decompress anywhere. All quantization methods in Qdrant are built to work directly on the compressed data — distances are computed from the codes, not from reconstructed vectors. For TurboQuant we wrote dedicated SIMD optimisations for this: scoring comes down to integer operations instead of floating point arithmetic.
This is worth separating from what compression usually means in storage, where data sits compressed and is unpacked before you compute on it. Here it is not packing, it is a different representation that the arithmetic works on directly. So savings on how much data you have to pull from memory holds all the way through to the computation — compression buys you speed, not just memory.
There are a couple of subtleties in what I just said, but they are all covered in our quantization documentation.
Blocks & Files: How does Qdrant compression compare and contrast with Pinecone and Zilliz vector compression schemes?
Ivan Pleshkov: I do not have enough knowledge of how our competitors' solutions work internally to make strong claims about them, so I would rather not.
On the scientific side I can say something. There has been a lot written about whether TurboQuant is a form of RaBitQ. I agree on the shared foundation; both methods are built on random rotations and asymmetric scoring. But as an engineer I also see the differences between them.
So instead of taking sides, in the Qdrant implementation I used techniques from both. Rotations come from both methods. The Lloyd-Max codebook comes from TurboQuant. Length renormalisation and 1-bit scoring comes from RaBitQ. In our articles and documentation we name the authors whose methods we use.
Our main addition is anisotropy compensation. The theory assumes isotropic data, but real embeddings can be anisotropic, so we calibrate the per-coordinate distribution onto the codebook grid. That is what makes the algorithm work on real production embeddings, not only on the ones that suit the theory.
**Blocks & Files: How can you set up an actionable evaluation plan to decide amongst the alternatives? **
**Ivan Pleshkov: **I have largely answered this in the question about choosing a method. The plan comes down to deciding where you are willing to balance cost, speed and accuracy, and then testing the candidates on your own data.
For the practical side, Qdrant has a web interface that shows resource consumption, down to how much memory quantization takes, what it costs to build, etc. That lets you evaluate the options against actual numbers rather than general reasoning. Beyond that, I would recommend going deeper into the documentation, where the finer quantization settings and the search parameters for quantized data are described - those are what let you tune the balance to where you need it.
Bootnote
A quick head's up: 1.19 ships soon with a new TurboQuant storage format with four-bit compression accurate enough to drop the original vectors and skip rescoring entirely. That gets you up to 9x less storage than TurboQuant today.