Honey, I shrunk the embeddings: Matryoshka vs. PCA

A new experiment by Dylan Castillo comparing Matryoshka Representation Learning (MRL) and Principal Component Analysis (PCA) for reducing embedding dimensions found that both methods preserve retrieval quality, but MRL requires training-time support while PCA works with any model. The study, using OpenAI's text-embedding-3-small and Alibaba's qwen3-embedding-8b across eight BEIR datasets, reduced embeddings to 512, 256, 128, 64, and 32 dimensions and found that PCA can be a viable alternative for models without MRL training.

As people began using LLMs with their own documents, storing and searching those documents became an important problem. That need drove the rapid adoption of vector databases https://en.wikipedia.org/wiki/Vector database . But these can be easily get expensive and slow, when you store large vector representations of documents. So AI labs https://openai.com/index/new-embedding-models-and-api-updates/ responded https://docs.cohere.com/changelog/embed-multimodal-v4 with a technique called Matryoshka Representation Learning MRL https://arxiv.org/abs/2205.13147 , which lets you use fewer embedding dimensions without sacrificing much accuracy in your retrieval. Since then, we’ve more or less lived happily ever after, with smaller vector database bills and faster queries. I’m not one to be worried about my vector database cost on a typical Wednesday, but Doug Turnbull’s article https://softwaredoug.com/blog/2026/07/24/pca-shrink-ray about using Principal Component Analysis PCA https://arxiv.org/abs/1404.1100 , to reduce vector dimensions, got me curious: how would it compare with MRL? So I ran an experiment comparing both methods across eight standard retrieval-quality datasets. In this article, I share what I found. All the code and data is available on GitHub https://github.com/dylanjcastillo/blog/tree/main/ extras/matryoshka-vs-pca . Both methods produce smaller vectors that behave almost like the full ones. But they get there in different ways. MRL works during training. You train the model with the loss applied at several prefix lengths at once: the first 64 dimensions, the first 128, and so on. This teaches it to pack the most important information at the start of the vector, like a set of nested matryoshka dolls. At inference time, you simply keep the first d dimensions and re-normalize the resulting vector. Many modern embedding models are trained this way, but older models aren’t, so MRL-based truncation is not available for many popular embedding models. PCA works after training, which means it can be used with any model. You take a sample of embeddings, find the directions along which they vary the most, and keep the top d of them as a projection matrix1. From then on, every document and query gets multiplied by that matrix and re-normalized before it goes into the index. But you get additional operational complexity. You need to store and version the PCA transformation, then apply the same version consistently when adding to and querying the index. The idea behind the experiment was simple: I would shrink the embeddings with each method, run the benchmarks, and see how much retrieval quality suffers at each size. I generated all embeddings through OpenRouter and evaluated retrieval on eight BEIR https://github.com/beir-cellar/beir datasets: SciFact, NFCorpus, ArguAna, FiQA, SciDocs, Quora, TREC-COVID, and Webis-Touché 2020. For each dataset, I reduced the embeddings to 512, 256, 128, 64, and 32 dimensions. | dataset | task | corpus | queries | |---|---|---|---| | SciFact | scientific claim verification | 5.2K | 300 | | NFCorpus | medical search | 3.6K | 323 | | ArguAna | counterargument retrieval | 8.7K | 1,406 | | FiQA-2018 | financial question answering | 57K | 648 | | SciDocs | citation recommendation | 25K | 1,000 | | Quora | duplicate question retrieval | 523K | 10,000 | | TREC-COVID | biomedical search COVID-19 | 171K | 50 | | Touché 2020 | argument retrieval from web docs | 382K | 49 | There were three questions I wanted to answer. To answer this, I used two MRL-trained models: OpenAI’s text-embedding-3-small 1,536 dims , and Alibaba’s qwen3-embedding-8b 4,096 dims , which sits at the top of the open-weights MTEB BEIR leaderboard. I reduced each model’s embeddings in two ways: Then I ran the benchmarks on the eight datasets, comparing the two methods on both models, to see which one preserves more retrieval quality as the embeddings get smaller. If PCA does well, that might be because MRL training has already organized the embedding space in a convenient way. To test that, I added text-embedding-ada-002 , an older 1,536-dimensional model that wasn’t trained using MRL, as a control. I applied PCA to both models. If PCA retains a similar share of retrieval quality on text-embedding-3-small and text-embedding-ada-002 , that suggests its performance does not depend on MRL training. PCA has to be fit on something, which raises two practical questions: how much fitting data do you need, and does it need to come from the corpus you’ll be searching? For the first question, I fit PCA on random samples of FiQA’s 57K documents 1,000, 5,000, 20,000, and the full corpus and checked how much the size of the fitting sample changes retrieval quality. For the second, I compared regular PCA against out-of-domain PCA : a projection fit once on 100,000 MS MARCO passages and reused, unchanged, on every other dataset. This is the lazy version: fit on whatever data you have at the start and never update it. Before running the experiment, I wanted to make sure the evaluation pipeline was producing sensible results. So I first tried to reproduce the official MTEB scores for the models. The table below compares NDCG@10 at full dimensions: the official MTEB score vs. the score produced by my evaluation pipeline. | dataset | ada-002 | 3-small | qwen3-8b | ||| |---|---|---|---|---|---|---| | MTEB | mine | MTEB | mine | MTEB | mine | | | SciFact | 0.7275 | 0.7277 | 0.7337 | 0.7296 | 0.7846 | 0.7863 | | NFCorpus | 0.3697 | 0.3705 | 0.3833 | 0.3847 | 0.4145 | 0.4150 | | ArguAna | 0.5744 | 0.5757 | 0.5549 | 0.5573 | 0.7685 | 0.7689 | | FiQA | 0.4441 | 0.4440 | 0.4491 | 0.4484 | 0.6457 | 0.6492 | | SciDocs | 0.1836 | 0.1837 | 0.2080 | 0.2077 | 0.3274 | 0.3268 | | Quora | 0.8760 | 0.8759 | 0.8883 | 0.8880 | 0.8890 | 0.8901 | | TREC-COVID | 0.6847 | 0.6884 | 0.7790 | 0.7775 | 0.9499 | 0.9492 | | Touché 2020 | 0.2161 | 0.2143 | 0.2428 | 0.2433 | 0.3593 | 0.3596 | Official numbers come from the MTEB results repository text-embedding-3-small https://github.com/embeddings-benchmark/results/tree/main/results/openai text-embedding-3-small , text-embedding-ada-002 https://github.com/embeddings-benchmark/results/tree/main/results/openai text-embedding-ada-002 , qwen3-embedding-8b https://github.com/embeddings-benchmark/results/tree/main/results/Qwen Qwen3-Embedding-8B . My results are close enough to the official numbers that I’m confident the pipeline is working as expected. PCA matched or outperformed MRL truncation at nearly every dimension across both MRL-trained models. Each cell below shows how much retrieval quality survives the reduction: NDCG@10 at that dimension divided by NDCG@10 at full dimensions 1,536 for text-embedding-3-small , 4,096 for qwen3-embedding-8b , averaged over the eight datasets. | dims | 3-small MRL | qwen3-8b MRL | || |---|---|---|---|---| | Truncation | PCA | Truncation | PCA | | | 512 | 98% | 97% | 99% | 98% | | 256 | 94% | 95% | 96% | 96% | | 128 | 86% | 90% | 91% | 91% | | 64 | 71% | 82% | 83% | 84% | | 32 | 46% | 65% | 68% | 71% | At 512 dimensions, MRL truncation has a slight advantage. But as the vectors get smaller, PCA catches up and either beats or matches MRL. On text-embedding-3-small , PCA is clearly better below 256 dims 65% vs 46% retained at 32 . On qwen3-embedding-8b the race is much closer 71% vs 68% at 32 . One thing I found interesting is that the MRL training in qwen3-embedding-8b is strong enough that its truncated 32-dim vectors, cut from 4,096 dimensions, retain more quality than 3-small’s cut from 1,536. Going dataset by dataset, you find some differences: At the smallest dimensions, PCA wins on seven of the eight datasets for text-embedding-3-small and six of the eight for qwen3-embedding-8b . Across dimensions, PCA performs better on SciFact, FiQA, and NFCorpus for both models. ArguAna and SciDocs also favor PCA with text-embedding-3-small , although the results are less conclusive with qwen3-embedding-8b . Truncation performs better on Quora for both models. Touché favors truncation on qwen3-embedding-8b but is basically a tie on text-embedding-3-small , and the result for TREC-COVID depends on the dimension. I worried that PCA might only look good on text-embedding-3-small and qwen3-embedding-8b because MRL training had already organized their embedding spaces in some convenient way. If that were true, PCA on text-embedding-ada-002 , which never saw MRL training, should be noticeably worse. For the most part, it isn’t. Down to 128 dimensions, PCA retains nearly the same share of quality on text-embedding-ada-002 as on text-embedding-3-small ; below that, a small gap opens: | dims | ada-002 no MRL | 3-small MRL | | |---|---|---|---| | PCA | Truncation | PCA | | | 512 | 99% | 98% | 97% | | 256 | 96% | 94% | 95% | | 128 | 89% | 86% | 90% | | 64 | 78% | 71% | 82% | | 32 | 59% | 46% | 65% | PCA on text-embedding-ada-002 retains 78% at 64 dims and 59% at 32, against 82% and 65% on text-embedding-3-small . It might be that MRL genuinely helps, or simply that text-embedding-3-small is a newer, better model. I can’t separate the two with this experiment. But either way, the hypothesis that PCA only works because of MRL seems unlikely given that even PCA on the non-MRL model beats truncation on the MRL model at those dimensions 59% vs 46% at 32 dims . Out of curiosity, I also tried truncation on text-embedding-ada-002 , even though the model was never trained for it. It held up much better than I imagined, with 83% retained at 128 dimensions. Compared to MRL, PCA adds more operational complexity: fitting and versioning the projection, using it while querying, updating the projection when the index changes. So I wanted to know what happens if you’re lazy: fit PCA once, when you first create the index, and never touch it again. In-domain fit on a small sample. In this scenario, you fit PCA on the documents you have at the start 1,000, 5,000, 20,000, or all of FiQA’s 57K and use that projection for everything you index afterwards. At this scale, it doesn’t matter much: the projection fit on 1,000 documents performs almost the same as the one fit on the full corpus, at every dimension, on both models. Out-of-domain fit. In this scenario, the data you fit on doesn’t match the data you end up searching, either because your corpus drifted over time or because you fit on whatever was available. To simulate the extreme version, I fit PCA once on 100K generic MS MARCO passages and searched every other dataset with that projection. On text-embedding-3-small , the transferred fit is as good or better on average from 512 through 64 dimensions. Dataset-level gaps are larger in both directions, as the chart shows. On qwen3-embedding-8b it keeps up down to 128 dimensions but falls behind at lower dimensions 56% vs 71% retained at 32 dims . Good news for lazy people like me. Fit-sample size matters much less than I expected, and an out-of-domain fit holds up at moderate compression. Qwen at 32 dimensions is the clear exception. These patterns may be a consequence of the scale of the experiments or the particular benchmarks I used. You can also shrink vectors using a technique called quantization. Instead of reducing the number of dimensions, quantization stores each dimension using fewer bits. With int8 , each dimension uses 1 byte 4x smaller than float32 ; with binary, just the sign bit 32x smaller . For the numbers below, int8 quantizes the document vectors while keeping queries at float32; binary quantizes both documents and queries and ranks by dot product, which is equivalent to Hamming-distance ranking. Using quantization alone, you get these results for text-embedding-3-small at full 1,536 dimensions: | config | bytes per vector | size vs full float32 | quality retained | |---|---|---|---| | int8 | 1,536 | 25% | 100% | | binary | 192 | 3.1% | 95% | You can push this further by combining quantization with truncation or PCA. The resulting vectors can be dramatically smaller while still preserving a surprising amount of retrieval quality. For example, combining binary quantization with PCA at 512 dimensions retains 82% of the original performance while reducing the raw vector size to about 1% of the float32 baseline. That seems a bit too optimistic, so it might just be my AI psychosis. Here are the full results https://github.com/dylanjcastillo/blog/tree/main/ extras/matryoshka-vs-pca/data/analysis/openai text-embedding-3-small/quantization summary.csv . PCA not only held its own against MRL truncation, it won on most dimensions. On text-embedding-3-small , PCA beat MRL truncation at nearly every dimension, and by a wide margin at aggressive compression: 65% of quality retained at 32 dims against truncation’s 46%. On qwen3-embedding-8b , MRL had a small edge at 512 dimensions and was effectively tied at 256, while PCA led below that. These results also don’t appear to be an artifact of MRL training. PCA worked similarly well on ada-002, which wasn’t trained for truncation. It’s also lower-maintenance than I expected. Results barely changed when fitting on a small in-domain sample. An out-of-domain fit also held up at moderate compression. Not bad for a technique that’s older than the washing machine2 If you want to look at the data yourself, the full pipeline is in the repo https://github.com/dylanjcastillo/blog/tree/main/ extras/matryoshka-vs-pca . this page https://setosa.io/ev/principal-component-analysis/ explains it quite well↩︎ electric washing machine↩︎ @online{castillo2026, author = {Castillo, Dylan}, title = {Honey, {I} Shrunk the Embeddings: {Matryoshka} Vs. {PCA}}, date = {2026-08-01}, url = {https://dylancastillo.co/posts/matryoshka-vs-pca.html}, langid = {en} }