{"slug": "why-your-embeddings-model-upgrade-silently-broke-retrieval", "title": "Why Your Embeddings Model Upgrade Silently Broke Retrieval", "summary": "Upgrading an embedding model silently breaks retrieval when stored document vectors and fresh query vectors come from different models, because each model maps text into its own coordinate space and the vector database still returns nearest neighbors without erroring, according to Zilliz's engineering team, whose guidance covers Milvus, Pinecone, Qdrant and Weaviate. Zilliz notes that mixing model versions in one HNSW or IVF index returns some neighbors from a completely wrong neighborhood, and the failure is only visible when vector sizes differ: Google's move from text-embedding-004 to gemini-embedding-001 jumped vector size from 768 to 3072 and produced a 400 error in an n8n community thread, while OpenAI's text-embedding-3-small can be configured to return 768 dimensions, which matches the shape of old data without aligning the spaces. Tian Pan's write-up on embedding model churn describes providers that keep the right to swap the underlying model without changing the endpoint name or giving notice, and calls in-place replacement the most dangerous strategy because it leaves a window where the index mixes old and new vectors.", "body_md": "The worst retrieval bug I know of throws no error. Latency stays flat. The error rate sits at zero. The index reports healthy, and every query still returns ten results.\n\nSome of those results are wrong, and nothing in your stack can tell you which ones.\n\nThat is what a bad embedding model upgrade looks like in production. To write this I read vendor docs, engineering write ups, a developer forum thread and one EMNLP paper. They agree on one point: the failure is quiet by design. Below are the three facts behind it, five paths into it, and the checks that make it loud.\n\nAn embedding model upgrade breaks retrieval when stored document vectors and fresh query vectors come from different models. Each model maps text into its own coordinate space, so similarity between the two is meaningless. The vector database still returns nearest neighbors, so nothing errors, and answers quietly get worse.\n\nThree facts explain the whole problem:\n\nFix the third fact first. You cannot protect a system you do not measure.\n\nZilliz’s engineering team states it directly: a vector for the same document from version one of a model and version two of that model sit in different spaces, so cosine similarity between them has no meaning. Their guidance covers Milvus, Pinecone, Qdrant and Weaviate alike.\n\nPicture two maps of one city, each drawn with a different origin and rotation. Both maps are accurate. Copy a pin from one onto the other and it lands in the wrong neighborhood.\n\nAn index holding both kinds of vectors behaves the same way. Zilliz notes that mixing versions in one HNSW or IVF index returns some neighbors from a completely wrong neighborhood. The database cannot see this. It only sees numbers of the right length.\n\nSome upgrades announce themselves. Google moved developers from text-embedding-004 to gemini-embedding-001, and the vector size jumped from 768 to 3072. In a public [n8n community thread](https://community.n8n.io/t/google-deprecating-text-embedding-004-but-gemini-embedding-001-doesnt-work/262008?tl=en), inserting the new vectors into a 768 dimension store returned a 400 error that named the mismatch.\n\nThat error is a gift. Someone files a ticket and the team fixes it the same day.\n\nThe same thread offers a workaround: OpenAI’s text-embedding-3-small can be configured to return 768 dimensions. That is fine if you reembed every document with the new model. If you keep the old vectors, matching the size has only matched the shape of the data. The spaces stay misaligned and the alarm is gone.\n\nSame size, different space is the case to fear. The index accepts every write, every query runs, and the only symptom is relevance.\n\nYou change the model name in config and the vector size stays the same. Existing vectors stay in the index. Every new query is encoded in the new space and compared against documents from the old one.\n\nTian Pan’s write up on embedding model churn describes providers that keep the right to upgrade or swap the underlying model without changing the endpoint name or giving notice. Most API terms permit it. You changed nothing, and your index is still invalidated.\n\nThe same write up calls in place replacement the most dangerous strategy. You stop writes, overwrite vectors, and resume, which leaves a window where the index mixes old and new vectors. Mixpeek’s glossary adds that large jobs that encode the whole corpus again will fail partway through. A job that dies midway leaves the mix in place until someone notices.\n\nA shared client library bumps its default model, for example. Queries now come from the new encoder while the corpus still holds old vectors. Tian Pan describes exactly this mismatch: an index full of vectors from a coordinate system that no longer matches what the query encoder produces.\n\nA Prompt/Deploy breakdown of reindex triggers says a model change needs a full reindex, a chunking change needs a full reindex, and doing both together cannot be separated because both are tied to the vectors. If quality drops, you cannot tell which change caused it. Ship one variable per release.\n\nApproximate nearest neighbor search has no concept of “no good answer.” It returns the closest points it can find. A broken index returns ten results with similarity scores that look like any other numbers.\n\nYour retriever hands those passages to the language model. The model writes a fluent answer from the wrong material. The incident now reads as a hallucination, and the team starts tuning prompts for a problem that lives in the data layer.\n\nThe cheapest detector is a self consistency test. Take a random sample of stored documents, embed them again with the model your query path uses today, and compare the fresh vector to the stored one. If nothing changed, the two should be nearly identical.\n\npython\n\n``` python\nimport randomimport numpy as npdef cosine(a, b):    a, b = np.asarray(a), np.asarray(b)    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))def drift_check(store, embed, sample_size=200, seed=7):    # store.all_ids(), store.get_text(id), store.get_vector(id) are your own accessors.    # embed(text) calls the same model your query path calls today.    ids = list(store.all_ids())    random.Random(seed).shuffle(ids)    scores = []    for doc_id in ids[:sample_size]:        stored = store.get_vector(doc_id)        fresh = embed(store.get_text(doc_id))        scores.append(cosine(stored, fresh))    return {        \"mean\": float(np.mean(scores)),        \"p5\": float(np.percentile(scores, 5)),        \"min\": float(np.min(scores)),    }\n```\n\nTreat the numbers carefully. Some providers return tiny differences between identical calls, so the right alarm threshold depends on your stack. Run the check on a day when you know nothing changed, record that baseline, and alert when the mean or the fifth percentile falls below it.\n\nSchedule it. A provider swap on a Tuesday should not wait for a quarterly review to be found.\n\nDrift tells you the vectors changed. A canary set tells you whether users feel it. Collect real queries from your logs, label which documents should come back for each, and track how often at least one correct document lands in the top 10.\n\n``` python\ndef hit_rate_at_k(search, golden, k=10):    # golden is a list of (query, set_of_relevant_doc_ids)    hits = 0    for query, relevant_ids in golden:        results = set(search(query, k))        if results & set(relevant_ids):            hits += 1    return hits / len(golden)\n```\n\nI would start with 50 to 100 queries and grow the set whenever a user reports a bad answer. Run it before and after every change to the embedding path. A migration without this gate is a guess.\n\nThese six steps follow the pattern the sources above converge on.\n\nNot every change needs the full treatment. The Prompt/Deploy breakdown draws the line this way:\n\nChangeReindex needed?New metadata field, no content changeUsually no, upsert metadata onlyDocument content updatePartial, changed documents onlyChunking strategy changeYes, full reindexEmbedding model changeYes, full reindexModel and chunking change togetherYes, full reindex, and the effects are inseparable\n\nMixpeek gives a useful anchor: a corpus of 100 million documents at $0.0001 per embedding costs $10,000 and several days of compute.\n\nThat bill is real. It is also small next to a quarter of degraded search that nobody measured. High cost is the reason teams postpone migrations until a deprecation deadline forces one, which Mixpeek lists as a common pitfall because it leaves no room for gradual rollout or validation.\n\nResearchers are working on ways to skip the full reembed. The Drift Adapter paper, published in EMNLP 2025 main, trains a small transformation that maps new queries into the legacy embedding space, so the existing index keeps serving. On MTEB text corpora and a CLIP image model upgrade covering 1 million items, it recovered 95 to 99 percent of the recall of a full reembed while adding only microseconds of query latency. ([arXiv](https://arxiv.org/html/2509.23471v1))\n\nTian Pan’s write up also points to Vec2Vec style translation between model spaces, reporting cosine similarity above 0.9 to native embeddings across architectures.\n\nThese are promising bridges. The paper itself frames the adapter as a way to defer full recomputation. Read the numbers as a loss of 1 to 5 percent of recall against a full reembed. For a contract search or compliance tool, that remainder is where the missed clause lives. Use an adapter to buy time, then finish the migration.\n\nFour steps, one afternoon, and the next silent break becomes a visible alert.\n\nNo. Each model creates its own coordinate space, so similarity between vectors from different models is meaningless. A mixed index returns some neighbors from the wrong neighborhood without raising an error.\n\nEmbed a random sample of stored documents again and compare the fresh vectors to the stored ones. A drop below your recorded baseline signals drift. Pair it with a canary query set so you see the effect on real retrieval.\n\nFor a model change or a chunking change, yes, a full reembed is needed. Metadata only changes usually need no reembedding, and content edits need only the changed documents. Adapter research can defer the full job but does not remove it.\n\nTian Pan suggests keeping it until you have two to four weeks of post migration signal. That window lets you roll back if a quality regression shows up late.\n\nNot entirely. Drift Adapter recovered 95 to 99 percent of full reembed recall in its experiments. Treat it as a bridge that buys time while you run the real migration.\n\nTreat an embedding model upgrade as a data migration. Version your vectors, measure drift on a schedule, gate cutover on a canary set, and keep the old index for a few weeks.\n\nIf this saved you from a silent break, follow The AI Engineer for more production failure analysis. And if you have already been bitten, tell me where it broke in the comments.\n\n[Why Your Embeddings Model Upgrade Silently Broke Retrieval](https://pub.towardsai.net/why-your-embeddings-model-upgrade-silently-broke-retrieval-aa59e7fdbd75) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/why-your-embeddings-model-upgrade-silently-broke-retrieval", "canonical_source": "https://pub.towardsai.net/why-your-embeddings-model-upgrade-silently-broke-retrieval-aa59e7fdbd75?source=rss----98111c9905da---4", "published_at": "2026-11-05 04:46:37+00:00", "updated_at": "2026-11-05 04:56:44.602528+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-infrastructure", "ai-tools"], "entities": ["Zilliz", "Milvus", "Pinecone", "Qdrant", "Weaviate", "Google", "text-embedding-004", "gemini-embedding-001"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/why-your-embeddings-model-upgrade-silently-broke-retrieval", "markdown": "https://wpnews.pro/news/why-your-embeddings-model-upgrade-silently-broke-retrieval.md", "text": "https://wpnews.pro/news/why-your-embeddings-model-upgrade-silently-broke-retrieval.txt", "jsonld": "https://wpnews.pro/news/why-your-embeddings-model-upgrade-silently-broke-retrieval.jsonld"}}