Consider a support search service that has found a promising replacement embedding model. The evaluation looks encouraging, but the collection already contains vectors from the current model. New documents keep arriving, users keep searching, and a configuration change cannot turn the stored vectors into a different representation. The model upgrade is also a data migration.
What concerns me is the interval between the first replacement vector and the last production request using the old model. During that interval, two retrieval versions coexist. Each needs a coherent pairing of document embeddings, query encoding, and index configuration. A migration can finish generating vectors and still be unsafe to activate.
Jack Li’s embedding-model migration tutorial demonstrates an online field-based approach with Milvus 3.0.2. The compact example below applies that mechanism to a disposable collection. Its purpose is to expose the decisions around backfill, readiness, and rollback. It is an implementation walkthrough, not a claim that these models have been benchmarked on your workload.
Use a running Milvus 3.0.2 server and a matching PyMilvus 3.0.2 client, with the model dependencies installed. The source repository pins its complete environment; consult that setup for reproducibility. The example loads two Sentence Transformers models on CPU and requires network access on their first download. Run the snippets in order in the same Python session, against a test endpoint.
I would start with a few documents whose expected retrieval behavior is easy to inspect. That makes schema and routing mistakes visible before a large backfill makes them expensive. These documents are deliberately small examples, not an evaluation dataset. Their job is to prove that the pieces connect, while a separate representative query set determines whether the replacement should receive traffic.
from pymilvus import DataType, MilvusClientfrom pymilvus.model.dense import SentenceTransformerEmbeddingFunctionclient = MilvusClient(uri="http://localhost:19530")collection = "james_embedding_upgrade_demo"options = dict(device="cpu", normalize_embeddings=True, query_instruction="Represent this sentence for searching relevant passages: ", doc_instruction="")old_model = SentenceTransformerEmbeddingFunction( model_name="BAAI/bge-small-en-v1.5", **options)new_model = SentenceTransformerEmbeddingFunction( model_name="BAAI/bge-base-en-v1.5", **options)rows = [ {"id": "a", "text": "Reset an expired API credential in account settings."}, {"id": "b", "text": "Retry transient failures with bounded exponential backoff."}, {"id": "c", "text": "Delete obsolete access tokens after a credential rotation."},]
The query instruction is configured once. Use the model’s query-encoding method for requests and document-encoding method for stored text. Adding the instruction a second time changes the input. For an experiment intended to isolate the embedding model, I would also keep document IDs, text preparation, and chunk boundaries fixed rather than quietly changing all three.
The current model uses a 384-dimensional field; the replacement uses 768 dimensions. Matching those dimensions is a basic validation requirement. The more important invariant is semantic: a field belongs to the model and preprocessing that generated it. Even two models with identical output dimensions should not be assumed to share the same embedding space.
This snippet creates the baseline collection only if the name is unused. It deliberately refuses to drop an existing collection. A convenient cleanup statement in a tutorial can become an unpleasant surprise when copied into a real environment. Choose a fresh collection name when repeating the experiment, and retain the old serving field throughout the migration exercise.
if client.has_collection(collection): raise RuntimeError("Choose a new demo collection name; existing data is preserved.")schema = client.create_schema(auto_id=False, enable_dynamic_field=False)schema.add_field("id", DataType.VARCHAR, is_primary=True, max_length=64)schema.add_field("text", DataType.VARCHAR, max_length=4096)schema.add_field("old_vector", DataType.FLOAT_VECTOR, dim=384)client.create_collection(collection, schema=schema, consistency_level="Strong")old_vectors = old_model.encode_documents([row["text"] for row in rows])client.insert(collection, data=[ {**row, "old_vector": vector.tolist()} for row, vector in zip(rows, old_vectors)])
A separate nullable field lets the replacement representation coexist with the baseline. The source tutorial explains that adding the field does not compute embeddings for existing records. Missing values remain missing until the application supplies them. That distinction is useful operationally: schema readiness and data completeness are different milestones, and they should appear separately in the rollout checklist.
The following helper builds one HNSW index per vector field. These settings illustrate a consistent comparison; they are not a universal tuning recommendation. I would keep the index parameters fixed for the first comparison, then investigate tuning independently. Otherwise an apparent model improvement may partly reflect a change in the retrieval configuration.
def create_index(field): params = client.prepare_index_params() params.add_index(field_name=field, index_name=field + "_hnsw", index_type="HNSW", metric_type="COSINE", params={"M": 16, "efConstruction": 100}) client.create_index(collection, index_params=params)client.flush(collection)create_index("old_vector")client.load_collection(collection)client.add_collection_field(collection, field_name="new_vector", data_type=DataType.FLOAT_VECTOR, dim=768, nullable=True)create_index("new_vector")
The existing primary key should connect the replacement vector to its document. Partial updates allow the backfill to supply the new field without resending unrelated values. This small example uses the known source rows; a real job should read from an authoritative source in bounded batches and record checkpoints so interrupted work can be reconciled.
new_vectors = new_model.encode_documents([row["text"] for row in rows])client.upsert(collection, data=[ {"id": row["id"], "new_vector": vector.tolist()} for row, vector in zip(rows, new_vectors)], partial_update=True)client.flush(collection)client.load_collection(collection, load_fields=["id", "text", "old_vector", "new_vector"])client.refresh_load(collection)
A fixed demonstration corpus avoids the hardest concurrency problem. In a live system, a document might change after the backfill reads it. Writing the older replacement embedding later can leave the vector inconsistent with the current text. I would design version checks or an ordered change-processing path around that possibility, rather than assuming that completing the initial scan establishes correctness.
New documents and deletions need the same attention. During the rollback window, both retrieval versions must reflect the application state they claim to serve. The exact synchronization mechanism depends on the ingestion system. What matters is making its progress measurable and defining how a late or failed update prevents an unsafe traffic switch.
An index can finish processing the vectors it received while some required records still lack replacement vectors. That is why I would separate two checks: whether the expected entities were backfilled, and whether the replacement index is ready. For the three-document exercise, the following verification can inspect every entity directly. A larger migration needs scalable reconciliation instead.
import timestored = client.query(collection, filter="", limit=10, output_fields=["id", "old_vector", "new_vector"])assert {row["id"] for row in stored} == {row["id"] for row in rows}assert all(len(row["old_vector"]) == 384 and row.get("new_vector") is not None and len(row["new_vector"]) == 768 for row in stored)deadline = time.monotonic() + 180while True: state = client.describe_index(collection, "new_vector_hnsw") if state.get("state") == "Finished" and state.get("pending_index_rows", 0) == 0: break if time.monotonic() >= deadline: raise TimeoutError("Replacement index did not become ready.") time.sleep(1)
Coverage checks do not prove retrieval quality. Run the same labeled queries through both versions and compare relevant-document coverage and ranking quality. Inspect disagreements rather than reporting only averages. A replacement that improves common requests but loses a critical operational document may be unsuitable even when its overall score increases.
I would keep application behavior in the evaluation too: encoding latency, search latency, and errors all affect the user. Model inference may cost more than retrieval for some requests. Treat the full retrieval path as the unit of comparison so a local improvement does not obscure an end-to-end regression.
Resolve the encoder and vector field together at the beginning of a request. That prevents a configuration update from changing the destination halfway through processing. The function below makes that pairing explicit and allows callers to choose a version for comparison. It performs no automatic traffic shifting and does not remove either field.
routes = {"old": (old_model, "old_vector"), "new": (new_model, "new_vector")}def search(question, version="old"): encoder, field = routes[version] vector = encoder.encode_queries([question])[0].tolist() return client.search(collection, data=[vector], anns_field=field, limit=3, output_fields=["text"], search_params={"metric_type": "COSINE", "params": {"ef": 128}}, consistency_level="Strong")print(search("How should I replace an expired token?", "old"))print(search("How should I replace an expired token?", "new"))
In production, I would introduce the replacement gradually and keep a clear route back while observing results. The old model must remain available, and its document vectors must remain current. A rollback option that points at stale data is not the protection its name suggests. Cleanup should follow the observation window, not the first successful replacement query.
The Milvus schema-change documentation and partial-update guide are useful implementation references. The migration decision itself belongs to the application: define acceptable quality, prove coverage, and preserve a coherent old route until the replacement earns its place. Better embeddings are a candidate improvement. A controlled migration is what makes that improvement usable.
An Embedding Upgrade Needs a Rollback Plan, Not Just a Better Model was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.