Show HN: Zero downtime embedding model upgrades

A developer has released EmbedFlow, an open-source tool that enables zero-downtime upgrades of embedding models by retrieving top-K documents with the old model and reranking them with the new model, avoiding the cost of re-embedding entire databases. In tests on a 1M-document Natural Questions dataset, the approach achieved nDCG@10 scores of 0.6816 for Qwen3-4B to Qwen3-8B with K=50, nearly matching the native Qwen3-Embedding-8B score of 0.6812. The tool is available via pip install embedflow and on GitHub.

People use embedding models all the time for rag/semantic retrieval. However, when a newer, more desireable model comes out, there is an expensive both in time and computational cost of re-embedding every document in the database. However, I figured out an interesting way to forgo that upfront embedding cost. algo: old model/index - retrieve top-K docs - score those docs with the new model - cache/materialize the new embeddings so instead of rebuilding the entire vector store upfront, the old index keeps getting retrieved from, while the new model reranks those candidates. This works surprisingly well for some model pairs, i tested 63 source- target migrations on h100s, on upto 1M documents . For example, on a 1M document Natural Questions dataset, native Qwen3-Embedding-8B: 0.6812 nDCG@10 Qwen3-4B - Qwen3-8B, K=50: 0.6816 Qwen3-0.6B - Qwen3-8B, K=50: 0.6638 MiniLM - Qwen3-8B, K=50: 0.6486 the hard part is determining k, I held the k constant above to give some sense of migratability . You can install it with pip pip install embedflow and the code is on github https://github.com/arnsri33/embedflow https://github.com/arnsri33/embedflow Comments URL: https://news.ycombinator.com/item?id=49605110 https://news.ycombinator.com/item?id=49605110 Points: 1 Comments: 0