{"slug": "run-local-embedder-for-0-24-of-the-openai-price", "title": "Run local embedder for 0.24% of the OpenAI price", "summary": "Running a local embedding model on a $100-per-month GPU can cost as little as 0.24% of the price of using OpenAI's small embedding model via API, according to a technical analysis by Dzen Platform. The analysis, based on the dzen_embedder benchmark, shows that a GTX 1080 can process up to 640 million 500-token fragments per month, which would cost $6,400 with OpenAI's small model, $41,800 with OpenAI's large model, $3,900 with Cloudflare's bge-m3, and from $6,400 with Voyage. Dzen Platform argues that local models offer cost savings, better language support for Russian and specialized domains, and improved security by keeping documents in-house.", "body_md": "Choosing an embedding model determines which documents search finds before a language model begins writing an answer. A general-purpose model offered by a large provider is convenient for getting started. But it is not necessarily the best at understanding Russian, the language of a particular country, specialist vocabulary, or a mixed corpus of instructions, contracts, and code.\n\nFor example, a visitor writes, “When will my purchase arrive?”, while the right help-centre section is called “Delivery times and methods”. Keyword search may miss that connection. Semantic search can find it — but only if the model maps these particular phrasings well enough.\n\nAn embedding is a numerical vector through which a model represents the meaning of a short piece of text. Phrases with similar meanings receive nearby vectors. That lets a question about delivery match a section about timeframes even when they share almost no words.\n\n## Choose the model before the provider\n\nA model has at least four important properties: the languages it was trained and evaluated on; its ability to distinguish similar but different intents; the length of the text it can process; and its vector size and speed. A model trained specifically on texts and queries in the relevant language can map its phrasing, morphology, and shades of meaning more accurately than a generic model from an external provider. The same applies to models for law, medicine, code, or another field: they can make finer distinctions between similar specialist concepts.\n\nThis matters especially for Russian-language and multilingual knowledge bases. You can compare a general external model with a specialised model trained for the relevant language group and see a practical difference: the right section appears in the top results more often, while a similar but wrong page ranks lower. That increases the chance that an answer is grounded in the right source.\n\n## What embedding quality changes\n\nEmbedding quality affects not the wording of an answer, but which evidence reaches its context in the first place. Weak matching produces three typical errors: the needed fragment is not found; a similar but incorrect fragment ranks higher; or search does not understand the user’s language and the organisation’s terminology.\n\nThat is why it is only fair to call one model “better” after testing it on a representative set of questions. Public leaderboard rank is not the only metric: top-result accuracy, the share of answers with valid evidence, latency, and vector size also matter.\n\n## Embedding economics\n\nExternal API prices per 1 million input tokens:\n\n- OpenAI, small model — $0.02;\n- OpenAI, large model — $0.13;\n- Cloudflare,\n`bge-m3`\n\n— $0.012; - Voyage — from $0.02.\n\nFor an estimate, take a 500-token fragment. In the `dzen_embedder`\n\nbenchmark, a\nGTX 1080 processed about 248 fragments per second. Running continuously, that\nis up to 640 million fragments, or about 320 billion tokens per month. This is\na theoretical ceiling: longer fragments and a real mixed workload reduce\nthroughput.\n\nThat monthly volume would cost roughly:\n\n- OpenAI, small model — $6,400;\n- OpenAI, large model — $41,800;\n- Cloudflare,\n`bge-m3`\n\n— $3,900; - Voyage — from $6,400.\n\n**Conclusion.** A server with an entry-level GPU at $100 per month can pay for\nitself under sustained high utilisation: even the theoretical ceiling costs\nthousands of dollars per month through an external API. In practice, calculate\nthis using your own data volume and workload. A local model is also chosen for\nsecurity: documents and embedding requests remain in your infrastructure rather\nthan being sent to an external provider. It also lets you choose a model that\nunderstands the language and terminology of your knowledge base better.\n\n## What [dzen_embedder](https://github.com/dzenplatform/embedder) does\n\n[ dzen_embedder](https://github.com/dzenplatform/embedder) is a standalone\nembedding service for Dzen Chat. At startup, it downloads the selected model\nfrom\n\n[Hugging Face](https://huggingface.co/models)and accepts requests through an OpenAI-compatible\n\n`/v1/embeddings`\n\nendpoint. The\n[MTEB Leaderboard](https://leaderboard.mteb.org/models), a public comparison of embedding models, can help you identify candidates. The web application and background jobs do not each load an additional copy of the model or compete for one GPU’s memory.\n\nThe service has two queues. A visitor’s question goes to the interactive queue and has strict priority. Indexing a large site or reindexing a knowledge base runs in the background queue and uses idle capacity. A batch that has already started is completed, but the next batch does not begin while user questions are waiting.\n\nBelow is a measurement of load from 1 to 64 parallel connections, up to the first error. A red cross marks the first level with an unsuccessful response.\n\n**Findings:** GPU and Apple Silicon completed the mixed workload up to 64\nconnections without errors; the CPU had its first two timeouts at 64\nconnections. A standalone service with one model preserves priority for visitor\nrequests while background indexing waits for idle capacity.\n\nA local service also shortens the network path. A question does not travel to a remote provider and back; when the service runs near the application, the request stays inside the local network. That reduces variable latency and keeps request text out of an external service. Local deployment still requires you to secure your own server, update the model, and monitor the queue.\n\n## How to start using the project\n\nOpen the [dzen_embedder repository](https://github.com/dzenplatform/embedder)\non GitHub and follow the instructions. Or copy this prompt into your agent:\n\n```\nRead and follow the deployment instructions at:\nhttps://raw.githubusercontent.com/dzenplatform/embedder/master/llm-setup.txt\n```\n\nThe agent will request access to the server and deploy the service itself.\n\nThe project is released under the permissive 0BSD licence: you can use, modify, and include it in your own products.\n\nHappy embedding!", "url": "https://wpnews.pro/news/run-local-embedder-for-0-24-of-the-openai-price", "canonical_source": "https://dzen.dev/blog/local-dzen-embedding-for-rag/", "published_at": "2026-08-31 21:47:50+00:00", "updated_at": "2026-08-31 22:22:22.533056+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "artificial-intelligence"], "entities": ["Dzen Platform", "OpenAI", "Cloudflare", "Voyage", "GTX 1080", "dzen_embedder", "bge-m3"], "alternates": {"html": "https://wpnews.pro/news/run-local-embedder-for-0-24-of-the-openai-price", "markdown": "https://wpnews.pro/news/run-local-embedder-for-0-24-of-the-openai-price.md", "text": "https://wpnews.pro/news/run-local-embedder-for-0-24-of-the-openai-price.txt", "jsonld": "https://wpnews.pro/news/run-local-embedder-for-0-24-of-the-openai-price.jsonld"}}