Choosing an embedding model determines which documents search finds before a language model begins writing an answer. A general-purpose model offered by a large provider is convenient for getting started. But it is not necessarily the best at understanding Russian, the language of a particular country, specialist vocabulary, or a mixed corpus of instructions, contracts, and code.
For example, a visitor writes, “When will my purchase arrive?”, while the right help-centre section is called “Delivery times and methods”. Keyword search may miss that connection. Semantic search can find it — but only if the model maps these particular phrasings well enough.
An embedding is a numerical vector through which a model represents the meaning of a short piece of text. Phrases with similar meanings receive nearby vectors. That lets a question about delivery match a section about timeframes even when they share almost no words.
Choose the model before the provider #
A model has at least four important properties: the languages it was trained and evaluated on; its ability to distinguish similar but different intents; the length of the text it can process; and its vector size and speed. A model trained specifically on texts and queries in the relevant language can map its phrasing, morphology, and shades of meaning more accurately than a generic model from an external provider. The same applies to models for law, medicine, code, or another field: they can make finer distinctions between similar specialist concepts.
This matters especially for Russian-language and multilingual knowledge bases. You can compare a general external model with a specialised model trained for the relevant language group and see a practical difference: the right section appears in the top results more often, while a similar but wrong page ranks lower. That increases the chance that an answer is grounded in the right source.
What embedding quality changes #
Embedding quality affects not the wording of an answer, but which evidence reaches its context in the first place. Weak matching produces three typical errors: the needed fragment is not found; a similar but incorrect fragment ranks higher; or search does not understand the user’s language and the organisation’s terminology.
That is why it is only fair to call one model “better” after testing it on a representative set of questions. Public leaderboard rank is not the only metric: top-result accuracy, the share of answers with valid evidence, latency, and vector size also matter.
Embedding economics #
External API prices per 1 million input tokens:
- OpenAI, small model — $0.02;
- OpenAI, large model — $0.13;
- Cloudflare,
bge-m3
— $0.012; - Voyage — from $0.02.
For an estimate, take a 500-token fragment. In the dzen_embedder
benchmark, a GTX 1080 processed about 248 fragments per second. Running continuously, that is up to 640 million fragments, or about 320 billion tokens per month. This is a theoretical ceiling: longer fragments and a real mixed workload reduce throughput.
That monthly volume would cost roughly:
- OpenAI, small model — $6,400;
- OpenAI, large model — $41,800;
- Cloudflare,
bge-m3
— $3,900; - Voyage — from $6,400.
Conclusion. A server with an entry-level GPU at $100 per month can pay for itself under sustained high utilisation: even the theoretical ceiling costs thousands of dollars per month through an external API. In practice, calculate this using your own data volume and workload. A local model is also chosen for security: documents and embedding requests remain in your infrastructure rather than being sent to an external provider. It also lets you choose a model that understands the language and terminology of your knowledge base better.
What dzen_embedder does #
dzen_embedder is a standalone embedding service for Dzen Chat. At startup, it downloads the selected model from
Hugging Faceand accepts requests through an OpenAI-compatible
/v1/embeddings
endpoint. The MTEB Leaderboard, a public comparison of embedding models, can help you identify candidates. The web application and background jobs do not each load an additional copy of the model or compete for one GPU’s memory.
The service has two queues. A visitor’s question goes to the interactive queue and has strict priority. Indexing a large site or reindexing a knowledge base runs in the background queue and uses idle capacity. A batch that has already started is completed, but the next batch does not begin while user questions are waiting.
Below is a measurement of load from 1 to 64 parallel connections, up to the first error. A red cross marks the first level with an unsuccessful response.
Findings: GPU and Apple Silicon completed the mixed workload up to 64 connections without errors; the CPU had its first two timeouts at 64 connections. A standalone service with one model preserves priority for visitor requests while background indexing waits for idle capacity.
A local service also shortens the network path. A question does not travel to a remote provider and back; when the service runs near the application, the request stays inside the local network. That reduces variable latency and keeps request text out of an external service. Local deployment still requires you to secure your own server, update the model, and monitor the queue.
How to start using the project #
Open the dzen_embedder repository on GitHub and follow the instructions. Or copy this prompt into your agent:
Read and follow the deployment instructions at:
https://raw.githubusercontent.com/dzenplatform/embedder/master/llm-setup.txt
The agent will request access to the server and deploy the service itself.
The project is released under the permissive 0BSD licence: you can use, modify, and include it in your own products.
Happy embedding!