{"slug": "jina-embeddings-v4-as-an-openai-compatible-embeddings-server", "title": "jina-embeddings-v4 as an OpenAI-Compatible Embeddings Server", "summary": "A developer has released jina-embeddings-v4, a self-hosted server for the jina-embeddings-v4 embedding model with an OpenAI-compatible /v1/embeddings endpoint. The server runs on a single NVIDIA GPU and allows applications that call OpenAI for embeddings to switch by only changing the base URL. It supports tasks such as text-matching, retrieval, and code, and returns 2048-dimensional float32 vectors.", "body_md": "[jina-embeddings-v4](https://github.com/Edgaras0x4E/jina-embeddings-v4) is a self-hosted server for the jina-embeddings-v4 embedding model with an OpenAI-compatible `/v1/embeddings`\n\nendpoint. It runs on a single NVIDIA GPU. An application that calls OpenAI for embeddings can call this server instead. The request and response bodies are the same, so setting the client's base URL is the only change needed.\n\n`task`\n\nis `code`\n\n.`float32`\n\nvectorsCreate a `docker-compose.yml`\n\n:\n\n```\nservices:\n  jina:\n    image: edgaras0x4e/jina-embeddings-v4:latest\n    ports:\n      - \"8081:80\"\n    volumes:\n      - jina-cache:/root/.cache/huggingface\n    environment:\n      HF_HOME: /root/.cache/huggingface\n      API_KEY: your-api-key-here\n    deploy:\n      resources:\n        reservations:\n          devices:\n            - driver: nvidia\n              count: 1\n              capabilities: [gpu]\n    restart: unless-stopped\n\nvolumes:\n  jina-cache:\ndocker compose up -d\n```\n\nThe image download is about 7.8 GB, and 14.7 GB unpacked on disk. The prebuilt image needs no build step. Building from source (`git clone`\n\nthe repo, then `docker compose up -d --build`\n\n) compiles `flash-attn`\n\nagainst PyTorch 2.5.1 and CUDA 12.4, which takes 10 to 20 minutes.\n\nThe weights are not in the image: on first start the server downloads about 7 GB from Hugging Face and loads them into GPU memory. On later starts the server reads the weights from the volume instead of downloading them again.\n\nWhen `/health`\n\nreturns `ok`\n\n, the server is ready to embed:\n\n```\ncurl http://localhost:8081/health\n{\"status\":\"ok\"}\ncurl http://localhost:8081/v1/embeddings \\\n  -H \"Authorization: Bearer your-api-key-here\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"input\": [\"The train leaves at eight in the morning\"]}'\n{\n  \"object\": \"list\",\n  \"data\": [\n    {\n      \"object\": \"embedding\",\n      \"embedding\": [-0.008472833782434464, -0.024404730647802353, 0.007389210630208254, ...],\n      \"index\": 0\n    }\n  ],\n  \"model\": \"jinaai/jina-embeddings-v4\",\n  \"usage\": {\"prompt_tokens\": 8, \"total_tokens\": 8}\n}\n```\n\nThe array holds 2048 values. Three are shown.\n\nThe response uses the OpenAI list format. `data`\n\nholds one object per input text, each with an `embedding`\n\narray and its `index`\n\n. `usage.prompt_tokens`\n\ncounts the input tokens. `total_tokens`\n\nequals it, since embedding produces no output tokens.\n\n`input`\n\nalso accepts a list of strings, one request for the whole batch.\n\nThe full request body:\n\n| Field | Required | Description |\n|---|---|---|\n`input` |\nyes | One string or a list of strings |\n`model` |\nno | Echoed back in the response. The server always serves the model set by `MODEL_ID` . |\n`task` |\nno |\n`text-matching` (default), `retrieval` , or `code`\n|\n`prompt_name` |\nno |\n`query` or `passage` . Used only when `task` is `retrieval` , defaults to `passage` . |\n`encoding_format` |\nno | Accepted for OpenAI compatibility and ignored. Vectors are always `float32` arrays. |\n\nThe model produces a different embedding for the same text depending on the value of `task`\n\n. The default, `text-matching`\n\n, is for comparing two texts of the same kind, such as two support tickets or two product descriptions.\n\n`retrieval`\n\nis for search, where a short query is matched against longer documents. Embed the documents with `prompt_name`\n\nset to `passage`\n\nand the query with `prompt_name`\n\nset to `query`\n\n:\n\n```\ncurl http://localhost:8081/v1/embeddings \\\n  -H \"Authorization: Bearer your-api-key-here\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n        \"input\": [\"when does the train leave\"],\n        \"task\": \"retrieval\",\n        \"prompt_name\": \"query\"\n      }'\n```\n\n`code`\n\nis for source code and code search.\n\n``` python\nfrom openai import OpenAI\n\nclient = OpenAI(api_key=\"your-api-key-here\", base_url=\"http://localhost:8081/v1\")\n\nresp = client.embeddings.create(\n    model=\"jinaai/jina-embeddings-v4\",\n    input=[\"how long does the journey take\"],\n)\nprint(len(resp.data[0].embedding))\n2048\n```\n\nThe `task`\n\nand `prompt_name`\n\nfields are not OpenAI parameters, so the SDK passes them through `extra_body`\n\n:\n\n```\nresp = client.embeddings.create(\n    model=\"jinaai/jina-embeddings-v4\",\n    input=[\"SELECT id, name FROM users WHERE active = 1\"],\n    extra_body={\"task\": \"code\"},\n)\n```\n\nIf the server runs without `API_KEY`\n\n, the OpenAI SDK still rejects an empty `api_key`\n\nstring. Pass any non-empty placeholder.\n\nEnvironment variables on the `jina`\n\nservice:\n\n| Variable | Default | Purpose |\n|---|---|---|\n`MODEL_ID` |\n`jinaai/jina-embeddings-v4` |\nHugging Face model id. Override only for a fork or finetune with the same architecture. |\n`HF_HOME` |\n`/root/.cache/huggingface` |\nCache path inside the container. The compose file mounts the `jina-cache` volume there, so a new container reuses the downloaded weights. |\n`API_KEY` |\nunset (optional) | Bearer token for `/v1/embeddings` . If unset, the endpoint accepts requests without a token. |\n\nThe compose file maps host port 8081 to port 80 in the container. If another service already listens on 8081, change the first number in `8081:80`\n\n.", "url": "https://wpnews.pro/news/jina-embeddings-v4-as-an-openai-compatible-embeddings-server", "canonical_source": "https://dev.to/edgaras/jina-embeddings-v4-as-an-openai-compatible-embeddings-server-35j8", "published_at": "2026-08-24 10:30:00+00:00", "updated_at": "2026-08-24 10:43:19.343638+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "developer-tools", "ai-infrastructure"], "entities": ["jina-embeddings-v4", "OpenAI", "Hugging Face", "NVIDIA", "PyTorch", "CUDA", "flash-attn"], "alternates": {"html": "https://wpnews.pro/news/jina-embeddings-v4-as-an-openai-compatible-embeddings-server", "markdown": "https://wpnews.pro/news/jina-embeddings-v4-as-an-openai-compatible-embeddings-server.md", "text": "https://wpnews.pro/news/jina-embeddings-v4-as-an-openai-compatible-embeddings-server.txt", "jsonld": "https://wpnews.pro/news/jina-embeddings-v4-as-an-openai-compatible-embeddings-server.jsonld"}}