jina-embeddings-v4 as an OpenAI-Compatible Embeddings Server

jina-embeddings-v4 is a self-hosted server for the jina-embeddings-v4 embedding model with an OpenAI-compatible /v1/embeddings

endpoint. It runs on a single NVIDIA GPU. An application that calls OpenAI for embeddings can call this server instead. The request and response bodies are the same, so setting the client's base URL is the only change needed.

task

is code

.float32

vectorsCreate a docker-compose.yml

:

services:
  jina:
    image: edgaras0x4e/jina-embeddings-v4:latest
    ports:
      - "8081:80"
    volumes:
      - jina-cache:/root/.cache/huggingface
    environment:
      HF_HOME: /root/.cache/huggingface
      API_KEY: your-api-key-here
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    restart: unless-stopped

volumes:
  jina-cache:
docker compose up -d

The image download is about 7.8 GB, and 14.7 GB unpacked on disk. The prebuilt image needs no build step. Building from source (git clone

the repo, then docker compose up -d --build

) compiles flash-attn

against PyTorch 2.5.1 and CUDA 12.4, which takes 10 to 20 minutes.

The weights are not in the image: on first start the server downloads about 7 GB from Hugging Face and loads them into GPU memory. On later starts the server reads the weights from the volume instead of down them again.

When /health

returns ok

, the server is ready to embed:

curl http://localhost:8081/health
{"status":"ok"}
curl http://localhost:8081/v1/embeddings \
  -H "Authorization: Bearer your-api-key-here" \
  -H "Content-Type: application/json" \
  -d '{"input": ["The train leaves at eight in the morning"]}'
{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "embedding": [-0.008472833782434464, -0.024404730647802353, 0.007389210630208254, ...],
      "index": 0
    }
  ],
  "model": "jinaai/jina-embeddings-v4",
  "usage": {"prompt_tokens": 8, "total_tokens": 8}
}

The array holds 2048 values. Three are shown.

The response uses the OpenAI list format. data

holds one object per input text, each with an embedding

array and its index

. usage.prompt_tokens

counts the input tokens. total_tokens

equals it, since embedding produces no output tokens.

input

also accepts a list of strings, one request for the whole batch.

The full request body:

The model produces a different embedding for the same text depending on the value of task

. The default, text-matching

, is for comparing two texts of the same kind, such as two support tickets or two product descriptions.

retrieval

is for search, where a short query is matched against longer documents. Embed the documents with prompt_name

set to passage

and the query with prompt_name

set to query

:

curl http://localhost:8081/v1/embeddings \
  -H "Authorization: Bearer your-api-key-here" \
  -H "Content-Type: application/json" \
  -d '{
        "input": ["when does the train leave"],
        "task": "retrieval",
        "prompt_name": "query"
      }'

code

is for source code and code search.

from openai import OpenAI

client = OpenAI(api_key="your-api-key-here", base_url="http://localhost:8081/v1")

resp = client.embeddings.create(
    model="jinaai/jina-embeddings-v4",
    input=["how long does the journey take"],
)
print(len(resp.data[0].embedding))
2048

The task

and prompt_name

fields are not OpenAI parameters, so the SDK passes them through extra_body

:

resp = client.embeddings.create(
    model="jinaai/jina-embeddings-v4",
    input=["SELECT id, name FROM users WHERE active = 1"],
    extra_body={"task": "code"},
)

If the server runs without API_KEY

, the OpenAI SDK still rejects an empty api_key

string. Pass any non-empty placeholder.

Environment variables on the jina

service:

The compose file maps host port 8081 to port 80 in the container. If another service already listens on 8081, change the first number in 8081:80

.