# Ollama API Key Setup: 12 Steps, 60 Min [2026] > Source: > Published: 2026-11-06 10:24:33+00:00 The local Ollama API at `http://localhost:11434` checks no credentials, so any Ollama API key for a self-hosted server has to come from a layer you build in front of it. The newest stable release, Ollama v0.40.14 according to [a release tracker](https://rwatcher.yadoc.fr/components/3efe10e0-9fd5-436f-a6f5-361b8dec5119/cc002b73-46f9-42db-9602-332441e28976), keeps that rule. Local requests need no key. The hosted API at ollama.com does. That gap trips up a lot of people. Agent frameworks such as the one in our [OpenHands local setup with Ollama](https://shattered.io/openhands-local-setup-agent-canvas-ollama-12-steps-2026/) expect an OpenAI-style endpoint plus a key field, and they send whatever string you hand them. If the server never validates it, that field is decoration. Anyone who can reach the port can run prompts, list models, pull new ones and delete the ones you already downloaded. The fix is a small gateway. Ollama runs in a Docker container with no published port. An Nginx container sits in front, terminates TLS, checks a Bearer token against a list of named keys, throttles each caller, and refuses model-management calls unless the key carries admin rights. Clients keep using the standard OpenAI SDK, because the SDK already sends `Authorization: Bearer` on every request. Twelve steps and roughly an hour of work get you there, and the whole thing fits in one folder you can commit (minus the secrets). Don't miss new tech stories on Google Add Shattered once in the Google app and our stories show up in your news suggestions. [Add now](https://www.google.com/preferences/source?hl=en&q=https%3A%2F%2Fshattered.io%2F) ## Why the Ollama API Needs a Key You Have to Build Ollama binds to `127.0.0.1:11434` by default, which is a sensible default for a desktop tool. Trouble starts when someone sets `OLLAMA_HOST=0.0.0.0` on a shared machine, publishes the port in a Docker command, or forwards it through a router so a laptop or phone can reach the model. At that point the server speaks to anyone, because there is no login screen, no token check and no per-user accounting in the local API. The exposure covers more than inference. The native API includes endpoints that pull, push, create, copy and delete models. An unauthenticated caller on your network can fill the disk by pulling large models, delete the model your team relies on, or keep the GPU busy with long generations while your own jobs wait. None of that needs an exploit. It only needs the port to be reachable. Ollama serves a large audience. The company said in its August 31, 2026 pricing announcement that it serves 8.9 million developers, and plenty of them will eventually flip a server from loopback to a shared network. A self-hosted deployment needs three controls the local server does not provide: identity (who is calling), authorization (what that caller may do) and rate control (how much). The gateway in this tutorial supplies all three with plain configuration and no custom server code. Two design choices keep the setup small. First, keys live in a flat map file that Nginx reads, so adding or revoking a caller is one line and one reload. Second, the OpenAI-compatible `/v1` path and the native `/api` path share the same key, so a single credential works for both the SDK and tools like curl or Postman. A gateway is not a force field, so set expectations before building one. It does not stop a legitimate key holder from sending a malicious prompt, and it does not make model output trustworthy. It does not protect the host if someone already has shell access, and it cannot hide a token that a client leaks. A stolen token works from anywhere until you revoke it, which is why per-caller names and logs matter as much as the check itself. What it does give you is a narrow, auditable front door instead of an open port, and for a self-hosted model that is the largest single improvement available. ## What Ollama Authenticates: Local API vs Cloud API Ollama now exposes two different API surfaces, and mixing them up is the most common source of invalid-key confusion. The local server answers on port 11434 and ignores credentials. The hosted service at ollama.com does require a key. Ollama’s documentation, which was updated again on November 4, 2026 per [a changelog digest](https://techgazette.com.br/company/ollama/), spells out the split, and the table below condenses it. | Property | Local server | Hosted API (ollama.com) | |---|---|---| | Base URL (native) | `http://localhost:11434/api` | `https://ollama.com/api` | | Base URL (OpenAI-style) | `http://localhost:11434/v1` | `https://ollama.com/v1` | | Credential | None required | API key sent as `Authorization: Bearer` | | Install needed | Yes, the Ollama app or container | No, per Ollama’s API docs | | Key management | You build it (this tutorial) | Ollama Keys page in your account | | Key lifetime | Whatever you configure | Non-expiring, revocable from settings | | Env var used by clients | Usually none | `OLLAMA_API_KEY` | A third credential exists and deserves a mention so it does not mislead you. Ollama’s docs also describe an Ollama public key that lets a local installation authenticate with ollama.com to push models, pull private models and run cloud-hosted models. That key identifies your machine to Ollama’s service. It does nothing to protect the port on your own network, and a reverse proxy does not create or replace it. One practical consequence follows. When you route cloud models through a local server by adding `:cloud` to the model name, the server needs Ollama v0.40.5 or later and a signed-in account. Your gateway then protects that route too, which matters because the signed-in server spends your cloud usage on behalf of every caller. An open local port with a signed-in account can burn paid credits, not just GPU time. ## Prerequisites and Versions Everything below ran against Ollama v0.40.14, the stable release at the time of writing. A v0.40.15 build also appears in release listings, but it is labeled a pre-release, so skip it for anything you plan to leave running. If you read an older tutorial that calls v0.40.13 or earlier the latest version, treat its version-specific advice with caution. | Component | Version or tag | Role in the project | |---|---|---| | Ollama container | `ollama/ollama` , stable v0.40.14 or newer | Model server, never published to the host | | Docker Engine | Current release with the Compose v2 plugin ( `docker compose` ) | Runs both containers | | Nginx container | `nginx:stable-alpine` | TLS, key check, rate limit, logging | | OpenSSL | 1.1.1 or newer (needed for `-addext` ) | Self-signed certificate and key generation | | Python | 3.10 or newer | Client script and test suite | | Python packages | Latest `openai` and`httpx` | SDK client, raw HTTP tests | | NVIDIA Container Toolkit | Optional, current release | GPU access inside Docker on Linux | | Test model | `gpt-oss:20b` , or a smaller library model | Anything that fits your memory works | For the model, `gpt-oss:20b` is the default in the scripts because it is listed in the Ollama library and answers through the chat API. If it does not fit your hardware, set the `MODEL` environment variable to something smaller, such as `gemma4`. NVIDIA’s Nemotron 3.5 Bolt E4B, which [one news report](https://news.yushang.ai/en/2026/10/ollama-adds-nemotron-3-5-bolt-e4b-to-e1e567) says Ollama added support for in October 2026, is another small option. Model files are the largest download in the whole project, so pick one that leaves headroom for context rather than one that barely loads. You also need a machine you control, either a Linux box, a Mac with Docker Desktop, or a Windows machine with WSL2. GPU passthrough works on Linux with the NVIDIA toolkit, while Docker Desktop on Mac cannot hand the GPU to containers. Mac users get better speed running the Ollama app natively and pointing the gateway at `host.docker.internal:11434`, which the troubleshooting section covers. Disk space deserves a thought before you pull anything. Model files are large, the Docker volume holds every model you ever pulled, and an admin key that pulls without looking can fill the drive. Check free space with `df -h` on the Docker data directory, and plan to prune models you no longer use with `ollama rm`. Keeping this housekeeping behind an admin key is part of why the gateway separates normal callers from model managers. ## Phase 1: Project Folder and Ollama in Docker (Steps 1-3) **Step 1: Create the folder layout and check your tools.** Every file lives under one directory, so you can zip it, commit it or move it to another machine. Keys and certificates get their own folders because you will keep those out of version control. ``` mkdir -p ollama-gateway/{nginx,certs,keys,scripts,tests} cd ollama-gateway touch nginx/keys-user.map nginx/keys-admin.map docker --version docker compose version openssl version python3 --version ``` Expected output on a typical Linux box looks like the block below. Your exact versions will differ. What matters is that [docker compose](https://shattered.io/docker-compose-tutorial-production-stack-12-steps-2026/) ``` Docker version 27.x.x, build xxxxxxx Docker Compose version v2.x.x OpenSSL 3.x.x (or 1.1.1 and up) Python 3.1x.x ``` The two empty `.map` files matter. Nginx refuses to start if an `include` points at a file that does not exist, so creating them now saves you a confusing error in Step 8. **Step 2: Write the Compose file.** The key detail is what is missing. The `ollama` service has no `ports` entry, so nothing on the host or the LAN can reach it directly. Only the `gateway` service publishes a port, and only on 8443. The Ollama server inside the container listens on all container interfaces so the gateway can reach it over the private Compose network. ``` # docker-compose.yml services: ollama: image: ollama/ollama:${OLLAMA_TAG:-latest} restart: unless-stopped environment: OLLAMA_HOST: 0.0.0.0:11434 OLLAMA_KEEP_ALIVE: 10m volumes: - ollama_models:/root/.ollama healthcheck: test: ["CMD", "ollama", "list"] interval: 10s timeout: 5s retries: 12 # Uncomment for NVIDIA GPUs (needs the NVIDIA Container Toolkit) # deploy: # resources: # reservations: # devices: # - driver: nvidia # count: all # capabilities: [gpu] gateway: image: nginx:stable-alpine restart: unless-stopped depends_on: ollama: condition: service_healthy ports: - "8443:8443" volumes: - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro - ./nginx/proxy.conf:/etc/nginx/proxy.conf:ro - ./nginx/keys-user.map:/etc/nginx/keys-user.map:ro - ./nginx/keys-admin.map:/etc/nginx/keys-admin.map:ro - ./certs:/etc/nginx/certs:ro - gateway_logs:/var/log/nginx volumes: ollama_models: gateway_logs: ``` The `OLLAMA_TAG` variable lets you pin the image later. Run with the default `latest` tag first, confirm the version in Step 3, then set `OLLAMA_TAG` in a `.env` file so an unattended `docker compose pull` cannot move you to a new release without you noticing. Binding to `0.0.0.0` inside the container is safe here only because no `ports` mapping exposes it. Copy that line to a bare-metal install and you have rebuilt the problem this tutorial exists to solve. **Step 3: Start Ollama alone, check the version and pull a model.** Bring up only the model server first. If something is wrong with the GPU or the volume, you want to see it before the gateway adds another layer. ``` docker compose up -d ollama docker compose ps docker compose exec ollama ollama --version docker compose exec ollama ollama pull gpt-oss:20b docker compose exec ollama ollama list NAME IMAGE SERVICE STATUS PORTS ollama-gateway-ollama-1 ollama/ollama:latest ollama Up 40 seconds (healthy) 11434/tcp ollama version is 0.40.14 NAME ID SIZE MODIFIED gpt-oss:20b ... ... a few seconds ago ``` The `11434/tcp` under PORTS has no arrow and no host address. That is Docker telling you the port is exposed to other containers only. Now prove the port is closed from the outside. This is the check people skip, and it is the one that tells you the architecture works. ``` curl -sS --max-time 3 http://localhost:11434/api/version || echo "closed, as intended" ``` You want a connection error followed by the echo. If you see a JSON version string instead, something else on the host is already serving 11434, most often a native Ollama install started by systemd or the desktop app. That is a separate process from the container, and Step 9 explains why it can make test results confusing. ## Phase 2: API Keys and TLS Certificate (Steps 4-5) **Step 4: Write the key generator.** Each caller gets its own random token, stored in a file only you can read. Nginx gets the token and a human-readable name through a map file. Per-caller names are what make the later logs and rate limits useful, because a log line that names Alice as the throttled caller is actionable and one that names nobody is not. ``` bash #!/usr/bin/env bash # scripts/make-key.sh set -euo pipefail cd "$(dirname "$0")/.." name="${1:?usage: scripts/make-key.sh NAME [--admin]}" if [[ ! "$name" =~ ^[a-z0-9_-]+$ ]]; then echo "name must match [a-z0-9_-]+" >&2; exit 1 fi if grep -q " ${name};\$" nginx/keys-user.map; then echo "key for ${name} already exists" >&2; exit 1 fi token="olk_$(openssl rand -hex 32)" umask 077 printf '%s' "$token" > "keys/${name}.txt" printf '"Bearer %s" %s;\n' "$token" "$name" >> nginx/keys-user.map if [ "${2:-}" = "--admin" ]; then printf '"Bearer %s" %s;\n' "$token" "$name" >> nginx/keys-admin.map fi if docker compose ps --status running gateway 2>/dev/null | grep -q gateway; then docker compose exec -T gateway nginx -t && docker compose exec -T gateway nginx -s reload fi echo "created keys/${name}.txt" ``` Three details are worth understanding. The `olk_` prefix is cosmetic, but it makes leaked tokens easy to find with a secret scanner and easy to spot in a pasted log. The 64 hex characters come from 32 random bytes, which is far beyond what anyone can guess. And the script appends with `>>` rather than rewriting the file, which keeps the same inode so the bind mount stays valid. Editors that save by replacing the file can break that link, a pitfall covered later. Create one admin key for you and one normal key for the first client. Admin keys are written to both map files, so an admin can also use the regular endpoints. ``` chmod +x scripts/make-key.sh scripts/make-key.sh admin --admin scripts/make-key.sh alice ls -l keys/ created keys/admin.txt created keys/alice.txt -rw------- 1 you you 68 Nov 6 10:20 admin.txt -rw------- 1 you you 68 Nov 6 10:20 alice.txt ``` The 68-byte size is the 4-character prefix plus 64 hex characters, with no trailing newline. That detail saves you from the classic copy-paste bug where a newline rides along inside the header. Permissions of `-rw-------` confirm the `umask 077` did its job. **Step 5: Generate the TLS certificate.** Bearer tokens travel in a header on every request, so sending them over plain HTTP hands the key to anyone on the path. A self-signed certificate is fine for a home lab or an office LAN as long as clients trust that one file. The `subjectAltName` entry is not optional, because modern clients ignore the common name and reject certificates that lack it. ``` openssl req -x509 -newkey rsa:2048 -nodes -days 365 \ -keyout certs/gateway.key -out certs/gateway.crt \ -subj "/CN=localhost" \ -addext "subjectAltName=DNS:localhost,IP:127.0.0.1" chmod 600 certs/gateway.key openssl x509 -in certs/gateway.crt -noout -subject -enddate -ext subjectAltName subject=CN = localhost notAfter=Nov 6 10:21:03 2027 GMT X509v3 Subject Alternative Name: DNS:localhost, IP Address:127.0.0.1 ``` If other machines will call the gateway, add their DNS name or LAN address to the `subjectAltName` list, for example `DNS:llm.home.lan,IP:192.168.1.50`, and regenerate. For a public hostname, skip the self-signed route and use a real certificate from Let’s Encrypt, which the advanced section covers with Caddy. ## Phase 3: The Nginx Gateway Config (Steps 6-7) **Step 6: Write nginx.conf.** This file does the real work, so it gets the longest explanation. [Nginx](https://nginx.org/) reads the incoming `Authorization` header, looks the exact string up in a map, and sets `$client_name` to the caller’s name or to an empty string. An empty name means no valid key, and the server answers 401 before any proxying happens. A second map does the same for admin rights. ``` # nginx/nginx.conf worker_processes auto; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/json; server_tokens off; # Identity: exact match on the full Authorization header value. map $http_authorization $client_name { default ""; include /etc/nginx/keys-user.map; } map $http_authorization $admin_name { default ""; include /etc/nginx/keys-admin.map; } # Structured access log. The Authorization header is never logged. log_format gw escape=json '{"t":"$time_iso8601","client":"$client_name","method":"$request_method",' '"uri":"$uri","status":$status,"bytes":$body_bytes_sent,' '"rt":$request_time,"ip":"$remote_addr"}'; access_log /var/log/nginx/access.log gw; # Rate limit per named caller: 30 requests per minute, bursts of 10. limit_req_zone $client_name zone=perclient:10m rate=30r/m; limit_req_status 429; upstream ollama_backend { server ollama:11434; keepalive 8; } server { listen 8443 ssl; server_name _; ssl_certificate /etc/nginx/certs/gateway.crt; ssl_certificate_key /etc/nginx/certs/gateway.key; ssl_protocols TLSv1.2 TLSv1.3; client_max_body_size 20m; # No valid key, no service. Runs before any location logic. if ($client_name = "") { add_header WWW-Authenticate 'Bearer realm="ollama"' always; return 401 '{"error":"missing or invalid API key"}'; } # Model management needs an admin key. location ~ ^/api/(pull|push|create|copy|delete|blobs) { if ($admin_name = "") { return 403 '{"error":"admin key required"}'; } include /etc/nginx/proxy.conf; proxy_pass http://ollama_backend; } # Everything else: inference and read-only calls. location / { limit_req zone=perclient burst=10 nodelay; include /etc/nginx/proxy.conf; proxy_pass http://ollama_backend; } } } ``` Walk through the decisions, because each one has a failure mode behind it. The `map` lookup is an exact string comparison, so `Bearer` must be capitalized as written and there must be exactly one space. The OpenAI SDK and curl examples below both send it that way. Using a map instead of a chain of `if` statements keeps Nginx’s configuration cost flat even with hundreds of keys, since map lookups use a hash table. The rate limit zone keys on `$client_name`, which means every caller gets an independent bucket. At 30 requests per minute with a burst of 10, one caller can send eleven requests instantly and then roughly one every two seconds. Chat completions are heavy, so that pace is generous for a single person and tight enough to stop a runaway script. Admin routes skip the limit on purpose, since pulls are rare and a throttled pull can corrupt your patience more than your disk. The 401 check runs at server level, in Nginx’s rewrite phase, which executes before `limit_req` and before proxying. Unauthenticated floods therefore never consume rate-limit memory or reach Ollama. If you have run a proxy in front of a long-lived service before, such as the SRI proxy in our [Stratum V2 mining setup](https://shattered.io/stratum-v2-mining-setup-2026/), the pattern is the same: terminate and authenticate at the edge, then forward plain traffic over a private network. ### How one request travels through the gateway 1. The client opens a TLS connection to port 8443 and sends the request with an `Authorization: Bearer` header. 2. Nginx looks up the header value in the user map. No match means an immediate 401 and the request ends there. 3. For paths such as `/api/pull` , Nginx also checks the admin map. No match means a 403. 4. For everything else, the per-caller rate limit runs. A caller over budget gets a 429. 5. Nginx removes the Authorization header, sets the Host header to `localhost:11434` and forwards the request over the private Compose network. 6. Ollama streams its reply back, Nginx passes each chunk through unbuffered, and the access log records the caller name, route, status and duration. That order is deliberate. Cheap checks run first, so the expensive work of loading a model or generating tokens only starts for callers who passed identity, authorization and rate control. **Step 7: Write proxy.conf and validate the whole config.** The shared proxy settings sit in their own file so both locations stay identical. Four of these lines exist specifically because LLM traffic is unlike ordinary web traffic. ``` # nginx/proxy.conf proxy_http_version 1.1; proxy_set_header Connection ""; proxy_set_header Host localhost:11434; proxy_set_header Authorization ""; proxy_set_header X-Real-IP $remote_addr; # Stream tokens as they are generated instead of buffering the reply. proxy_buffering off; proxy_request_buffering off; # A cold model load can take minutes. Do not cut it off. proxy_connect_timeout 10s; proxy_send_timeout 600s; proxy_read_timeout 600s; ``` - `proxy_set_header Host localhost:11434` follows Ollama’s FAQ guidance for Nginx, which says to forward the Host header as`localhost:11434` . Without it, some requests are rejected because the server sees an unexpected host name. - `proxy_set_header Authorization ""` strips the gateway token before the request reaches Ollama, so the key never lands in the model server’s logs. - `proxy_buffering off` lets streamed tokens flow through immediately. With buffering on, a streamed answer arrives as one lump at the end. - The 600-second timeouts cover the first request after a model unloads. The default 60 seconds is too short for a large model that must load into memory. Check the syntax before starting anything. The test container below mounts your files read-only and runs `nginx -t`, which parses the config and exits. ``` docker run --rm \ -v "$PWD/nginx/nginx.conf:/etc/nginx/nginx.conf:ro" \ -v "$PWD/nginx/proxy.conf:/etc/nginx/proxy.conf:ro" \ -v "$PWD/nginx/keys-user.map:/etc/nginx/keys-user.map:ro" \ -v "$PWD/nginx/keys-admin.map:/etc/nginx/keys-admin.map:ro" \ -v "$PWD/certs:/etc/nginx/certs:ro" \ --add-host ollama:127.0.0.1 \ nginx:stable-alpine nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful ``` The `--add-host` flag is a trick for this one-off test. The config names an upstream host called `ollama`, and the standalone container cannot resolve it. Pointing the name at loopback lets the syntax check finish without the full stack running. ## Phase 4: Start the Gateway and Call It With curl (Steps 8-9) **Step 8: Bring up the full stack and read the first log lines.** The gateway waits for Ollama’s health check, so a cold start takes a few seconds longer than the model server alone. ``` docker compose up -d docker compose ps docker compose logs --tail=5 gateway NAME IMAGE SERVICE STATUS PORTS ollama-gateway-gateway-1 nginx:stable-alpine gateway Up 6 seconds 0.0.0.0:8443->8443/tcp ollama-gateway-ollama-1 ollama/ollama:latest ollama Up 2 minutes (healthy) 11434/tcp ``` Only 8443 shows a host mapping. If the gateway exits immediately, run `docker compose logs gateway` and look for the first error. A missing map file, a certificate path typo and an unresolvable upstream cause nearly all startup failures, and the troubleshooting section lists the exact messages. **Step 9: Test with curl, first without a key and then with one.** Always point tests at port 8443. If you already run the native Ollama app on the same machine, it answers on 11434 and has no gateway in front of it, so a test against that port proves nothing about the stack you just built. Use `--cacert` so curl trusts your self-signed certificate instead of failing or, worse, being told to ignore verification with `-k`. ``` export GW=https://localhost:8443 export KEY=$(cat keys/alice.txt) # 1. No key: expect 401 curl -sS --cacert certs/gateway.crt -i $GW/api/version | head -n 12 # 2. Valid key: expect 200 and a version string curl -sS --cacert certs/gateway.crt -H "Authorization: Bearer $KEY" $GW/api/version # 3. Valid key, OpenAI-compatible chat curl -sS --cacert certs/gateway.crt $GW/v1/chat/completions \ -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \ -d '{"model":"gpt-oss:20b","messages":[{"role":"user","content":"Reply with the single word ready."}]}' HTTP/1.1 401 Unauthorized Server: nginx Content-Type: application/json WWW-Authenticate: Bearer realm="ollama" {"error":"missing or invalid API key"} {"version":"0.40.14"} {"id":"chatcmpl-...","object":"chat.completion","model":"gpt-oss:20b", "choices":[{"index":0,"message":{"role":"assistant","content":"ready"},"finish_reason":"stop"}], ...} ``` The first request on a cold model can take a while because Ollama loads the weights into memory. That delay is why the gateway uses long proxy timeouts, and the `OLLAMA_KEEP_ALIVE` setting of 10 minutes in the Compose file keeps the model resident between requests so later calls return quickly. Note that the version string in your output should match the image you pulled, which is a handy way to confirm you are hitting the container and not a stray native install. ## Phase 5: Python Clients and a Streaming Check (Steps 10-11) **Step 10: Call the gateway with the official OpenAI Python SDK.** This is the payoff for choosing Bearer tokens. The SDK takes a `base_url` and an `api_key`, and it puts the key in the right header on every call. The only extra is an `httpx` client that trusts your certificate. The same two settings work in editors and agents, for instance the custom server fields in our [Kilo Code local setup](https://shattered.io/kilo-code-local-setup-ollama-vs-code-2026/). ``` python3 -m venv .venv source .venv/bin/activate pip install --upgrade openai httpx python #!/usr/bin/env python3 # scripts/chat.py import os import sys from pathlib import Path import httpx from openai import OpenAI ROOT = Path(__file__).resolve().parent.parent BASE = os.getenv("GATEWAY_URL", "https://localhost:8443") MODEL = os.getenv("MODEL", "gpt-oss:20b") TOKEN = (ROOT / "keys" / f"{os.getenv('KEY_NAME', 'alice')}.txt").read_text().strip() client = OpenAI( base_url=f"{BASE}/v1", api_key=TOKEN, http_client=httpx.Client(verify=str(ROOT / "certs" / "gateway.crt"), timeout=600), ) prompt = " ".join(sys.argv[1:]) or "In two sentences, why should a self-hosted LLM API require a key?" stream = client.chat.completions.create( model=MODEL, messages=[{"role": "user", "content": prompt}], stream=True, ) for chunk in stream: if chunk.choices and chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="", flush=True) print() python scripts/chat.py "Name three risks of an open Ollama port." An open Ollama port lets strangers run prompts on your hardware, pull or delete models without asking, and keep the GPU busy while your own jobs wait. ``` Many other tools skip code entirely and read two environment variables, `OPENAI_BASE_URL` and `OPENAI_API_KEY`. Export your gateway URL with the `/v1` suffix and the contents of a key file, and any tool built on the OpenAI SDK will talk to your server. Tools that wrap other runtimes usually have a custom endpoint field instead. Either way, give every tool its own key from `make-key.sh`, so the log shows which one is busy and you can revoke it alone. Reasoning models such as `gpt-oss` may pause before the first visible token while they think. That pause is the model working, not the gateway stalling. The next script separates the two by measuring the gap before the first chunk and the generation speed Ollama reports for itself. **Step 11: Verify that streaming survives the proxy.** A misconfigured proxy is the most common reason a streaming API feels frozen, because buffering collects the whole answer and releases it at once. This script calls the native `/api/chat` endpoint, which streams one JSON object per line, and prints the time to the first chunk, the chunk count and the tokens per second computed from the `eval_count` and `eval_duration` fields in the final object. ``` python #!/usr/bin/env python3 # scripts/stream_check.py import json import os import time from pathlib import Path import httpx ROOT = Path(__file__).resolve().parent.parent BASE = os.getenv("GATEWAY_URL", "https://localhost:8443") MODEL = os.getenv("MODEL", "gpt-oss:20b") TOKEN = (ROOT / "keys" / f"{os.getenv('KEY_NAME', 'alice')}.txt").read_text().strip() payload = { "model": MODEL, "messages": [{"role": "user", "content": "Count from 1 to 40, one number per line."}], "stream": True, } start = time.perf_counter() first = None chunks = 0 final = {} with httpx.Client(base_url=BASE, verify=str(ROOT / "certs" / "gateway.crt"), headers={"Authorization": f"Bearer {TOKEN}"}, timeout=600) as c: with c.stream("POST", "/api/chat", json=payload) as r: r.raise_for_status() for line in r.iter_lines(): if not line: continue msg = json.loads(line) chunks += 1 body = msg.get("message", {}) if first is None and (body.get("content") or body.get("thinking")): first = time.perf_counter() - start if msg.get("done"): final = msg total = time.perf_counter() - start tokens = final.get("eval_count", 0) gen_s = final.get("eval_duration", 0) / 1e9 print(f"first chunk after : {first:.2f} s" if first else "no content received") print(f"total time : {total:.2f} s") print(f"stream chunks : {chunks}") if tokens and gen_s: print(f"generation speed : {tokens / gen_s:.1f} tokens/s ({tokens} tokens)") if first and chunks > 3 and first > 0.8 * total: print("WARNING: first chunk arrived near the end. Check proxy_buffering.") first chunk after : 1.84 s total time : 3.97 s stream chunks : 118 generation speed : 52.3 tokens/s (112 tokens) ``` Treat those numbers as an example. Your hardware, model and quantization decide the real values. The signal to look for is the shape: many chunks, a first chunk well before the total time, and no warning. If you see two or three chunks and a first-chunk time close to the total, buffering is on somewhere in the path, and the pitfalls section tells you where to look. ## Phase 6: Automated Tests for Auth, Limits and Streaming (Step 12) **Step 12: Lock the behavior in with a smoke test.** Manual curl checks prove the stack works once. A script proves it keeps working after you edit a map file, upgrade the image or tweak a timeout. The suite below checks nine behaviors, exits non-zero on any failure, and is short enough to run from a cron job or a CI step after every change. ``` python #!/usr/bin/env python3 # tests/smoke_test.py import json import os import sys from pathlib import Path import httpx ROOT = Path(__file__).resolve().parent.parent BASE = os.getenv("GATEWAY_URL", "https://localhost:8443") CERT = str(ROOT / "certs" / "gateway.crt") MODEL = os.getenv("MODEL", "gpt-oss:20b") def key(name): return (ROOT / "keys" / f"{name}.txt").read_text().strip() def client(token=None): headers = {"Authorization": f"Bearer {token}"} if token else {} return httpx.Client(base_url=BASE, verify=CERT, headers=headers, timeout=300) results = [] def check(name, ok, detail=""): results.append(ok) print(f"{'PASS' if ok else 'FAIL'} {name} {detail}") # 1. Plain HTTP on the TLS port is refused try: r = httpx.get(BASE.replace("https://", "http://") + "/", timeout=10) check("plain HTTP is refused", r.status_code == 400, f"got {r.status_code}") except httpx.HTTPError as exc: check("plain HTTP is refused", True, f"connection error: {type(exc).__name__}") # 2-3. Missing and wrong keys with client() as c: r = c.get("/api/version") check("no key is rejected", r.status_code == 401, f"got {r.status_code}") with client("olk_" + "0" * 64) as c: r = c.get("/api/version") check("wrong key is rejected", r.status_code == 401, f"got {r.status_code}") # 4-6. Normal user with client(key("alice")) as c: r = c.get("/api/version") check("user key reads version", r.status_code == 200, r.text.strip()) r = c.post("/api/pull", json={"model": MODEL, "stream": False}) check("user key cannot pull", r.status_code == 403, f"got {r.status_code}") r = c.post("/v1/chat/completions", json={ "model": MODEL, "messages": [{"role": "user", "content": "Reply with ok."}], }) check("user key can chat via /v1", r.status_code == 200, f"got {r.status_code}") # 7. Admin gate with client(key("admin")) as c: r = c.post("/api/pull", json={"model": MODEL, "stream": False}) check("admin key passes the admin gate", r.status_code == 200, f"got {r.status_code}") # 8. Streaming produces several chunks with client(key("alice")) as c: payload = {"model": MODEL, "stream": True, "messages": [{"role": "user", "content": "Count from 1 to 20."}]} n = 0 with c.stream("POST", "/api/chat", json=payload) as r: for line in r.iter_lines(): if line and json.loads(line): n += 1 check("native chat streams in several chunks", n > 3, f"chunks={n}") # 9. Rate limit trips under a burst (keep this check last) with client(key("alice")) as c: codes = [c.get("/api/version").status_code for _ in range(25)] limited = codes.count(429) check("burst of 25 requests hits the rate limit", limited >= 1, f"429s={limited}") failed = results.count(False) print(f"\n{len(results) - failed}/{len(results)} checks passed") sys.exit(1 if failed else 0) python tests/smoke_test.py PASS plain HTTP is refused got 400 PASS no key is rejected got 401 PASS wrong key is rejected got 401 PASS user key reads version {"version":"0.40.14"} PASS user key cannot pull got 403 PASS user key can chat via /v1 got 200 PASS admin key passes the admin gate got 200 PASS native chat streams in several chunks chunks=61 PASS burst of 25 requests hits the rate limit 429s=13 9/9 checks passed ``` The rate-limit check stays last for a reason. Once Alice’s bucket drains, her requests return 429 for a few seconds, which would fail any test that ran after it. The expected status for every combination of caller and endpoint is summarized below, which doubles as a checklist when you add routes. | Request | No key | User key | Admin key | |---|---|---|---| | `GET /api/version` | 401 | 200 | 200 | | `GET /api/tags` | 401 | 200 | 200 | | `POST /v1/chat/completions` | 401 | 200 | 200 | | `POST /api/chat` | 401 | 200 | 200 | | `POST /api/pull` | 401 | 403 | 200 | | `POST /api/delete` | 401 | 403 | 200 | | Plain HTTP to port 8443 | 400 | 400 | 400 | The admin pull in the test returns quickly because the model is already on disk, so the call completes without downloading anything. Do not add a delete test that targets a real model. If you want to cover that route, point it at a throwaway model name and accept a not-found response as proof the gate opened. Tune the rate limit to the workload instead of copying the numbers from Step 6. An interactive chat user rarely sends more than a few requests a minute, so 30 per minute leaves plenty of room. An agent that fires tool calls in a loop can burn through that in seconds, and a 429 mid-task looks like a flaky model. For agents, raise the burst, keep the sustained rate modest, and give the agent its own key so its bucket cannot starve a human. For batch jobs that embed thousands of documents, a separate location with its own zone keeps that traffic from eating the interactive budget. Concurrency is a different limit from request rate. Ollama decides how many requests it processes at once, and extra requests queue inside the server. A caller who stays under the rate limit can still hold the GPU for a long time with one huge generation. If that becomes a problem, add a `limit_conn` zone keyed on the caller name to cap simultaneous connections per key. ## Using Ollama Cloud API Keys Without Mixing Them Up Plenty of readers searching for an Ollama API key actually want the hosted one. The flow is separate from everything above. Open the Ollama Keys page in your account settings, create a key, and send it as a Bearer token to `https://ollama.com/api` or the OpenAI-style `https://ollama.com/v1`. According to Ollama’s documentation, these keys do not expire but can be revoked from the same settings page. Direct cloud calls need no local install. ``` export OLLAMA_API_KEY="paste-your-key-here" curl https://ollama.com/api/chat \ -H "Authorization: Bearer $OLLAMA_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model":"gpt-oss:120b","stream":false, "messages":[{"role":"user","content":"Reply with the single word ready."}]}' ``` Check the cloud model list for the exact model names your plan can use. Ollama moved its Pro, Max and Team plans to per-token pricing on August 31, 2026, and the Team plan starts at $500 per month with $1,000 of usage credits included, per the company’s pricing page. Hosted usage costs money, so guard the key like a payment credential. Ollama’s quickstart guidance on hosted keys tells developers to keep them on the application’s server, away from browser code and source control. The same rule applies to the gateway tokens you created in Step 4. A token embedded in a web page or a mobile app is public the moment someone opens the developer tools. ### When to use which key - **Gateway token (this tutorial):** protects your own server and your own GPU. You issue, rotate and revoke it. - **Ollama API key:** authenticates you to the hosted service and its cloud models. Ollama issues and bills it. - **Ollama public key:** links a local installation to your Ollama account for pushing models and running cloud models. It never appears in an HTTP header you write by hand. If you run cloud models through your local server with the `:cloud` suffix, the gateway is what stops a stranger from spending your hosted quota. The gateway key identifies the caller to you. The signed-in account then pays Ollama for the usage. ## Common Pitfalls That Break Ollama Gateways Most failures in this kind of setup repeat. These eight account for the bulk of the reports that start with it worked yesterday. 1. **Publishing 11434 anyway.** A`ports: "11434:11434"` line or a`-p 11434:11434` flag puts the unauthenticated server back on the network and bypasses your gateway. Docker also writes its own firewall rules that can sidestep host tools like ufw, so a rule that looks like it blocks the port may not. 2. **Setting OLLAMA_HOST to 0.0.0.0 on a host install.** People do it to reach the model from a phone, forget about it, and leave every model-management endpoint open. Keep the host install on loopback and put the gateway in front. 3. **Routing pulls through HTTP_PROXY.** Ollama’s documentation says model downloads use HTTPS, so set`HTTPS_PROXY` for outbound pulls and avoid`HTTP_PROXY` , which can interrupt client connections. 4. **Committing keys or certificates.** Add`keys/` ,`certs/` and both`.map` files to`.gitignore` before the first commit. Rotating a leaked token is easy, but scrubbing it from Git history is not. 5. **Editing a map file the wrong way.** Some editors and`sed -i` replace the file with a new inode, and a single-file bind mount keeps pointing at the old one. The gateway then ignores your change even after a reload. Append with`>>` or restart the container. 6. **Forgetting to reload.** Nginx reads the map files at startup and on reload. A new key does nothing until`nginx -s reload` runs, which the key script does for you when the gateway is up. 7. **Logging the Authorization header.** Adding`$http_authorization` to the log format writes every token to disk. The format in Step 6 logs the caller name instead, which is all you need. 8. **Wrong Host header.** Ollama’s Nginx guidance says to forward`Host: localhost:11434` . Passing the public host name through can make the server reject requests that look fine otherwise. ## Troubleshooting: 12 Errors and Fixes ### Authentication and TLS errors - **401 even though the key is correct.** The map does an exact string match. Check for a trailing newline in the key file, a lowercase`bearer` , or a double space. Compare with`cat -A keys/alice.txt` , which should end without a`$` line break marker. - **A newly created key returns 401.** Nginx has not reloaded, or the bind mount points at a stale inode. Run`docker compose exec gateway nginx -s reload` . If that fails, run`docker compose restart gateway` . - **403 for an admin key.** The token is in`keys-user.map` but not in`keys-admin.map` . Admin keys must appear in both files, which`make-key.sh --admin` handles. - **curl reports `SSL certificate problem: self-signed certificate`.** Pass`--cacert certs/gateway.crt` . If the error mentions a name mismatch, the certificate lacks the host name you typed in its`subjectAltName` list. Regenerate it with the right entries. - **The Python SDK raises a certificate verify error.** You skipped the custom`httpx` client. Pass`http_client=httpx.Client(verify="certs/gateway.crt")` as in Step 10, or set the`SSL_CERT_FILE` environment variable to the certificate path. - **Browser requests fail with a CORS error.** Preflight`OPTIONS` requests carry no Authorization header, so the gateway answers 401. More importantly, calling the API from browser code exposes the token. Put a small server-side function between the browser and the gateway instead. ### Startup, streaming and performance errors - **The gateway exits with `host not found in upstream ollama`.** The Ollama container was not on the network yet. The` depends_on` health condition prevents this, so confirm it is in your Compose file and that the Ollama health check passes. - **502 Bad Gateway.** The gateway cannot reach Ollama. Run`docker compose logs ollama` . A container started with`OLLAMA_HOST` left at loopback refuses connections from other containers, so keep the`0.0.0.0:11434` setting from Step 2. - **504 or a cut-off on the first request.** The model took longer to load than the proxy timeout. Raise`proxy_read_timeout` , and raise`OLLAMA_KEEP_ALIVE` so the model stays loaded between calls. - **The answer arrives all at once.** Buffering is on somewhere. Confirm`proxy_buffering off` is in`proxy.conf` , that the file is actually included, and that no CDN or second proxy sits between you and the gateway. - **413 Request Entity Too Large.** Multimodal requests carry base64 images that exceed`client_max_body_size` . Raise it from 20m to something that fits your largest image. - **The model runs on CPU inside Docker.** The GPU block in Compose is still commented out, or the NVIDIA Container Toolkit is missing. Run`docker compose exec ollama ollama ps` and read the processor column after a request. On a Mac, Docker Desktop cannot pass the GPU through to containers. Run the native Ollama app instead, remove the `ollama` service from Compose, and change the upstream to `host.docker.internal:11434`. The native app listens on loopback only, which is what you want, and the gateway container reaches it through that special host name. Depending on the version, you may need to set `OLLAMA_HOST` for the app so it accepts traffic from the Docker bridge, and in that case firewall the port to the local machine. ## Advanced Tips: Caddy, Private Networks, Rotation and Audit **Swap Nginx for Caddy when you want real certificates.** Caddy obtains and renews Let’s Encrypt certificates on its own, which removes Step 5 for public hostnames. The Caddyfile below supports one shared key read from an environment variable. For several named callers, stay with the Nginx map. ``` # Caddyfile llm.example.com { @noauth not header Authorization "Bearer {env.OLLAMA_GATEWAY_KEY}" respond @noauth 401 reverse_proxy ollama:11434 { flush_interval -1 header_up Host localhost:11434 header_up -Authorization } } ``` **Skip the public internet entirely.** A private overlay network such as Tailscale or WireGuard removes most of the risk by limiting who can reach the gateway at all. Bind the published port to the overlay address, for example `"100.64.0.10:8443:8443"` in the Compose file, and keep the token as a second layer. Two weak barriers together beat one strong one that you misconfigure. **Rotate keys with an overlap.** Create the replacement first, move the client, then delete the old line. The remove command below keeps the file’s inode intact so the bind mount survives. ``` scripts/make-key.sh alice2 # switch the client to keys/alice2.txt, then retire the old key: grep -v ' alice;$' nginx/keys-user.map > /tmp/keys.tmp cat /tmp/keys.tmp > nginx/keys-user.map rm /tmp/keys.tmp keys/alice.txt docker compose exec gateway nginx -s reload ``` **Mine the JSON logs.** Because the log format writes one JSON object per line with the caller name, a one-line `jq` filter answers questions like who is being throttled or who is calling model-management routes. ``` docker compose exec -T gateway cat /var/log/nginx/access.log \ | jq -r 'select(.status == 429) | .client' | sort | uniq -c | sort -rn docker compose exec -T gateway cat /var/log/nginx/access.log \ | jq -r 'select(.uri | test("^/api/(pull|delete|create)")) | [.t, .client, .uri, .status] | @tsv' ``` **Mind the filesystem on Windows.** If you run Docker through WSL2, keep the project inside the Linux filesystem, not under a mounted Windows drive. Permission bits such as `chmod 600` do not work reliably on mounted drives, and file-change detection for bind mounts can lag, which makes a freshly added key seem to vanish until you restart the container. Feed repeated 401 lines from one address into fail2ban or a firewall rule if the gateway faces a hostile network. For per-user token budgets and spend tracking rather than request counts, a dedicated LLM proxy such as LiteLLM can sit between your clients and Ollama and keep a ledger per key. That adds a database and another moving part, so adopt it only when a plain request limit stops being enough. Other upgrades worth considering include mutual TLS for machine-to-machine clients, hashing stored tokens (Nginx can compare hashes through the njs module or a small auth service), and separate rate zones for chat and embedding calls. For wider background on running and securing models, the [AI and machine learning](https://shattered.io/category/ai-machine-learning/) section collects our related tutorials. Recent Ollama releases also widened compatibility, since v0.40.14 reportedly improved OpenAI-style endpoint handling including `/v1/responses`, so rerun the smoke test after each upgrade. ## The Finished Project at a Glance When every step is done, the folder looks like this. Everything above the keys and certificates folders is safe to commit. ``` ollama-gateway/ docker-compose.yml .env # OLLAMA_TAG=0.40.14 (optional pin) .gitignore # keys/ certs/ nginx/keys-*.map nginx/ nginx.conf proxy.conf keys-user.map keys-admin.map certs/ gateway.crt gateway.key keys/ admin.txt alice.txt scripts/ make-key.sh chat.py stream_check.py tests/ smoke_test.py ``` | Step | What it produced | How to confirm | |---|---|---| | 1-3 | Folder, Compose file, Ollama container with a model | `curl localhost:11434` fails from the host | | 4-5 | Named keys and a TLS certificate | `ls -l keys/` shows mode 600 | | 6-7 | Nginx config and proxy settings | `nginx -t` reports success | | 8-9 | Running gateway, curl calls | 401 without a key, 200 with one | | 10-11 | SDK client and streaming check | Many chunks, early first chunk | | 12 | Automated smoke test | 9/9 checks passed | Run `python tests/smoke_test.py` after every change, keep the key files out of version control, and pin `OLLAMA_TAG` once you have confirmed a stable build. The same gateway then serves a script, an editor and an agent at once, each with its own name in the logs. ## FAQ **Does the local Ollama API have a built-in API key?** No. The local server at port 11434 accepts requests without credentials. Ollama API keys apply to the hosted service at ollama.com. For a self-hosted server you add authentication yourself, for example with the Nginx gateway above. **Can I use the OpenAI Python SDK with Ollama?** Yes. Point `base_url` at the `/v1` path, either on localhost or on your gateway, and pass any string as `api_key` for the local server. Behind the gateway, pass the gateway token so the SDK sends it as a Bearer header. **How do I get an Ollama API key for ollama.com?** Sign in to your Ollama account, open the Ollama Keys page in settings, and create a key. Send it as `Authorization: Bearer` to `https://ollama.com/api`, or set the `OLLAMA_API_KEY` environment variable for tools that read it. Per the documentation, keys do not expire and you can revoke them from the same page. **Is a self-signed certificate good enough?** For a home lab or an office LAN, yes, provided clients trust that exact certificate file and the subjectAltName list covers the names they use. For a public hostname, use a real certificate through Let’s Encrypt, either with Caddy or with Certbot in front of Nginx. **Will the gateway slow down responses?** The extra hop adds little compared with model inference, and streaming still works because buffering is off. Measure with the stream check script before and after any change rather than trusting a general claim. **Can I run this without Docker?** Yes. Keep Ollama on its default loopback address, install Nginx on the host, and change the upstream to `127.0.0.1:11434`. The key maps, rate limit, TLS and admin rules stay the same. **How many keys can the map file hold?** Hundreds work fine, since Nginx builds a hash table from the map at load time. Beyond that scale, move identity to an auth service or an identity-aware proxy so you can manage users without editing files. **Why block the pull, create and delete endpoints for normal keys?** Those calls change what is installed on the server and can fill the disk or remove a model other people depend on. Normal callers only need inference and read-only listing, so the safest default is to reserve model management for an admin key.