Ollama API Key Setup: 12 Steps, 60 Min [2026]

Ollama's local API at http://localhost:11434 validates no credentials, so self-hosted users must build their own authentication layer in front of it, according to a tutorial published by Shattered.io covering Ollama v0.40.14. The guide's approach places Ollama in a Docker container with no published port behind an Nginx gateway that terminates TLS, checks a Bearer token against a list of named keys, throttles each caller, and blocks model-management calls unless the key carries admin rights, a setup it says takes 12 steps and roughly 60 minutes. The exposure extends beyond inference to endpoints that pull, push, create, copy and delete models, and Ollama said in its August 31, 2026 pricing announcement that it serves 8.9 million developers.

The local Ollama API at http://localhost:11434 checks no credentials, so any Ollama API key for a self-hosted server has to come from a layer you build in front of it. The newest stable release, Ollama v0.40.14 according to a release tracker https://rwatcher.yadoc.fr/components/3efe10e0-9fd5-436f-a6f5-361b8dec5119/cc002b73-46f9-42db-9602-332441e28976 , keeps that rule. Local requests need no key. The hosted API at ollama.com does. That gap trips up a lot of people. Agent frameworks such as the one in our OpenHands local setup with Ollama https://shattered.io/openhands-local-setup-agent-canvas-ollama-12-steps-2026/ expect an OpenAI-style endpoint plus a key field, and they send whatever string you hand them. If the server never validates it, that field is decoration. Anyone who can reach the port can run prompts, list models, pull new ones and delete the ones you already downloaded. The fix is a small gateway. Ollama runs in a Docker container with no published port. An Nginx container sits in front, terminates TLS, checks a Bearer token against a list of named keys, throttles each caller, and refuses model-management calls unless the key carries admin rights. Clients keep using the standard OpenAI SDK, because the SDK already sends Authorization: Bearer on every request. Twelve steps and roughly an hour of work get you there, and the whole thing fits in one folder you can commit minus the secrets . Don't miss new tech stories on Google Add Shattered once in the Google app and our stories show up in your news suggestions. Add now https://www.google.com/preferences/source?hl=en&q=https%3A%2F%2Fshattered.io%2F Why the Ollama API Needs a Key You Have to Build Ollama binds to 127.0.0.1:11434 by default, which is a sensible default for a desktop tool. Trouble starts when someone sets OLLAMA HOST=0.0.0.0 on a shared machine, publishes the port in a Docker command, or forwards it through a router so a laptop or phone can reach the model. At that point the server speaks to anyone, because there is no login screen, no token check and no per-user accounting in the local API. The exposure covers more than inference. The native API includes endpoints that pull, push, create, copy and delete models. An unauthenticated caller on your network can fill the disk by pulling large models, delete the model your team relies on, or keep the GPU busy with long generations while your own jobs wait. None of that needs an exploit. It only needs the port to be reachable. Ollama serves a large audience. The company said in its August 31, 2026 pricing announcement that it serves 8.9 million developers, and plenty of them will eventually flip a server from loopback to a shared network. A self-hosted deployment needs three controls the local server does not provide: identity who is calling , authorization what that caller may do and rate control how much . The gateway in this tutorial supplies all three with plain configuration and no custom server code. Two design choices keep the setup small. First, keys live in a flat map file that Nginx reads, so adding or revoking a caller is one line and one reload. Second, the OpenAI-compatible /v1 path and the native /api path share the same key, so a single credential works for both the SDK and tools like curl or Postman. A gateway is not a force field, so set expectations before building one. It does not stop a legitimate key holder from sending a malicious prompt, and it does not make model output trustworthy. It does not protect the host if someone already has shell access, and it cannot hide a token that a client leaks. A stolen token works from anywhere until you revoke it, which is why per-caller names and logs matter as much as the check itself. What it does give you is a narrow, auditable front door instead of an open port, and for a self-hosted model that is the largest single improvement available. What Ollama Authenticates: Local API vs Cloud API Ollama now exposes two different API surfaces, and mixing them up is the most common source of invalid-key confusion. The local server answers on port 11434 and ignores credentials. The hosted service at ollama.com does require a key. Ollama’s documentation, which was updated again on November 4, 2026 per a changelog digest https://techgazette.com.br/company/ollama/ , spells out the split, and the table below condenses it. | Property | Local server | Hosted API ollama.com | |---|---|---| | Base URL native | http://localhost:11434/api | https://ollama.com/api | | Base URL OpenAI-style | http://localhost:11434/v1 | https://ollama.com/v1 | | Credential | None required | API key sent as Authorization: Bearer | | Install needed | Yes, the Ollama app or container | No, per Ollama’s API docs | | Key management | You build it this tutorial | Ollama Keys page in your account | | Key lifetime | Whatever you configure | Non-expiring, revocable from settings | | Env var used by clients | Usually none | OLLAMA API KEY | A third credential exists and deserves a mention so it does not mislead you. Ollama’s docs also describe an Ollama public key that lets a local installation authenticate with ollama.com to push models, pull private models and run cloud-hosted models. That key identifies your machine to Ollama’s service. It does nothing to protect the port on your own network, and a reverse proxy does not create or replace it. One practical consequence follows. When you route cloud models through a local server by adding :cloud to the model name, the server needs Ollama v0.40.5 or later and a signed-in account. Your gateway then protects that route too, which matters because the signed-in server spends your cloud usage on behalf of every caller. An open local port with a signed-in account can burn paid credits, not just GPU time. Prerequisites and Versions Everything below ran against Ollama v0.40.14, the stable release at the time of writing. A v0.40.15 build also appears in release listings, but it is labeled a pre-release, so skip it for anything you plan to leave running. If you read an older tutorial that calls v0.40.13 or earlier the latest version, treat its version-specific advice with caution. | Component | Version or tag | Role in the project | |---|---|---| | Ollama container | ollama/ollama , stable v0.40.14 or newer | Model server, never published to the host | | Docker Engine | Current release with the Compose v2 plugin docker compose | Runs both containers | | Nginx container | nginx:stable-alpine | TLS, key check, rate limit, logging | | OpenSSL | 1.1.1 or newer needed for -addext | Self-signed certificate and key generation | | Python | 3.10 or newer | Client script and test suite | | Python packages | Latest openai and httpx | SDK client, raw HTTP tests | | NVIDIA Container Toolkit | Optional, current release | GPU access inside Docker on Linux | | Test model | gpt-oss:20b , or a smaller library model | Anything that fits your memory works | For the model, gpt-oss:20b is the default in the scripts because it is listed in the Ollama library and answers through the chat API. If it does not fit your hardware, set the MODEL environment variable to something smaller, such as gemma4 . NVIDIA’s Nemotron 3.5 Bolt E4B, which one news report https://news.yushang.ai/en/2026/10/ollama-adds-nemotron-3-5-bolt-e4b-to-e1e567 says Ollama added support for in October 2026, is another small option. Model files are the largest download in the whole project, so pick one that leaves headroom for context rather than one that barely loads. You also need a machine you control, either a Linux box, a Mac with Docker Desktop, or a Windows machine with WSL2. GPU passthrough works on Linux with the NVIDIA toolkit, while Docker Desktop on Mac cannot hand the GPU to containers. Mac users get better speed running the Ollama app natively and pointing the gateway at host.docker.internal:11434 , which the troubleshooting section covers. Disk space deserves a thought before you pull anything. Model files are large, the Docker volume holds every model you ever pulled, and an admin key that pulls without looking can fill the drive. Check free space with df -h on the Docker data directory, and plan to prune models you no longer use with ollama rm . Keeping this housekeeping behind an admin key is part of why the gateway separates normal callers from model managers. Phase 1: Project Folder and Ollama in Docker Steps 1-3 Step 1: Create the folder layout and check your tools. Every file lives under one directory, so you can zip it, commit it or move it to another machine. Keys and certificates get their own folders because you will keep those out of version control. mkdir -p ollama-gateway/{nginx,certs,keys,scripts,tests} cd ollama-gateway touch nginx/keys-user.map nginx/keys-admin.map docker --version docker compose version openssl version python3 --version Expected output on a typical Linux box looks like the block below. Your exact versions will differ. What matters is that docker compose https://shattered.io/docker-compose-tutorial-production-stack-12-steps-2026/ Docker version 27.x.x, build xxxxxxx Docker Compose version v2.x.x OpenSSL 3.x.x or 1.1.1 and up Python 3.1x.x The two empty .map files matter. Nginx refuses to start if an include points at a file that does not exist, so creating them now saves you a confusing error in Step 8. Step 2: Write the Compose file. The key detail is what is missing. The ollama service has no ports entry, so nothing on the host or the LAN can reach it directly. Only the gateway service publishes a port, and only on 8443. The Ollama server inside the container listens on all container interfaces so the gateway can reach it over the private Compose network. docker-compose.yml services: ollama: image: ollama/ollama:${OLLAMA TAG:-latest} restart: unless-stopped environment: OLLAMA HOST: 0.0.0.0:11434 OLLAMA KEEP ALIVE: 10m volumes: - ollama models:/root/.ollama healthcheck: test: "CMD", "ollama", "list" interval: 10s timeout: 5s retries: 12 Uncomment for NVIDIA GPUs needs the NVIDIA Container Toolkit deploy: resources: reservations: devices: - driver: nvidia count: all capabilities: gpu gateway: image: nginx:stable-alpine restart: unless-stopped depends on: ollama: condition: service healthy ports: - "8443:8443" volumes: - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro - ./nginx/proxy.conf:/etc/nginx/proxy.conf:ro - ./nginx/keys-user.map:/etc/nginx/keys-user.map:ro - ./nginx/keys-admin.map:/etc/nginx/keys-admin.map:ro - ./certs:/etc/nginx/certs:ro - gateway logs:/var/log/nginx volumes: ollama models: gateway logs: The OLLAMA TAG variable lets you pin the image later. Run with the default latest tag first, confirm the version in Step 3, then set OLLAMA TAG in a .env file so an unattended docker compose pull cannot move you to a new release without you noticing. Binding to 0.0.0.0 inside the container is safe here only because no ports mapping exposes it. Copy that line to a bare-metal install and you have rebuilt the problem this tutorial exists to solve. Step 3: Start Ollama alone, check the version and pull a model. Bring up only the model server first. If something is wrong with the GPU or the volume, you want to see it before the gateway adds another layer. docker compose up -d ollama docker compose ps docker compose exec ollama ollama --version docker compose exec ollama ollama pull gpt-oss:20b docker compose exec ollama ollama list NAME IMAGE SERVICE STATUS PORTS ollama-gateway-ollama-1 ollama/ollama:latest ollama Up 40 seconds healthy 11434/tcp ollama version is 0.40.14 NAME ID SIZE MODIFIED gpt-oss:20b ... ... a few seconds ago The 11434/tcp under PORTS has no arrow and no host address. That is Docker telling you the port is exposed to other containers only. Now prove the port is closed from the outside. This is the check people skip, and it is the one that tells you the architecture works. curl -sS --max-time 3 http://localhost:11434/api/version || echo "closed, as intended" You want a connection error followed by the echo. If you see a JSON version string instead, something else on the host is already serving 11434, most often a native Ollama install started by systemd or the desktop app. That is a separate process from the container, and Step 9 explains why it can make test results confusing. Phase 2: API Keys and TLS Certificate Steps 4-5 Step 4: Write the key generator. Each caller gets its own random token, stored in a file only you can read. Nginx gets the token and a human-readable name through a map file. Per-caller names are what make the later logs and rate limits useful, because a log line that names Alice as the throttled caller is actionable and one that names nobody is not. bash /usr/bin/env bash scripts/make-key.sh set -euo pipefail cd "$ dirname "$0" /.." name="${1:?usage: scripts/make-key.sh NAME --admin }" if "$name" =~ ^ a-z0-9 - +$ ; then echo "name must match a-z0-9 - +" &2; exit 1 fi if grep -q " ${name};\$" nginx/keys-user.map; then echo "key for ${name} already exists" &2; exit 1 fi token="olk $ openssl rand -hex 32 " umask 077 printf '%s' "$token" "keys/${name}.txt" printf '"Bearer %s" %s;\n' "$token" "$name" nginx/keys-user.map if "${2:-}" = "--admin" ; then printf '"Bearer %s" %s;\n' "$token" "$name" nginx/keys-admin.map fi if docker compose ps --status running gateway 2 /dev/null | grep -q gateway; then docker compose exec -T gateway nginx -t && docker compose exec -T gateway nginx -s reload fi echo "created keys/${name}.txt" Three details are worth understanding. The olk prefix is cosmetic, but it makes leaked tokens easy to find with a secret scanner and easy to spot in a pasted log. The 64 hex characters come from 32 random bytes, which is far beyond what anyone can guess. And the script appends with rather than rewriting the file, which keeps the same inode so the bind mount stays valid. Editors that save by replacing the file can break that link, a pitfall covered later. Create one admin key for you and one normal key for the first client. Admin keys are written to both map files, so an admin can also use the regular endpoints. chmod +x scripts/make-key.sh scripts/make-key.sh admin --admin scripts/make-key.sh alice ls -l keys/ created keys/admin.txt created keys/alice.txt -rw------- 1 you you 68 Nov 6 10:20 admin.txt -rw------- 1 you you 68 Nov 6 10:20 alice.txt The 68-byte size is the 4-character prefix plus 64 hex characters, with no trailing newline. That detail saves you from the classic copy-paste bug where a newline rides along inside the header. Permissions of -rw------- confirm the umask 077 did its job. Step 5: Generate the TLS certificate. Bearer tokens travel in a header on every request, so sending them over plain HTTP hands the key to anyone on the path. A self-signed certificate is fine for a home lab or an office LAN as long as clients trust that one file. The subjectAltName entry is not optional, because modern clients ignore the common name and reject certificates that lack it. openssl req -x509 -newkey rsa:2048 -nodes -days 365 \ -keyout certs/gateway.key -out certs/gateway.crt \ -subj "/CN=localhost" \ -addext "subjectAltName=DNS:localhost,IP:127.0.0.1" chmod 600 certs/gateway.key openssl x509 -in certs/gateway.crt -noout -subject -enddate -ext subjectAltName subject=CN = localhost notAfter=Nov 6 10:21:03 2027 GMT X509v3 Subject Alternative Name: DNS:localhost, IP Address:127.0.0.1 If other machines will call the gateway, add their DNS name or LAN address to the subjectAltName list, for example DNS:llm.home.lan,IP:192.168.1.50 , and regenerate. For a public hostname, skip the self-signed route and use a real certificate from Let’s Encrypt, which the advanced section covers with Caddy. Phase 3: The Nginx Gateway Config Steps 6-7 Step 6: Write nginx.conf. This file does the real work, so it gets the longest explanation. Nginx https://nginx.org/ reads the incoming Authorization header, looks the exact string up in a map, and sets $client name to the caller’s name or to an empty string. An empty name means no valid key, and the server answers 401 before any proxying happens. A second map does the same for admin rights. nginx/nginx.conf worker processes auto; events { worker connections 1024; } http { include /etc/nginx/mime.types; default type application/json; server tokens off; Identity: exact match on the full Authorization header value. map $http authorization $client name { default ""; include /etc/nginx/keys-user.map; } map $http authorization $admin name { default ""; include /etc/nginx/keys-admin.map; } Structured access log. The Authorization header is never logged. log format gw escape=json '{"t":"$time iso8601","client":"$client name","method":"$request method",' '"uri":"$uri","status":$status,"bytes":$body bytes sent,' '"rt":$request time,"ip":"$remote addr"}'; access log /var/log/nginx/access.log gw; Rate limit per named caller: 30 requests per minute, bursts of 10. limit req zone $client name zone=perclient:10m rate=30r/m; limit req status 429; upstream ollama backend { server ollama:11434; keepalive 8; } server { listen 8443 ssl; server name ; ssl certificate /etc/nginx/certs/gateway.crt; ssl certificate key /etc/nginx/certs/gateway.key; ssl protocols TLSv1.2 TLSv1.3; client max body size 20m; No valid key, no service. Runs before any location logic. if $client name = "" { add header WWW-Authenticate 'Bearer realm="ollama"' always; return 401 '{"error":"missing or invalid API key"}'; } Model management needs an admin key. location ~ ^/api/ pull|push|create|copy|delete|blobs { if $admin name = "" { return 403 '{"error":"admin key required"}'; } include /etc/nginx/proxy.conf; proxy pass http://ollama backend; } Everything else: inference and read-only calls. location / { limit req zone=perclient burst=10 nodelay; include /etc/nginx/proxy.conf; proxy pass http://ollama backend; } } } Walk through the decisions, because each one has a failure mode behind it. The map lookup is an exact string comparison, so Bearer must be capitalized as written and there must be exactly one space. The OpenAI SDK and curl examples below both send it that way. Using a map instead of a chain of if statements keeps Nginx’s configuration cost flat even with hundreds of keys, since map lookups use a hash table. The rate limit zone keys on $client name , which means every caller gets an independent bucket. At 30 requests per minute with a burst of 10, one caller can send eleven requests instantly and then roughly one every two seconds. Chat completions are heavy, so that pace is generous for a single person and tight enough to stop a runaway script. Admin routes skip the limit on purpose, since pulls are rare and a throttled pull can corrupt your patience more than your disk. The 401 check runs at server level, in Nginx’s rewrite phase, which executes before limit req and before proxying. Unauthenticated floods therefore never consume rate-limit memory or reach Ollama. If you have run a proxy in front of a long-lived service before, such as the SRI proxy in our Stratum V2 mining setup https://shattered.io/stratum-v2-mining-setup-2026/ , the pattern is the same: terminate and authenticate at the edge, then forward plain traffic over a private network. How one request travels through the gateway 1. The client opens a TLS connection to port 8443 and sends the request with an Authorization: Bearer header. 2. Nginx looks up the header value in the user map. No match means an immediate 401 and the request ends there. 3. For paths such as /api/pull , Nginx also checks the admin map. No match means a 403. 4. For everything else, the per-caller rate limit runs. A caller over budget gets a 429. 5. Nginx removes the Authorization header, sets the Host header to localhost:11434 and forwards the request over the private Compose network. 6. Ollama streams its reply back, Nginx passes each chunk through unbuffered, and the access log records the caller name, route, status and duration. That order is deliberate. Cheap checks run first, so the expensive work of loading a model or generating tokens only starts for callers who passed identity, authorization and rate control. Step 7: Write proxy.conf and validate the whole config. The shared proxy settings sit in their own file so both locations stay identical. Four of these lines exist specifically because LLM traffic is unlike ordinary web traffic. nginx/proxy.conf proxy http version 1.1; proxy set header Connection ""; proxy set header Host localhost:11434; proxy set header Authorization ""; proxy set header X-Real-IP $remote addr; Stream tokens as they are generated instead of buffering the reply. proxy buffering off; proxy request buffering off; A cold model load can take minutes. Do not cut it off. proxy connect timeout 10s; proxy send timeout 600s; proxy read timeout 600s; - proxy set header Host localhost:11434 follows Ollama’s FAQ guidance for Nginx, which says to forward the Host header as localhost:11434 . Without it, some requests are rejected because the server sees an unexpected host name. - proxy set header Authorization "" strips the gateway token before the request reaches Ollama, so the key never lands in the model server’s logs. - proxy buffering off lets streamed tokens flow through immediately. With buffering on, a streamed answer arrives as one lump at the end. - The 600-second timeouts cover the first request after a model unloads. The default 60 seconds is too short for a large model that must load into memory. Check the syntax before starting anything. The test container below mounts your files read-only and runs nginx -t , which parses the config and exits. docker run --rm \ -v "$PWD/nginx/nginx.conf:/etc/nginx/nginx.conf:ro" \ -v "$PWD/nginx/proxy.conf:/etc/nginx/proxy.conf:ro" \ -v "$PWD/nginx/keys-user.map:/etc/nginx/keys-user.map:ro" \ -v "$PWD/nginx/keys-admin.map:/etc/nginx/keys-admin.map:ro" \ -v "$PWD/certs:/etc/nginx/certs:ro" \ --add-host ollama:127.0.0.1 \ nginx:stable-alpine nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful The --add-host flag is a trick for this one-off test. The config names an upstream host called ollama , and the standalone container cannot resolve it. Pointing the name at loopback lets the syntax check finish without the full stack running. Phase 4: Start the Gateway and Call It With curl Steps 8-9 Step 8: Bring up the full stack and read the first log lines. The gateway waits for Ollama’s health check, so a cold start takes a few seconds longer than the model server alone. docker compose up -d docker compose ps docker compose logs --tail=5 gateway NAME IMAGE SERVICE STATUS PORTS ollama-gateway-gateway-1 nginx:stable-alpine gateway Up 6 seconds 0.0.0.0:8443- 8443/tcp ollama-gateway-ollama-1 ollama/ollama:latest ollama Up 2 minutes healthy 11434/tcp Only 8443 shows a host mapping. If the gateway exits immediately, run docker compose logs gateway and look for the first error. A missing map file, a certificate path typo and an unresolvable upstream cause nearly all startup failures, and the troubleshooting section lists the exact messages. Step 9: Test with curl, first without a key and then with one. Always point tests at port 8443. If you already run the native Ollama app on the same machine, it answers on 11434 and has no gateway in front of it, so a test against that port proves nothing about the stack you just built. Use --cacert so curl trusts your self-signed certificate instead of failing or, worse, being told to ignore verification with -k . export GW=https://localhost:8443 export KEY=$ cat keys/alice.txt 1. No key: expect 401 curl -sS --cacert certs/gateway.crt -i $GW/api/version | head -n 12 2. Valid key: expect 200 and a version string curl -sS --cacert certs/gateway.crt -H "Authorization: Bearer $KEY" $GW/api/version 3. Valid key, OpenAI-compatible chat curl -sS --cacert certs/gateway.crt $GW/v1/chat/completions \ -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \ -d '{"model":"gpt-oss:20b","messages": {"role":"user","content":"Reply with the single word ready."} }' HTTP/1.1 401 Unauthorized Server: nginx Content-Type: application/json WWW-Authenticate: Bearer realm="ollama" {"error":"missing or invalid API key"} {"version":"0.40.14"} {"id":"chatcmpl-...","object":"chat.completion","model":"gpt-oss:20b", "choices": {"index":0,"message":{"role":"assistant","content":"ready"},"finish reason":"stop"} , ...} The first request on a cold model can take a while because Ollama loads the weights into memory. That delay is why the gateway uses long proxy timeouts, and the OLLAMA KEEP ALIVE setting of 10 minutes in the Compose file keeps the model resident between requests so later calls return quickly. Note that the version string in your output should match the image you pulled, which is a handy way to confirm you are hitting the container and not a stray native install. Phase 5: Python Clients and a Streaming Check Steps 10-11 Step 10: Call the gateway with the official OpenAI Python SDK. This is the payoff for choosing Bearer tokens. The SDK takes a base url and an api key , and it puts the key in the right header on every call. The only extra is an httpx client that trusts your certificate. The same two settings work in editors and agents, for instance the custom server fields in our Kilo Code local setup https://shattered.io/kilo-code-local-setup-ollama-vs-code-2026/ . python3 -m venv .venv source .venv/bin/activate pip install --upgrade openai httpx python /usr/bin/env python3 scripts/chat.py import os import sys from pathlib import Path import httpx from openai import OpenAI ROOT = Path file .resolve .parent.parent BASE = os.getenv "GATEWAY URL", "https://localhost:8443" MODEL = os.getenv "MODEL", "gpt-oss:20b" TOKEN = ROOT / "keys" / f"{os.getenv 'KEY NAME', 'alice' }.txt" .read text .strip client = OpenAI base url=f"{BASE}/v1", api key=TOKEN, http client=httpx.Client verify=str ROOT / "certs" / "gateway.crt" , timeout=600 , prompt = " ".join sys.argv 1: or "In two sentences, why should a self-hosted LLM API require a key?" stream = client.chat.completions.create model=MODEL, messages= {"role": "user", "content": prompt} , stream=True, for chunk in stream: if chunk.choices and chunk.choices 0 .delta.content: print chunk.choices 0 .delta.content, end="", flush=True print python scripts/chat.py "Name three risks of an open Ollama port." An open Ollama port lets strangers run prompts on your hardware, pull or delete models without asking, and keep the GPU busy while your own jobs wait. Many other tools skip code entirely and read two environment variables, OPENAI BASE URL and OPENAI API KEY . Export your gateway URL with the /v1 suffix and the contents of a key file, and any tool built on the OpenAI SDK will talk to your server. Tools that wrap other runtimes usually have a custom endpoint field instead. Either way, give every tool its own key from make-key.sh , so the log shows which one is busy and you can revoke it alone. Reasoning models such as gpt-oss may pause before the first visible token while they think. That pause is the model working, not the gateway stalling. The next script separates the two by measuring the gap before the first chunk and the generation speed Ollama reports for itself. Step 11: Verify that streaming survives the proxy. A misconfigured proxy is the most common reason a streaming API feels frozen, because buffering collects the whole answer and releases it at once. This script calls the native /api/chat endpoint, which streams one JSON object per line, and prints the time to the first chunk, the chunk count and the tokens per second computed from the eval count and eval duration fields in the final object. python /usr/bin/env python3 scripts/stream check.py import json import os import time from pathlib import Path import httpx ROOT = Path file .resolve .parent.parent BASE = os.getenv "GATEWAY URL", "https://localhost:8443" MODEL = os.getenv "MODEL", "gpt-oss:20b" TOKEN = ROOT / "keys" / f"{os.getenv 'KEY NAME', 'alice' }.txt" .read text .strip payload = { "model": MODEL, "messages": {"role": "user", "content": "Count from 1 to 40, one number per line."} , "stream": True, } start = time.perf counter first = None chunks = 0 final = {} with httpx.Client base url=BASE, verify=str ROOT / "certs" / "gateway.crt" , headers={"Authorization": f"Bearer {TOKEN}"}, timeout=600 as c: with c.stream "POST", "/api/chat", json=payload as r: r.raise for status for line in r.iter lines : if not line: continue msg = json.loads line chunks += 1 body = msg.get "message", {} if first is None and body.get "content" or body.get "thinking" : first = time.perf counter - start if msg.get "done" : final = msg total = time.perf counter - start tokens = final.get "eval count", 0 gen s = final.get "eval duration", 0 / 1e9 print f"first chunk after : {first:.2f} s" if first else "no content received" print f"total time : {total:.2f} s" print f"stream chunks : {chunks}" if tokens and gen s: print f"generation speed : {tokens / gen s:.1f} tokens/s {tokens} tokens " if first and chunks 3 and first 0.8 total: print "WARNING: first chunk arrived near the end. Check proxy buffering." first chunk after : 1.84 s total time : 3.97 s stream chunks : 118 generation speed : 52.3 tokens/s 112 tokens Treat those numbers as an example. Your hardware, model and quantization decide the real values. The signal to look for is the shape: many chunks, a first chunk well before the total time, and no warning. If you see two or three chunks and a first-chunk time close to the total, buffering is on somewhere in the path, and the pitfalls section tells you where to look. Phase 6: Automated Tests for Auth, Limits and Streaming Step 12 Step 12: Lock the behavior in with a smoke test. Manual curl checks prove the stack works once. A script proves it keeps working after you edit a map file, upgrade the image or tweak a timeout. The suite below checks nine behaviors, exits non-zero on any failure, and is short enough to run from a cron job or a CI step after every change. python /usr/bin/env python3 tests/smoke test.py import json import os import sys from pathlib import Path import httpx ROOT = Path file .resolve .parent.parent BASE = os.getenv "GATEWAY URL", "https://localhost:8443" CERT = str ROOT / "certs" / "gateway.crt" MODEL = os.getenv "MODEL", "gpt-oss:20b" def key name : return ROOT / "keys" / f"{name}.txt" .read text .strip def client token=None : headers = {"Authorization": f"Bearer {token}"} if token else {} return httpx.Client base url=BASE, verify=CERT, headers=headers, timeout=300 results = def check name, ok, detail="" : results.append ok print f"{'PASS' if ok else 'FAIL'} {name} {detail}" 1. Plain HTTP on the TLS port is refused try: r = httpx.get BASE.replace "https://", "http://" + "/", timeout=10 check "plain HTTP is refused", r.status code == 400, f"got {r.status code}" except httpx.HTTPError as exc: check "plain HTTP is refused", True, f"connection error: {type exc . name }" 2-3. Missing and wrong keys with client as c: r = c.get "/api/version" check "no key is rejected", r.status code == 401, f"got {r.status code}" with client "olk " + "0" 64 as c: r = c.get "/api/version" check "wrong key is rejected", r.status code == 401, f"got {r.status code}" 4-6. Normal user with client key "alice" as c: r = c.get "/api/version" check "user key reads version", r.status code == 200, r.text.strip r = c.post "/api/pull", json={"model": MODEL, "stream": False} check "user key cannot pull", r.status code == 403, f"got {r.status code}" r = c.post "/v1/chat/completions", json={ "model": MODEL, "messages": {"role": "user", "content": "Reply with ok."} , } check "user key can chat via /v1", r.status code == 200, f"got {r.status code}" 7. Admin gate with client key "admin" as c: r = c.post "/api/pull", json={"model": MODEL, "stream": False} check "admin key passes the admin gate", r.status code == 200, f"got {r.status code}" 8. Streaming produces several chunks with client key "alice" as c: payload = {"model": MODEL, "stream": True, "messages": {"role": "user", "content": "Count from 1 to 20."} } n = 0 with c.stream "POST", "/api/chat", json=payload as r: for line in r.iter lines : if line and json.loads line : n += 1 check "native chat streams in several chunks", n 3, f"chunks={n}" 9. Rate limit trips under a burst keep this check last with client key "alice" as c: codes = c.get "/api/version" .status code for in range 25 limited = codes.count 429 check "burst of 25 requests hits the rate limit", limited = 1, f"429s={limited}" failed = results.count False print f"\n{len results - failed}/{len results } checks passed" sys.exit 1 if failed else 0 python tests/smoke test.py PASS plain HTTP is refused got 400 PASS no key is rejected got 401 PASS wrong key is rejected got 401 PASS user key reads version {"version":"0.40.14"} PASS user key cannot pull got 403 PASS user key can chat via /v1 got 200 PASS admin key passes the admin gate got 200 PASS native chat streams in several chunks chunks=61 PASS burst of 25 requests hits the rate limit 429s=13 9/9 checks passed The rate-limit check stays last for a reason. Once Alice’s bucket drains, her requests return 429 for a few seconds, which would fail any test that ran after it. The expected status for every combination of caller and endpoint is summarized below, which doubles as a checklist when you add routes. | Request | No key | User key | Admin key | |---|---|---|---| | GET /api/version | 401 | 200 | 200 | | GET /api/tags | 401 | 200 | 200 | | POST /v1/chat/completions | 401 | 200 | 200 | | POST /api/chat | 401 | 200 | 200 | | POST /api/pull | 401 | 403 | 200 | | POST /api/delete | 401 | 403 | 200 | | Plain HTTP to port 8443 | 400 | 400 | 400 | The admin pull in the test returns quickly because the model is already on disk, so the call completes without downloading anything. Do not add a delete test that targets a real model. If you want to cover that route, point it at a throwaway model name and accept a not-found response as proof the gate opened. Tune the rate limit to the workload instead of copying the numbers from Step 6. An interactive chat user rarely sends more than a few requests a minute, so 30 per minute leaves plenty of room. An agent that fires tool calls in a loop can burn through that in seconds, and a 429 mid-task looks like a flaky model. For agents, raise the burst, keep the sustained rate modest, and give the agent its own key so its bucket cannot starve a human. For batch jobs that embed thousands of documents, a separate location with its own zone keeps that traffic from eating the interactive budget. Concurrency is a different limit from request rate. Ollama decides how many requests it processes at once, and extra requests queue inside the server. A caller who stays under the rate limit can still hold the GPU for a long time with one huge generation. If that becomes a problem, add a limit conn zone keyed on the caller name to cap simultaneous connections per key. Using Ollama Cloud API Keys Without Mixing Them Up Plenty of readers searching for an Ollama API key actually want the hosted one. The flow is separate from everything above. Open the Ollama Keys page in your account settings, create a key, and send it as a Bearer token to https://ollama.com/api or the OpenAI-style https://ollama.com/v1 . According to Ollama’s documentation, these keys do not expire but can be revoked from the same settings page. Direct cloud calls need no local install. export OLLAMA API KEY="paste-your-key-here" curl https://ollama.com/api/chat \ -H "Authorization: Bearer $OLLAMA API KEY" \ -H "Content-Type: application/json" \ -d '{"model":"gpt-oss:120b","stream":false, "messages": {"role":"user","content":"Reply with the single word ready."} }' Check the cloud model list for the exact model names your plan can use. Ollama moved its Pro, Max and Team plans to per-token pricing on August 31, 2026, and the Team plan starts at $500 per month with $1,000 of usage credits included, per the company’s pricing page. Hosted usage costs money, so guard the key like a payment credential. Ollama’s quickstart guidance on hosted keys tells developers to keep them on the application’s server, away from browser code and source control. The same rule applies to the gateway tokens you created in Step 4. A token embedded in a web page or a mobile app is public the moment someone opens the developer tools. When to use which key - Gateway token this tutorial : protects your own server and your own GPU. You issue, rotate and revoke it. - Ollama API key: authenticates you to the hosted service and its cloud models. Ollama issues and bills it. - Ollama public key: links a local installation to your Ollama account for pushing models and running cloud models. It never appears in an HTTP header you write by hand. If you run cloud models through your local server with the :cloud suffix, the gateway is what stops a stranger from spending your hosted quota. The gateway key identifies the caller to you. The signed-in account then pays Ollama for the usage. Common Pitfalls That Break Ollama Gateways Most failures in this kind of setup repeat. These eight account for the bulk of the reports that start with it worked yesterday. 1. Publishing 11434 anyway. A ports: "11434:11434" line or a -p 11434:11434 flag puts the unauthenticated server back on the network and bypasses your gateway. Docker also writes its own firewall rules that can sidestep host tools like ufw, so a rule that looks like it blocks the port may not. 2. Setting OLLAMA HOST to 0.0.0.0 on a host install. People do it to reach the model from a phone, forget about it, and leave every model-management endpoint open. Keep the host install on loopback and put the gateway in front. 3. Routing pulls through HTTP PROXY. Ollama’s documentation says model downloads use HTTPS, so set HTTPS PROXY for outbound pulls and avoid HTTP PROXY , which can interrupt client connections. 4. Committing keys or certificates. Add keys/ , certs/ and both .map files to .gitignore before the first commit. Rotating a leaked token is easy, but scrubbing it from Git history is not. 5. Editing a map file the wrong way. Some editors and sed -i replace the file with a new inode, and a single-file bind mount keeps pointing at the old one. The gateway then ignores your change even after a reload. Append with or restart the container. 6. Forgetting to reload. Nginx reads the map files at startup and on reload. A new key does nothing until nginx -s reload runs, which the key script does for you when the gateway is up. 7. Logging the Authorization header. Adding $http authorization to the log format writes every token to disk. The format in Step 6 logs the caller name instead, which is all you need. 8. Wrong Host header. Ollama’s Nginx guidance says to forward Host: localhost:11434 . Passing the public host name through can make the server reject requests that look fine otherwise. Troubleshooting: 12 Errors and Fixes Authentication and TLS errors - 401 even though the key is correct. The map does an exact string match. Check for a trailing newline in the key file, a lowercase bearer , or a double space. Compare with cat -A keys/alice.txt , which should end without a $ line break marker. - A newly created key returns 401. Nginx has not reloaded, or the bind mount points at a stale inode. Run docker compose exec gateway nginx -s reload . If that fails, run docker compose restart gateway . - 403 for an admin key. The token is in keys-user.map but not in keys-admin.map . Admin keys must appear in both files, which make-key.sh --admin handles. - curl reports SSL certificate problem: self-signed certificate . Pass --cacert certs/gateway.crt . If the error mentions a name mismatch, the certificate lacks the host name you typed in its subjectAltName list. Regenerate it with the right entries. - The Python SDK raises a certificate verify error. You skipped the custom httpx client. Pass http client=httpx.Client verify="certs/gateway.crt" as in Step 10, or set the SSL CERT FILE environment variable to the certificate path. - Browser requests fail with a CORS error. Preflight OPTIONS requests carry no Authorization header, so the gateway answers 401. More importantly, calling the API from browser code exposes the token. Put a small server-side function between the browser and the gateway instead. Startup, streaming and performance errors - The gateway exits with host not found in upstream ollama . The Ollama container was not on the network yet. The depends on health condition prevents this, so confirm it is in your Compose file and that the Ollama health check passes. - 502 Bad Gateway. The gateway cannot reach Ollama. Run docker compose logs ollama . A container started with OLLAMA HOST left at loopback refuses connections from other containers, so keep the 0.0.0.0:11434 setting from Step 2. - 504 or a cut-off on the first request. The model took longer to load than the proxy timeout. Raise proxy read timeout , and raise OLLAMA KEEP ALIVE so the model stays loaded between calls. - The answer arrives all at once. Buffering is on somewhere. Confirm proxy buffering off is in proxy.conf , that the file is actually included, and that no CDN or second proxy sits between you and the gateway. - 413 Request Entity Too Large. Multimodal requests carry base64 images that exceed client max body size . Raise it from 20m to something that fits your largest image. - The model runs on CPU inside Docker. The GPU block in Compose is still commented out, or the NVIDIA Container Toolkit is missing. Run docker compose exec ollama ollama ps and read the processor column after a request. On a Mac, Docker Desktop cannot pass the GPU through to containers. Run the native Ollama app instead, remove the ollama service from Compose, and change the upstream to host.docker.internal:11434 . The native app listens on loopback only, which is what you want, and the gateway container reaches it through that special host name. Depending on the version, you may need to set OLLAMA HOST for the app so it accepts traffic from the Docker bridge, and in that case firewall the port to the local machine. Advanced Tips: Caddy, Private Networks, Rotation and Audit Swap Nginx for Caddy when you want real certificates. Caddy obtains and renews Let’s Encrypt certificates on its own, which removes Step 5 for public hostnames. The Caddyfile below supports one shared key read from an environment variable. For several named callers, stay with the Nginx map. Caddyfile llm.example.com { @noauth not header Authorization "Bearer {env.OLLAMA GATEWAY KEY}" respond @noauth 401 reverse proxy ollama:11434 { flush interval -1 header up Host localhost:11434 header up -Authorization } } Skip the public internet entirely. A private overlay network such as Tailscale or WireGuard removes most of the risk by limiting who can reach the gateway at all. Bind the published port to the overlay address, for example "100.64.0.10:8443:8443" in the Compose file, and keep the token as a second layer. Two weak barriers together beat one strong one that you misconfigure. Rotate keys with an overlap. Create the replacement first, move the client, then delete the old line. The remove command below keeps the file’s inode intact so the bind mount survives. scripts/make-key.sh alice2 switch the client to keys/alice2.txt, then retire the old key: grep -v ' alice;$' nginx/keys-user.map /tmp/keys.tmp cat /tmp/keys.tmp nginx/keys-user.map rm /tmp/keys.tmp keys/alice.txt docker compose exec gateway nginx -s reload Mine the JSON logs. Because the log format writes one JSON object per line with the caller name, a one-line jq filter answers questions like who is being throttled or who is calling model-management routes. docker compose exec -T gateway cat /var/log/nginx/access.log \ | jq -r 'select .status == 429 | .client' | sort | uniq -c | sort -rn docker compose exec -T gateway cat /var/log/nginx/access.log \ | jq -r 'select .uri | test "^/api/ pull|delete|create " | .t, .client, .uri, .status | @tsv' Mind the filesystem on Windows. If you run Docker through WSL2, keep the project inside the Linux filesystem, not under a mounted Windows drive. Permission bits such as chmod 600 do not work reliably on mounted drives, and file-change detection for bind mounts can lag, which makes a freshly added key seem to vanish until you restart the container. Feed repeated 401 lines from one address into fail2ban or a firewall rule if the gateway faces a hostile network. For per-user token budgets and spend tracking rather than request counts, a dedicated LLM proxy such as LiteLLM can sit between your clients and Ollama and keep a ledger per key. That adds a database and another moving part, so adopt it only when a plain request limit stops being enough. Other upgrades worth considering include mutual TLS for machine-to-machine clients, hashing stored tokens Nginx can compare hashes through the njs module or a small auth service , and separate rate zones for chat and embedding calls. For wider background on running and securing models, the AI and machine learning https://shattered.io/category/ai-machine-learning/ section collects our related tutorials. Recent Ollama releases also widened compatibility, since v0.40.14 reportedly improved OpenAI-style endpoint handling including /v1/responses , so rerun the smoke test after each upgrade. The Finished Project at a Glance When every step is done, the folder looks like this. Everything above the keys and certificates folders is safe to commit. ollama-gateway/ docker-compose.yml .env OLLAMA TAG=0.40.14 optional pin .gitignore keys/ certs/ nginx/keys- .map nginx/ nginx.conf proxy.conf keys-user.map keys-admin.map certs/ gateway.crt gateway.key keys/ admin.txt alice.txt scripts/ make-key.sh chat.py stream check.py tests/ smoke test.py | Step | What it produced | How to confirm | |---|---|---| | 1-3 | Folder, Compose file, Ollama container with a model | curl localhost:11434 fails from the host | | 4-5 | Named keys and a TLS certificate | ls -l keys/ shows mode 600 | | 6-7 | Nginx config and proxy settings | nginx -t reports success | | 8-9 | Running gateway, curl calls | 401 without a key, 200 with one | | 10-11 | SDK client and streaming check | Many chunks, early first chunk | | 12 | Automated smoke test | 9/9 checks passed | Run python tests/smoke test.py after every change, keep the key files out of version control, and pin OLLAMA TAG once you have confirmed a stable build. The same gateway then serves a script, an editor and an agent at once, each with its own name in the logs. FAQ Does the local Ollama API have a built-in API key? No. The local server at port 11434 accepts requests without credentials. Ollama API keys apply to the hosted service at ollama.com. For a self-hosted server you add authentication yourself, for example with the Nginx gateway above. Can I use the OpenAI Python SDK with Ollama? Yes. Point base url at the /v1 path, either on localhost or on your gateway, and pass any string as api key for the local server. Behind the gateway, pass the gateway token so the SDK sends it as a Bearer header. How do I get an Ollama API key for ollama.com? Sign in to your Ollama account, open the Ollama Keys page in settings, and create a key. Send it as Authorization: Bearer to https://ollama.com/api , or set the OLLAMA API KEY environment variable for tools that read it. Per the documentation, keys do not expire and you can revoke them from the same page. Is a self-signed certificate good enough? For a home lab or an office LAN, yes, provided clients trust that exact certificate file and the subjectAltName list covers the names they use. For a public hostname, use a real certificate through Let’s Encrypt, either with Caddy or with Certbot in front of Nginx. Will the gateway slow down responses? The extra hop adds little compared with model inference, and streaming still works because buffering is off. Measure with the stream check script before and after any change rather than trusting a general claim. Can I run this without Docker? Yes. Keep Ollama on its default loopback address, install Nginx on the host, and change the upstream to 127.0.0.1:11434 . The key maps, rate limit, TLS and admin rules stay the same. How many keys can the map file hold? Hundreds work fine, since Nginx builds a hash table from the map at load time. Beyond that scale, move identity to an auth service or an identity-aware proxy so you can manage users without editing files. Why block the pull, create and delete endpoints for normal keys? Those calls change what is installed on the server and can fill the disk or remove a model other people depend on. Normal callers only need inference and read-only listing, so the safest default is to reserve model management for an admin key.