Testing OmniRoute locally: verify your AI gateway actually works before you trust it

OmniRoute is a local AI gateway: it puts one OpenAI-compatible endpoint (http://localhost:20128/v1

) in front of hundreds of providers, then routes requests between them with automatic fallback. Install it, point your coding tool at auto

, and it "just works."

But "it starts" and "it works" are not the same thing. A gateway that answers Hello!

but falls over on streaming, tool calls, or a provider outage is decorative, not functional. This article is a checklist for proving a fresh local install actually does its job before you wire a coding agent into it.

npm install -g omniroute
omniroute

The server boots on port 20128

, the dashboard opens at http://localhost:20128

, and the API base URL is http://localhost:20128/v1

.

If you use pnpm, add the native build flags:

pnpm add -g omniroute@latest --allow-build=better-sqlite3 --allow-build=@swc/core

.

A fresh install is zero-config: the free providers (OpenCode Free, Felo) are pre-wired into the auto

combo, so a brand-new server answers out of the box with no API key.

omniroute doctor

) OmniRoute ships a health checker that runs without starting the server:

omniroute doctor
omniroute doctor --json        # machine-readable output
omniroute doctor --no-liveness # skip live checks

Treat this as the first gate. If doctor

reports a broken runtime, native module, or DB issue, fix it before touching the network.

Other useful CLI probes:

omniroute status            # offline dashboard: version, DB, tools, config
omniroute providers list    # what's actually connected
omniroute providers validate

A connected provider should show up in /v1/models

. If you've created an API key on the Endpoints page, query with it:

curl http://localhost:20128/v1/models \
  -H "Authorization: Bearer YOUR_KEY"

A non-empty list means the gateway can see providers. But don't stop here — a model list is not a working request.

The fastest end-to-end proof:

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'

You can also call a specific free backend directly (e.g. oc/...

for OpenCode Free) to isolate one provider, then graduate to auto

and let the router pick.

Each response carries an X-OmniRoute-Decision

header naming the strategy, provider, and latency that served it — use it to confirm routing is doing what you expect.

Many tools depend on streaming. Send the same request with "stream": true

:

curl -N http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","stream":true,"messages":[{"role":"user","content":"Count to ten."}]}'

You should see data:

chunks arriving incrementally rather than one blob. A gateway that only works in non-streaming mode will break most agent CLIs.

Coding agents live on tool/function calling. Verify it round-trips:

curl http://localhost:20128/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "What is the weather in Lisbon?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }]
  }'

A valid reply should include a tool_calls

block with correctly structured arguments, not a plain-text answer pretending to be a tool call.

The reason you run a gateway is resilience. To prove it:

X-OmniRoute-Decision

header to confirm the primary provider is serving.The request should still succeed, now routed to the next provider — silently, with no error surfaced to your client. If it 500s instead, your fallback chain isn't wired the way you think.

Once the API surface checks out, connect an actual agent:

Base URL: http://localhost:20128/v1
API Key:  [copy from the Endpoints page]
Model:    auto

Or let OmniRoute write the config for you, per tool:

omniroute setup-codex      # ~/.codex/<name>.config.toml profiles
omniroute setup-claude     # ~/.claude/profiles/<name>/settings.json
omniroute setup-opencode   # opencode.json
omniroute setup-cursor     # prints Cursor's in-app steps

Even simpler — launch a CLI through the gateway with no config written:

omniroute run claude   --model auto
omniroute run codex    --model auto

Then do a real task (edit a file, run a tool) rather than a hello-world. A finished task proves the whole pipeline.

doctor

complains about a native moduleSTREAM_IDLE_TIMEOUT_MS

.Ten minutes of this checklist beats an hour of debugging why your agent silently "can't reach the model" in the middle of real work.