{"slug": "testing-omniroute-locally-verify-your-ai-gateway-actually-works-before-you-trust", "title": "Testing OmniRoute locally: verify your AI gateway actually works before you trust it", "summary": "OmniRoute, a local AI gateway that provides a single OpenAI-compatible endpoint for routing requests across hundreds of providers, has been released. The project's developers emphasize that a fresh install must be thoroughly tested for streaming, tool calls, and fallback behavior before being trusted with coding agents. They provide a checklist of curl commands and CLI probes to verify the gateway's functionality, including health checks, model listing, chat completions, streaming, tool calling, and failover testing.", "body_md": "OmniRoute is a local AI gateway: it puts one OpenAI-compatible endpoint (`http://localhost:20128/v1`\n\n) in front of hundreds of providers, then routes requests between them with automatic fallback. Install it, point your coding tool at `auto`\n\n, and it \"just works.\"\n\nBut \"it starts\" and \"it works\" are not the same thing. A gateway that answers `Hello!`\n\nbut falls over on streaming, tool calls, or a provider outage is decorative, not functional. This article is a checklist for proving a fresh local install actually does its job before you wire a coding agent into it.\n\n```\nnpm install -g omniroute\nomniroute\n```\n\nThe server boots on port `20128`\n\n, the dashboard opens at `http://localhost:20128`\n\n, and the API base URL is `http://localhost:20128/v1`\n\n.\n\nIf you use pnpm, add the native build flags:\n\n`pnpm add -g omniroute@latest --allow-build=better-sqlite3 --allow-build=@swc/core`\n\n.\n\nA fresh install is zero-config: the free providers (OpenCode Free, Felo) are pre-wired into the `auto`\n\ncombo, so a brand-new server answers out of the box with no API key.\n\n`omniroute doctor`\n\n)\nOmniRoute ships a health checker that runs without starting the server:\n\n```\nomniroute doctor\nomniroute doctor --json        # machine-readable output\nomniroute doctor --no-liveness # skip live checks\n```\n\nTreat this as the first gate. If `doctor`\n\nreports a broken runtime, native module, or DB issue, fix it before touching the network.\n\nOther useful CLI probes:\n\n```\nomniroute status            # offline dashboard: version, DB, tools, config\nomniroute providers list    # what's actually connected\nomniroute providers validate\n```\n\nA connected provider should show up in `/v1/models`\n\n. If you've created an API key on the **Endpoints** page, query with it:\n\n```\ncurl http://localhost:20128/v1/models \\\n  -H \"Authorization: Bearer YOUR_KEY\"\n```\n\nA non-empty list means the gateway can see providers. But don't stop here — a model list is not a working request.\n\nThe fastest end-to-end proof:\n\n```\ncurl http://localhost:20128/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"auto\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello!\"}]}'\n```\n\nYou can also call a specific free backend directly (e.g. `oc/...`\n\nfor OpenCode Free) to isolate one provider, then graduate to `auto`\n\nand let the router pick.\n\nEach response carries an `X-OmniRoute-Decision`\n\nheader naming the strategy, provider, and latency that served it — use it to confirm routing is doing what you expect.\n\nMany tools depend on streaming. Send the same request with `\"stream\": true`\n\n:\n\n```\ncurl -N http://localhost:20128/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"auto\",\"stream\":true,\"messages\":[{\"role\":\"user\",\"content\":\"Count to ten.\"}]}'\n```\n\nYou should see `data:`\n\nchunks arriving incrementally rather than one blob. A gateway that only works in non-streaming mode will break most agent CLIs.\n\nCoding agents live on tool/function calling. Verify it round-trips:\n\n```\ncurl http://localhost:20128/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"auto\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"What is the weather in Lisbon?\"}],\n    \"tools\": [{\n      \"type\": \"function\",\n      \"function\": {\n        \"name\": \"get_weather\",\n        \"description\": \"Get the current weather\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\"city\": {\"type\": \"string\"}},\n          \"required\": [\"city\"]\n        }\n      }\n    }]\n  }'\n```\n\nA valid reply should include a `tool_calls`\n\nblock with correctly structured arguments, not a plain-text answer pretending to be a tool call.\n\nThe reason you run a gateway is resilience. To prove it:\n\n`X-OmniRoute-Decision`\n\nheader to confirm the primary provider is serving.The request should still succeed, now routed to the next provider — silently, with no error surfaced to your client. If it 500s instead, your fallback chain isn't wired the way you think.\n\nOnce the API surface checks out, connect an actual agent:\n\n```\nBase URL: http://localhost:20128/v1\nAPI Key:  [copy from the Endpoints page]\nModel:    auto\n```\n\nOr let OmniRoute write the config for you, per tool:\n\n```\nomniroute setup-codex      # ~/.codex/<name>.config.toml profiles\nomniroute setup-claude     # ~/.claude/profiles/<name>/settings.json\nomniroute setup-opencode   # opencode.json\nomniroute setup-cursor     # prints Cursor's in-app steps\n```\n\nEven simpler — launch a CLI through the gateway with no config written:\n\n```\nomniroute run claude   --model auto\nomniroute run codex    --model auto\n```\n\nThen do a real task (edit a file, run a tool) rather than a hello-world. A finished task proves the whole pipeline.\n\n| Check | Pass condition |\n|---|---|\n`omniroute doctor` |\nNo failing checks |\n`curl /v1/models` |\nProviders listed |\n| Plain chat | Valid completion + `X-OmniRoute-Decision` header |\n| Streaming | Incremental `data:` chunks |\n| Tool calling | Structured `tool_calls` in the response |\n| Fallback | Request survives a dead primary provider |\n| Real agent task | A coding CLI completes an actual edit |\n\n`doctor`\n\ncomplains about a native module`STREAM_IDLE_TIMEOUT_MS`\n\n.Ten minutes of this checklist beats an hour of debugging why your agent silently \"can't reach the model\" in the middle of real work.", "url": "https://wpnews.pro/news/testing-omniroute-locally-verify-your-ai-gateway-actually-works-before-you-trust", "canonical_source": "https://dev.to/ahmed_nafies_3a55c907115c/testing-omniroute-locally-verify-your-ai-gateway-actually-works-before-you-trust-it-1pde", "published_at": "2026-08-25 18:11:45+00:00", "updated_at": "2026-08-25 18:44:13.310359+00:00", "lang": "en", "topics": ["ai-infrastructure", "developer-tools", "ai-products"], "entities": ["OmniRoute", "OpenCode Free", "Felo"], "alternates": {"html": "https://wpnews.pro/news/testing-omniroute-locally-verify-your-ai-gateway-actually-works-before-you-trust", "markdown": "https://wpnews.pro/news/testing-omniroute-locally-verify-your-ai-gateway-actually-works-before-you-trust.md", "text": "https://wpnews.pro/news/testing-omniroute-locally-verify-your-ai-gateway-actually-works-before-you-trust.txt", "jsonld": "https://wpnews.pro/news/testing-omniroute-locally-verify-your-ai-gateway-actually-works-before-you-trust.jsonld"}}