{"slug": "set-up-an-llm-gateway-by-asking-your-coding-agent-the-keystone-plugin-for-claude", "title": "Set up an LLM gateway by asking your coding agent: the Keystone plugin for Claude Code", "summary": "Keystone shipped a Claude Code plugin that installs with one command and provides eight skills plus two MCP servers for deploying and integrating its LLM gateway, with the /ktms:assess skill reading a repository to map every LLM call site by file and line before any change is made. On a three-service demo project, /ktms:assess found all five call sites, flagged that every service loads all three vendor keys through env_file, that temperature=0.3 in draft_reply is ignored at /v1, and that rerank.ts does not rerank at all, while identifying the nightly worker calling Gemini about 40,000 times a night as the largest cost and risk. The plugin's version tracks Keystone's release, and the skills call the ktms CLI, which must be on PATH from a release archive or via go install github.com/ai-process/ktms/cmd/ktms@latest.", "body_md": "# Set up an LLM gateway by asking your coding agent: the Keystone plugin for Claude Code\n\nOne command installs the Keystone plugin. Then /ktms:assess reads your repository and says where the gateway goes, /ktms:deploy writes the deployment and /ktms:integrate moves a service over, without a single key passing through the chat.\n\nPutting a gateway in front of your LLM calls is not hard, but it touches a lot of places: a Compose file or a Helm release, secrets, a client module in every service, a model value at every call site, limits and routing rules. Ask a general-purpose coding agent to do it and it will happily invent environment variables that don’t exist.\n\nSo Keystone ships a Claude Code plugin: eight skills that know the real commands, flags and config fields, and two MCP servers that let the agent read what is running. This post walks through it on a small demo project.\n\n## 1. Install\n\n```\nclaude plugin marketplace add ai-process/ktms && claude plugin install ktms@ai-process\n✔ Successfully added marketplace: ai-process (declared in user settings)\nInstalling plugin \"ktms@ai-process\"...✔ Successfully installed plugin: ktms@ai-process (scope: user)\n```\n\nThe plugin’s version is Keystone’s release. The skills call the `ktms` CLI, so put it on `PATH`\nfrom a [release archive](https://github.com/ai-process/ktms/releases) or with\n`go install github.com/ai-process/ktms/cmd/ktms@latest`.\n\n| Skill | For | \n|---|---|\n| `/ktms:assess` | Where Keystone fits the system you run in, and the plan to get there | \n| `/ktms:deploy` | Compose or Helm for the stack, and the first-start runbook | \n| `/ktms:integrate` | Moving a service’s LLM calls onto Keystone | \n| `/ktms:configure` | Routing, keys and limits as a ktms config file, planned and applied | \n| `/ktms:migrate` | Moving from LiteLLM or vendor SDKs | \n| `/ktms:node` | A local GPU node with cloud fallback | \n| `/ktms:usage-reports` | Spend questions, reports and alerts from light-tokenmeter | \n| `/ktms:production-review` | Checking a deployment before and after it goes live | \n\nYou don’t have to remember the names: the agent picks a skill when the task fits, and the slash command starts one directly.\n\n## 2. Assess: where does the gateway go?\n\nThe demo project has three services: a support bot on the OpenAI SDK with `model=\"gpt-4o\"`\nhardcoded twice, a search API on the Anthropic SDK plus OpenAI embeddings, and a nightly worker\nthat calls Gemini about 40,000 times a night. Postgres, Redis and Prometheus are already in the\nCompose file.\n\n`/ktms:assess` changes nothing. It reads the code and the deployment, lists every LLM call site\nwith file and line, and reports where Keystone would run, what each service changes, what it\nbrings and costs here, and which skill to run next. It also says plainly when Keystone is not\nworth it, for example for one service with one model and no limits to keep.\n\nThe answer is a report of about a hundred lines. It found all five call sites with file and line, and a few things nobody asked about:\n\nEvery service loads it through `env_file`, so each one gets all three vendor keys.\n\n`temperature=0.3` in `draft_reply`:`/v1` ignores it, so check reply quality after the switch.\n\nIt also noticed that `rerank.ts` doesn’t rerank at all, only explains results, and that the\nnightly worker is where most of the money and risk is. This is where it puts the gateway:\n\n```\n  Compose project \"demo-shop\" (one network, nothing exposed to the host)\n  ┌─────────────────────────────────────────────────────────────────┐\n  │  support-bot ─┐                                                 │\n  │  search-api ──┼── HTTP :8080 /v1 ──▶  ktms ──▶ OpenAI           │\n  │  digest-worker┘   (gRPC :9090 later        ├─▶ Anthropic        │\n  │                    for batch)              └─▶ Gemini           │\n  │                                  │  │  │                        │\n  │      postgres:16 ◀── db \"ktms\" ──┘  │  └── :9091 ◀── prometheus │\n  │      redis:7 ◀──── limits ──────────┘                           │\n  │      light-tokenmeter ◀── usage events (spend per action_id)    │\n  │      ktms-admin  (internal only; reach it via SSH tunnel/VPN)   │\n  └─────────────────────────────────────────────────────────────────┘\n```\n\nThe verdict comes with a condition:\n\n**Verdict:** worth it. If the system were only support-bot on `gpt-4o`, it wouldn’t be.\n\nThe plan that follows starts with the nightly worker, because it has one call site, isn’t user-facing and a failed night can simply be re-run.\n\n## 3. Deploy and configure, with your keys out of the chat\n\n`/ktms:deploy` asks first: Compose on one host or Kubernetes with Helm, managed or in-cluster\nPostgres and Redis, where secrets live, hostnames, monitoring. Then it writes the files with every\nsecret as a reference to your store, probes on `/readyz`, and a first-start runbook.\n\nThe runbook has one step the agent leaves to you on purpose: `ktms init` in your own terminal.\nIt asks for the admin key and the vendor keys without echo, so they never reach the chat, writes\n`ktms.yaml` with default models and rules, shows the plan and applies it.\n\nFrom then on `/ktms:configure` works on that file, like Terraform: edit, `ktms config plan`, show\nthe plan, apply only after you agree. The guard rails are in the tools, not in the prompt:\n\n- The `ktms` MCP server is**read-only by default** . Write tools exist only if you add`--allow-write` to its arguments.\n- `config_plan` returns a`plan_id` ;`config_apply` applies exactly that plan, once, and refuses\nif Keystone changed in between.\n- A vendor key in a plan is read from the MCP server’s environment, and only from variables named\n`KTMS_VENDOR_KEY_*` , so the agent cannot point a config at the admin key or another secret.\n- Names and values that come back from Keystone are marked as data, not instructions.\n\n## 4. Integrate one service\n\n`/ktms:integrate` starts by listing call sites and changing nothing. Then it puts the base URL,\nkey, timeouts, retries and attribution in one small client module, and replaces each hardcoded\nmodel with an effort, with a one-line reason per call site: low for classification and short\nanswers, medium for ordinary generation, high for reasoning and long documents.\n\nIt also turns off the SDK’s own retries and applies Keystone’s error contract once (wait out\n`Retry-After` on 429, back off on 503, never retry a configuration error), and flags what doesn’t\nmap: embeddings keep an exact model id, and `temperature`, `top_p`, `stop` and `seed` are ignored\non Keystone’s OpenAI-compatible API.\n\nThe check at the end uses real routing, not a guess:\n\n``` bash\n$ ktms route explain billing --effort low\nrule: default   effort: low   route mode: direct\n#  MODEL             VENDOR  STATE\n1  gpt-4o-mini       openai  kept\n2  gemini-2.5-flash  google  dropped: no_vendor_key\n3  gpt-4o            openai  dropped: effort\ntries: gpt-4o-mini\n```\n\nThat output is from Keystone 1.3.6: a chain that ends in a kept model is fine; one that ends in\n`no_vendor_key` means a key is missing before the service ever goes live.\n\n## 5. Why the skills don’t drift\n\nA skill that names a flag removed two releases ago sends the agent into an error, and the agent\nwill then improvise. So Keystone’s CI reads every `SKILL.md` and fails when one names a `ktms`\ncommand, a flag, an MCP tool or a README section that doesn’t exist in that version.\n\n## Other agents\n\nThe skills are plain Agent Skills, a folder with a `SKILL.md`: copy `plugin/skills/*` into the\nskills directory of Codex, Cursor or another agent that reads them, and add the two MCP servers to\nits config. Assistants without skills or MCP can use the same instructions as\n[prompts](https://ai-process.dev/docs/prompts/).\n\n- Plugin: [github.com/ai-process/ktms/tree/main/plugin](https://github.com/ai-process/ktms/tree/main/plugin)\n- Keystone: [github.com/ai-process/ktms](https://github.com/ai-process/ktms)\n- Release notes: [Keystone 1.0](https://blog.ai-process.dev/2026/10/27/keystone-1-0/)", "url": "https://wpnews.pro/news/set-up-an-llm-gateway-by-asking-your-coding-agent-the-keystone-plugin-for-claude", "canonical_source": "https://blog.ai-process.dev/2026/11/04/set-up-keystone-from-claude-code/", "published_at": "2026-11-04 07:00:00+00:00", "updated_at": "2026-11-04 18:50:56.035326+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "developer-tools", "large-language-models", "ai-infrastructure"], "entities": ["Keystone", "Claude Code", "ktms", "ai-process", "OpenAI", "Anthropic", "Gemini", "LiteLLM"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/set-up-an-llm-gateway-by-asking-your-coding-agent-the-keystone-plugin-for-claude", "markdown": "https://wpnews.pro/news/set-up-an-llm-gateway-by-asking-your-coding-agent-the-keystone-plugin-for-claude.md", "text": "https://wpnews.pro/news/set-up-an-llm-gateway-by-asking-your-coding-agent-the-keystone-plugin-for-claude.txt", "jsonld": "https://wpnews.pro/news/set-up-an-llm-gateway-by-asking-your-coding-agent-the-keystone-plugin-for-claude.jsonld"}}