Keystone 1.0: what stable means for a self-hosted LLM gateway Keystone. The Model Switchboard. (KTMS) reached version 1.0 alongside its ktms-admin web console and light-tokenmeter usage meter, with the Go-based, Apache 2.0-licensed gateway now guaranteeing its API won't break within the 1.x line. Keystone routes service requests by effort level (LOW, MEDIUM or HIGH) plus attributes through prioritized, runtime-editable rules to model fallback chains, keeping vendor API keys encrypted in Postgres with AES-256-GCM, and idles at 7 MiB of memory. The current release, 1.3, adds a node agent for Ollama, vLLM, TGI and llama.cpp, an OpenAI-compatible HTTP API and typed gRPC ktms.v1, and an MCP server via `ktms mcp`. Keystone 1.0: what stable means for a self-hosted LLM gateway Keystone. The Model Switchboard. KTMS reaches 1.0 together with ktms-admin and light-tokenmeter. What the gateway does, what is now stable across 1.x, how we test that promise, and what changed on the way. A model name in your code goes stale the day the next model ships. With ten services, you rewrite it ten times, and spread vendor keys across ten configs. Keystone. The Model Switchboard. KTMS takes that out of your code. A service says how hard the task is; the gateway decides which model to call, with which key, and whether your own GPU should take it. It idles at 7 MiB of memory, and from 1.0 on its API doesn’t break within 1.x. The web console ktms-admin and the usage meter light-tokenmeter reach 1.0 with it. All three are written in Go and licensed under Apache 2.0. The features below are as of the current 1.3. What Keystone does Keystone sits between your services and your model vendors. Your services call one endpoint, and KTMS: - keeps the vendor API keys, encrypted in Postgres with AES-256-GCM, so clients never see them; - picks the model from rules, not from a name hardcoded in your code; - falls back to the next model when a vendor fails or throttles; - enforces requests per minute, daily token budgets and hard spend caps per key, user and team; - sends one usage event per call to a sink such as light-tokenmeter. Effort instead of model names A service doesn’t ask for gpt-4o . It asks for an effort, LOW , MEDIUM or HIGH , and adds free-form attributes such as language or task . An ordered list of rules maps that to a chain of models: {"rules": {"name":"spanish-high", "priority":30, "effort":"EFFORT HIGH", "attributes":{"language":"es"}, "use": "gemini-2.5-pro","gpt-4o" , "enabled":true}, {"name":"low-default", "priority":20, "effort":"EFFORT LOW", "use": "deepseek-chat","gpt-4o-mini" , "enabled":true}, {"name":"anything-else", "priority":1, "use": "gpt-4o-mini" , "enabled":true} } Rules change at runtime over the admin API or the console. When a new model comes out, you edit a rule, not forty services. Two ways in Typed gRPC ktms.v1 for services, and an OpenAI-compatible HTTP API for everything that already speaks OpenAI: python from openai import OpenAI client = OpenAI base url="http://localhost:8080/v1", api key="ktms