{"slug": "run-the-grok-cli-on-ollama-cloud-and-custom-providers", "title": "Run the Grok CLI on Ollama Cloud and custom providers", "summary": "XAI's open-source terminal AI coding agent grok now supports custom model endpoints, allowing users to run models like GLM 5.2, DeepSeek V4 Pro, and Kimi K2 via Ollama Cloud or any OpenAI-compatible provider by editing a single config file. The feature, documented in a blog post by xAI, uses a ~/.grok/config.toml file where users specify model id, base URL, authentication, and protocol, with a critical caveat that dots in model names must be quoted in TOML headers to avoid parsing errors.", "body_md": "## What this is\n\n`grok`\n\nis the open-source terminal AI coding agent from xAI - a full-screen TUI that edits files, runs shell commands, searches the web, and drives long tasks, headless or interactive. The source lives at [grok-build](https://github.com/xai-org/grok-build); the released binary installs as `grok`\n\n. Out of the box it talks to grok.com. The interesting part is that the model layer is **bring-your-own-endpoint**: grok speaks three wire protocols - OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages - and any `base_url`\n\n+ model id + key triple works. There is no per-vendor SDK and no provider enum. You point it at a provider with one config file edit, no rebuild, no code.\n\nThe practical payoff: a single Ollama Cloud key gives you [GLM 5.2](/models/glm-5-2), [DeepSeek V4 Pro](/models/deepseek-v4-pro) and [V4 Flash](/models/deepseek-v4-flash), [Kimi K2](/models/kimi-k2), GPT-OSS, Qwen, and more through one OpenAI-compatible endpoint - and the same mechanism adds local Ollama, OpenRouter, Moonshot, Anthropic, or any vLLM/llama.cpp server. Switch between them live with `Ctrl+M`\n\n.\n\n*grok driving a session on GLM 5.2 via Ollama Cloud - the model picker, scrollback, and prompt all work as they do with the default grok.com backend.*\n\n## The one file: ~/.grok/config.toml\n\nCustom models live in `~/.grok/config.toml`\n\nunder `[model.<id>]`\n\nsections. Each entry is a small struct: the model id to send the API, the endpoint, how to authenticate, which protocol to use, and the context window. A minimal entry looks like this:\n\n```\n[model.\"glm-5.2\"]\nmodel = \"glm-5.2\"\nbase_url = \"https://ollama.com/v1\"\nname = \"GLM 5.2 (Ollama Cloud)\"\nenv_key = \"OLLAMA_API_KEY\"\napi_backend = \"chat_completions\"\ncontext_window = 200000\n```\n\nThen export the key in your shell and run grok against it:\n\n```\necho 'export OLLAMA_API_KEY=\"your-key-here\"' >> ~/.zshrc\nsource ~/.zshrc\ngrok -m glm-5.2\n```\n\nThat is the whole interaction. The key is read from the environment, never written into the config file, and grok sends it as `Authorization: Bearer`\n\non every request to `https://ollama.com/v1/chat/completions`\n\n.\n\nOllama Cloud’s OpenAI-compatible endpoint is `https://ollama.com/v1`\n\n(the `api.ollama.com`\n\nhost just 301-redirects there). Its catalog at the time of writing includes glm-5.2, glm-5.1, deepseek-v4-pro, deepseek-v4-flash, kimi-k2.7-code, kimi-k2.6, gpt-oss:120b, gpt-oss:20b, mistral-large-3, qwen3.5, and the Nemotron and MiniMax families. List them with `curl -H \"Authorization: Bearer $OLLAMA_API_KEY\" https://ollama.com/v1/models`\n\n.\n\n## The dot gotcha (the part that bites everyone)\n\nTOML treats dots in a table header as nested keys. `[model.glm-5.2]`\n\nparses as `model`\n\n-> `glm-5`\n\n-> `2`\n\n, so grok registers the model id as `glm-5`\n\n, not `glm-5.2`\n\n. `grok models`\n\nwill list `glm-5`\n\n(truncated), and `grok -m glm-5.2`\n\nwill fail with `unknown model id`\n\n. The fix is to quote any id that contains a dot:\n\n```\n[model.\"glm-5.2\"]       # correct - id stays \"glm-5.2\"\n[model.glm-5.2]         # wrong - parses as nested, id becomes \"glm-5\"\n```\n\nThe same applies to `kimi-k2.7-code`\n\n, `qwen3.5`\n\n, and anything else with a dot. Colons in the `model = \"...\"`\n\nvalue are fine - only the dots in the **section name** need quoting. If you see `unknown model id`\n\non a model you definitely configured, this is why.\n\n## Fields, briefly\n\n-\n- the id sent to the API (e.g.`model`\n\n`glm-5.2`\n\n,`gpt-oss:120b`\n\n). -\n- the OpenAI-compatible root, no trailing path. grok appends`base_url`\n\n`/chat/completions`\n\n,`/responses`\n\n, or`/messages`\n\n. -\n- the label shown in the model picker.`name`\n\n-\n- the key.`api_key`\n\nor`env_key`\n\n`env_key`\n\ntakes a string or an array of env-var names; the first non-empty one wins. The array form is useful for SSH`LC_*`\n\nforwarding (e.g.`env_key = [\"ANTHROPIC_AUTH_TOKEN\", \"LC_ANTHROPIC_AUTH_TOKEN\"]`\n\n). Prefer`env_key`\n\nover`api_key`\n\nso the secret stays out of the file. -\n-`api_backend`\n\n`chat_completions`\n\n(default),`responses`\n\n, or`messages`\n\n(Anthropic). -\n- total tokens; controls when grok triggers auto-compaction. If you omit it on a new model, grok assumes 200k - set it to match the real provider.`context_window`\n\n-\n- sent verbatim on every request. Use this for Anthropic’s`extra_headers`\n\n`x-api-key`\n\nand`anthropic-version`\n\n, or for proxy attribution tags. -\n- sampling knobs.`temperature`\n\n,`top_p`\n\n,`max_completion_tokens`\n\nCredential resolution order, highest first: the `api_key`\n\nfield, then `env_key`\n\n, then your `grok login`\n\nsession token, then the `XAI_API_KEY`\n\nenvironment variable. Precedence for the model itself: CLI flag `-m`\n\n> env var > `config.toml`\n\n> remote `/v1/models`\n\nlist > hardcoded defaults.\n\n## Other providers, same pattern\n\nBecause every OpenAI-compatible server speaks the same protocol, adding a provider is one section. None of these require code changes or a rebuild.\n\nLocal Ollama (no key, your own machine):\n\n```\n[model.local-llama]\nmodel = \"llama3.1\"\nbase_url = \"http://localhost:11434/v1\"\nname = \"Llama 3.1 (local)\"\n```\n\nOpenRouter (one key, hundreds of models - use their `/models`\n\nendpoint to find ids):\n\n```\n[model.\"claude-sonnet\"]\nmodel = \"anthropic/claude-sonnet-4\"\nbase_url = \"https://openrouter.ai/api/v1\"\nname = \"Claude Sonnet (OpenRouter)\"\nenv_key = \"OPENROUTER_API_KEY\"\ncontext_window = 200000\n```\n\nDeepSeek direct:\n\n```\n[model.\"deepseek-chat\"]\nmodel = \"deepseek-chat\"\nbase_url = \"https://api.deepseek.com/v1\"\nname = \"DeepSeek Chat\"\nenv_key = \"DEEPSEEK_API_KEY\"\ncontext_window = 64000\n```\n\nMoonshot / Kimi direct:\n\n```\n[model.\"moonshot-128k\"]\nmodel = \"moonshot-v1-128k\"\nbase_url = \"https://api.moonshot.cn/v1\"\nname = \"Kimi (Moonshot)\"\nenv_key = \"MOONSHOT_API_KEY\"\ncontext_window = 128000\n```\n\nAnthropic direct (note the `messages`\n\nbackend and the two required headers):\n\n```\n[model.\"claude-opus\"]\nmodel = \"claude-opus-4\"\nbase_url = \"https://api.anthropic.com/v1\"\nname = \"Claude Opus\"\napi_backend = \"messages\"\ncontext_window = 200000\nextra_headers = { \"x-api-key\" = \"sk-ant-REPLACE\", \"anthropic-version\" = \"2023-06-01\" }\n```\n\nFor Anthropic, put the key in `extra_headers`\n\n(or reference an env var there) because Anthropic authenticates with `x-api-key`\n\n, not `Authorization: Bearer`\n\n. Any vLLM, llama.cpp, or other OpenAI-compatible server is just a `base_url`\n\nchange.\n\n## Switching models\n\nPick a model for a session with the CLI flag, or switch mid-session in the TUI:\n\n```\ngrok -m glm-5.2 -p \"fix the failing test in src/auth.rs\"   # headless\ngrok -m kimi-k2.7-code                                       # start TUI on a model\n```\n\nInside the TUI: `/model glm-5.2`\n\n(or the `/m`\n\nalias), or `Ctrl+M`\n\nto open the picker, which lists every built-in and custom model. Set a persistent default so every new session starts on it:\n\n```\n[models]\ndefault = \"glm-5.2\"\n```\n\n`grok models`\n\nprints the full catalog. Global defaults that apply to every model - `temperature`\n\n, `top_p`\n\n, `max_retries`\n\n, `extra_headers`\n\n- go under a single `[models]`\n\nsection instead of being repeated per entry.\n\n## Keep the key out of the file\n\nThe single most important hygiene rule: **never paste an API key into config.toml**. The file gets backed up, synced across machines, and occasionally committed by accident. Use\n\n`env_key`\n\nand put the export in your shell profile. If a key has been shared in a chat, pasted into a ticket, or checked into a repo, rotate it at the provider’s dashboard and treat the old one as burned.Two refinements: if you would rather not put secrets in `~/.zshrc`\n\neither, a secrets manager that exports them on shell start works, and for first-party xAI access `grok login`\n\nstores a session token in `~/.grok/auth.json`\n\nso you don’t need an API key at all for the default provider.\n\n## Verify it\n\nThree checks, in order:\n\n-\n`grok models`\n\nlists your entries with the full id (e.g.`glm-5.2`\n\n, not`glm-5`\n\n). If it shows a truncated id, you hit the dot gotcha - quote the section name. -\n`grok -m glm-5.2 -p \"reply with one word: pong\"`\n\nreturns`pong`\n\n. This confirms endpoint, auth, and the chat-completions path end to end. - If you get an auth error, the env var is not set in the shell you launched grok from -\n`source ~/.zshrc`\n\nor open a new terminal.\n\n*The custom Ollama Cloud models - glm-5.2, deepseek-v4-pro, kimi-k2.7-code, and the rest - appearing in grok’s model list alongside the built-in defaults, ready to switch to with /model or Ctrl+M.*\n\n## The bigger picture\n\nThis works on the released `grok`\n\nbinary and on the open-source [grok-build](https://github.com/xai-org/grok-build) fork alike - they share `~/.grok/config.toml`\n\n. The provider layer is intentionally provider-agnostic: three wire protocols, generalized `Bearer`\n\n/ `x-api-key`\n\nauth, and arbitrary extra headers. Adding a new provider is a config entry, not a code change. New wire protocols - something that is not Chat Completions, Responses, or Messages - would need a backend variant plus a stream transform in the sampler crate, but for the vast majority of providers, including every one on this page, you never touch Rust.\n\nFor model-level detail on the candidates above - throughput, context, pricing - see [GLM 5.2](/models/glm-5-2), [DeepSeek V4 Pro](/models/deepseek-v4-pro), [DeepSeek V4 Flash](/models/deepseek-v4-flash), [Kimi K2](/models/kimi-k2), and [Nemotron 3 Ultra](/models/nemotron-3-ultra) in the catalog.", "url": "https://wpnews.pro/news/run-the-grok-cli-on-ollama-cloud-and-custom-providers", "canonical_source": "https://tokenstead.ai/guides/run-grok-cli-on-ollama-cloud-and-custom-providers", "published_at": "2026-07-20 13:12:56+00:00", "updated_at": "2026-07-20 13:12:59.806286+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "developer-tools", "large-language-models"], "entities": ["xAI", "Ollama Cloud", "GLM 5.2", "DeepSeek V4 Pro", "Kimi K2", "GPT-OSS", "Qwen"], "alternates": {"html": "https://wpnews.pro/news/run-the-grok-cli-on-ollama-cloud-and-custom-providers", "markdown": "https://wpnews.pro/news/run-the-grok-cli-on-ollama-cloud-and-custom-providers.md", "text": "https://wpnews.pro/news/run-the-grok-cli-on-ollama-cloud-and-custom-providers.txt", "jsonld": "https://wpnews.pro/news/run-the-grok-cli-on-ollama-cloud-and-custom-providers.jsonld"}}