Run the Grok CLI on Ollama Cloud and custom providers

XAI's open-source terminal AI coding agent grok now supports custom model endpoints, allowing users to run models like GLM 5.2, DeepSeek V4 Pro, and Kimi K2 via Ollama Cloud or any OpenAI-compatible provider by editing a single config file. The feature, documented in a blog post by xAI, uses a ~/.grok/config.toml file where users specify model id, base URL, authentication, and protocol, with a critical caveat that dots in model names must be quoted in TOML headers to avoid parsing errors.

What this is grok is the open-source terminal AI coding agent from xAI - a full-screen TUI that edits files, runs shell commands, searches the web, and drives long tasks, headless or interactive. The source lives at grok-build https://github.com/xai-org/grok-build ; the released binary installs as grok . Out of the box it talks to grok.com. The interesting part is that the model layer is bring-your-own-endpoint : grok speaks three wire protocols - OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages - and any base url + model id + key triple works. There is no per-vendor SDK and no provider enum. You point it at a provider with one config file edit, no rebuild, no code. The practical payoff: a single Ollama Cloud key gives you GLM 5.2 /models/glm-5-2 , DeepSeek V4 Pro /models/deepseek-v4-pro and V4 Flash /models/deepseek-v4-flash , Kimi K2 /models/kimi-k2 , GPT-OSS, Qwen, and more through one OpenAI-compatible endpoint - and the same mechanism adds local Ollama, OpenRouter, Moonshot, Anthropic, or any vLLM/llama.cpp server. Switch between them live with Ctrl+M . grok driving a session on GLM 5.2 via Ollama Cloud - the model picker, scrollback, and prompt all work as they do with the default grok.com backend. The one file: ~/.grok/config.toml Custom models live in ~/.grok/config.toml under model.<id sections. Each entry is a small struct: the model id to send the API, the endpoint, how to authenticate, which protocol to use, and the context window. A minimal entry looks like this: model."glm-5.2" model = "glm-5.2" base url = "https://ollama.com/v1" name = "GLM 5.2 Ollama Cloud " env key = "OLLAMA API KEY" api backend = "chat completions" context window = 200000 Then export the key in your shell and run grok against it: echo 'export OLLAMA API KEY="your-key-here"' ~/.zshrc source ~/.zshrc grok -m glm-5.2 That is the whole interaction. The key is read from the environment, never written into the config file, and grok sends it as Authorization: Bearer on every request to https://ollama.com/v1/chat/completions . Ollama Cloud’s OpenAI-compatible endpoint is https://ollama.com/v1 the api.ollama.com host just 301-redirects there . Its catalog at the time of writing includes glm-5.2, glm-5.1, deepseek-v4-pro, deepseek-v4-flash, kimi-k2.7-code, kimi-k2.6, gpt-oss:120b, gpt-oss:20b, mistral-large-3, qwen3.5, and the Nemotron and MiniMax families. List them with curl -H "Authorization: Bearer $OLLAMA API KEY" https://ollama.com/v1/models . The dot gotcha the part that bites everyone TOML treats dots in a table header as nested keys. model.glm-5.2 parses as model - glm-5 - 2 , so grok registers the model id as glm-5 , not glm-5.2 . grok models will list glm-5 truncated , and grok -m glm-5.2 will fail with unknown model id . The fix is to quote any id that contains a dot: model."glm-5.2" correct - id stays "glm-5.2" model.glm-5.2 wrong - parses as nested, id becomes "glm-5" The same applies to kimi-k2.7-code , qwen3.5 , and anything else with a dot. Colons in the model = "..." value are fine - only the dots in the section name need quoting. If you see unknown model id on a model you definitely configured, this is why. Fields, briefly - - the id sent to the API e.g. model glm-5.2 , gpt-oss:120b . - - the OpenAI-compatible root, no trailing path. grok appends base url /chat/completions , /responses , or /messages . - - the label shown in the model picker. name - - the key. api key or env key env key takes a string or an array of env-var names; the first non-empty one wins. The array form is useful for SSH LC forwarding e.g. env key = "ANTHROPIC AUTH TOKEN", "LC ANTHROPIC AUTH TOKEN" . Prefer env key over api key so the secret stays out of the file. - - api backend chat completions default , responses , or messages Anthropic . - - total tokens; controls when grok triggers auto-compaction. If you omit it on a new model, grok assumes 200k - set it to match the real provider. context window - - sent verbatim on every request. Use this for Anthropic’s extra headers x-api-key and anthropic-version , or for proxy attribution tags. - - sampling knobs. temperature , top p , max completion tokens Credential resolution order, highest first: the api key field, then env key , then your grok login session token, then the XAI API KEY environment variable. Precedence for the model itself: CLI flag -m env var config.toml remote /v1/models list hardcoded defaults. Other providers, same pattern Because every OpenAI-compatible server speaks the same protocol, adding a provider is one section. None of these require code changes or a rebuild. Local Ollama no key, your own machine : model.local-llama model = "llama3.1" base url = "http://localhost:11434/v1" name = "Llama 3.1 local " OpenRouter one key, hundreds of models - use their /models endpoint to find ids : model."claude-sonnet" model = "anthropic/claude-sonnet-4" base url = "https://openrouter.ai/api/v1" name = "Claude Sonnet OpenRouter " env key = "OPENROUTER API KEY" context window = 200000 DeepSeek direct: model."deepseek-chat" model = "deepseek-chat" base url = "https://api.deepseek.com/v1" name = "DeepSeek Chat" env key = "DEEPSEEK API KEY" context window = 64000 Moonshot / Kimi direct: model."moonshot-128k" model = "moonshot-v1-128k" base url = "https://api.moonshot.cn/v1" name = "Kimi Moonshot " env key = "MOONSHOT API KEY" context window = 128000 Anthropic direct note the messages backend and the two required headers : model."claude-opus" model = "claude-opus-4" base url = "https://api.anthropic.com/v1" name = "Claude Opus" api backend = "messages" context window = 200000 extra headers = { "x-api-key" = "sk-ant-REPLACE", "anthropic-version" = "2023-06-01" } For Anthropic, put the key in extra headers or reference an env var there because Anthropic authenticates with x-api-key , not Authorization: Bearer . Any vLLM, llama.cpp, or other OpenAI-compatible server is just a base url change. Switching models Pick a model for a session with the CLI flag, or switch mid-session in the TUI: grok -m glm-5.2 -p "fix the failing test in src/auth.rs" headless grok -m kimi-k2.7-code start TUI on a model Inside the TUI: /model glm-5.2 or the /m alias , or Ctrl+M to open the picker, which lists every built-in and custom model. Set a persistent default so every new session starts on it: models default = "glm-5.2" grok models prints the full catalog. Global defaults that apply to every model - temperature , top p , max retries , extra headers - go under a single models section instead of being repeated per entry. Keep the key out of the file The single most important hygiene rule: never paste an API key into config.toml . The file gets backed up, synced across machines, and occasionally committed by accident. Use env key and put the export in your shell profile. If a key has been shared in a chat, pasted into a ticket, or checked into a repo, rotate it at the provider’s dashboard and treat the old one as burned.Two refinements: if you would rather not put secrets in ~/.zshrc either, a secrets manager that exports them on shell start works, and for first-party xAI access grok login stores a session token in ~/.grok/auth.json so you don’t need an API key at all for the default provider. Verify it Three checks, in order: - grok models lists your entries with the full id e.g. glm-5.2 , not glm-5 . If it shows a truncated id, you hit the dot gotcha - quote the section name. - grok -m glm-5.2 -p "reply with one word: pong" returns pong . This confirms endpoint, auth, and the chat-completions path end to end. - If you get an auth error, the env var is not set in the shell you launched grok from - source ~/.zshrc or open a new terminal. The custom Ollama Cloud models - glm-5.2, deepseek-v4-pro, kimi-k2.7-code, and the rest - appearing in grok’s model list alongside the built-in defaults, ready to switch to with /model or Ctrl+M. The bigger picture This works on the released grok binary and on the open-source grok-build https://github.com/xai-org/grok-build fork alike - they share ~/.grok/config.toml . The provider layer is intentionally provider-agnostic: three wire protocols, generalized Bearer / x-api-key auth, and arbitrary extra headers. Adding a new provider is a config entry, not a code change. New wire protocols - something that is not Chat Completions, Responses, or Messages - would need a backend variant plus a stream transform in the sampler crate, but for the vast majority of providers, including every one on this page, you never touch Rust. For model-level detail on the candidates above - throughput, context, pricing - see GLM 5.2 /models/glm-5-2 , DeepSeek V4 Pro /models/deepseek-v4-pro , DeepSeek V4 Flash /models/deepseek-v4-flash , Kimi K2 /models/kimi-k2 , and Nemotron 3 Ultra /models/nemotron-3-ultra in the catalog.