What this is #
grok
is the open-source terminal AI coding agent from xAI - a full-screen TUI that edits files, runs shell commands, searches the web, and drives long tasks, headless or interactive. The source lives at grok-build; the released binary installs as grok
. Out of the box it talks to grok.com. The interesting part is that the model layer is bring-your-own-endpoint: grok speaks three wire protocols - OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages - and any base_url
- model id + key triple works. There is no per-vendor SDK and no provider enum. You point it at a provider with one config file edit, no rebuild, no code.
The practical payoff: a single Ollama Cloud key gives you GLM 5.2, DeepSeek V4 Pro and V4 Flash, Kimi K2, GPT-OSS, Qwen, and more through one OpenAI-compatible endpoint - and the same mechanism adds local Ollama, OpenRouter, Moonshot, Anthropic, or any vLLM/llama.cpp server. Switch between them live with Ctrl+M
.
grok driving a session on GLM 5.2 via Ollama Cloud - the model picker, scrollback, and prompt all work as they do with the default grok.com backend.
The one file: ~/.grok/config.toml #
Custom models live in ~/.grok/config.toml
under [model.<id>]
sections. Each entry is a small struct: the model id to send the API, the endpoint, how to authenticate, which protocol to use, and the context window. A minimal entry looks like this:
[model."glm-5.2"]
model = "glm-5.2"
base_url = "https://ollama.com/v1"
name = "GLM 5.2 (Ollama Cloud)"
env_key = "OLLAMA_API_KEY"
api_backend = "chat_completions"
context_window = 200000
Then export the key in your shell and run grok against it:
echo 'export OLLAMA_API_KEY="your-key-here"' >> ~/.zshrc
source ~/.zshrc
grok -m glm-5.2
That is the whole interaction. The key is read from the environment, never written into the config file, and grok sends it as Authorization: Bearer
on every request to https://ollama.com/v1/chat/completions
.
Ollama Cloud’s OpenAI-compatible endpoint is https://ollama.com/v1
(the api.ollama.com
host just 301-redirects there). Its catalog at the time of writing includes glm-5.2, glm-5.1, deepseek-v4-pro, deepseek-v4-flash, kimi-k2.7-code, kimi-k2.6, gpt-oss:120b, gpt-oss:20b, mistral-large-3, qwen3.5, and the Nemotron and MiniMax families. List them with curl -H "Authorization: Bearer $OLLAMA_API_KEY" https://ollama.com/v1/models
.
The dot gotcha (the part that bites everyone) #
TOML treats dots in a table header as nested keys. [model.glm-5.2]
parses as model
-> glm-5
-> 2
, so grok registers the model id as glm-5
, not glm-5.2
. grok models
will list glm-5
(truncated), and grok -m glm-5.2
will fail with unknown model id
. The fix is to quote any id that contains a dot:
[model."glm-5.2"] # correct - id stays "glm-5.2"
[model.glm-5.2] # wrong - parses as nested, id becomes "glm-5"
The same applies to kimi-k2.7-code
, qwen3.5
, and anything else with a dot. Colons in the model = "..."
value are fine - only the dots in the section name need quoting. If you see unknown model id
on a model you definitely configured, this is why.
Fields, briefly #
- the id sent to the API (e.g.
model
glm-5.2
,gpt-oss:120b
). -
- the OpenAI-compatible root, no trailing path. grok appends
base_url
/chat/completions
,/responses
, or/messages
. -
-
the label shown in the model picker.
name -
the key.
api_key
orenv_key
env_key
takes a string or an array of env-var names; the first non-empty one wins. The array form is useful for SSHLC_*
forwarding (e.g.env_key = ["ANTHROPIC_AUTH_TOKEN", "LC_ANTHROPIC_AUTH_TOKEN"]
). Preferenv_key
overapi_key
so the secret stays out of the file. -
-api_backend
chat_completions
(default),responses
, ormessages
(Anthropic). -
-
total tokens; controls when grok triggers auto-compaction. If you omit it on a new model, grok assumes 200k - set it to match the real provider.
context_window -
sent verbatim on every request. Use this for Anthropic’s
extra_headers
x-api-key
andanthropic-version
, or for proxy attribution tags. -
- sampling knobs.
temperature
,top_p
,max_completion_tokens
Credential resolution order, highest first: the api_key
field, then env_key
, then your grok login
session token, then the XAI_API_KEY
environment variable. Precedence for the model itself: CLI flag -m
env var >
config.toml
remote
/v1/models
list > hardcoded defaults.
Other providers, same pattern #
Because every OpenAI-compatible server speaks the same protocol, adding a provider is one section. None of these require code changes or a rebuild.
Local Ollama (no key, your own machine):
[model.local-llama]
model = "llama3.1"
base_url = "http://localhost:11434/v1"
name = "Llama 3.1 (local)"
OpenRouter (one key, hundreds of models - use their /models
endpoint to find ids):
[model."claude-sonnet"]
model = "anthropic/claude-sonnet-4"
base_url = "https://openrouter.ai/api/v1"
name = "Claude Sonnet (OpenRouter)"
env_key = "OPENROUTER_API_KEY"
context_window = 200000
DeepSeek direct:
[model."deepseek-chat"]
model = "deepseek-chat"
base_url = "https://api.deepseek.com/v1"
name = "DeepSeek Chat"
env_key = "DEEPSEEK_API_KEY"
context_window = 64000
Moonshot / Kimi direct:
[model."moonshot-128k"]
model = "moonshot-v1-128k"
base_url = "https://api.moonshot.cn/v1"
name = "Kimi (Moonshot)"
env_key = "MOONSHOT_API_KEY"
context_window = 128000
Anthropic direct (note the messages
backend and the two required headers):
[model."claude-opus"]
model = "claude-opus-4"
base_url = "https://api.anthropic.com/v1"
name = "Claude Opus"
api_backend = "messages"
context_window = 200000
extra_headers = { "x-api-key" = "sk-ant-REPLACE", "anthropic-version" = "2023-06-01" }
For Anthropic, put the key in extra_headers
(or reference an env var there) because Anthropic authenticates with x-api-key
, not Authorization: Bearer
. Any vLLM, llama.cpp, or other OpenAI-compatible server is just a base_url
change.
Switching models #
Pick a model for a session with the CLI flag, or switch mid-session in the TUI:
grok -m glm-5.2 -p "fix the failing test in src/auth.rs" # headless
grok -m kimi-k2.7-code # start TUI on a model
Inside the TUI: /model glm-5.2
(or the /m
alias), or Ctrl+M
to open the picker, which lists every built-in and custom model. Set a persistent default so every new session starts on it:
[models]
default = "glm-5.2"
grok models
prints the full catalog. Global defaults that apply to every model - temperature
, top_p
, max_retries
, extra_headers
- go under a single
[models]
section instead of being repeated per entry.
Keep the key out of the file #
The single most important hygiene rule: never paste an API key into config.toml. The file gets backed up, synced across machines, and occasionally committed by accident. Use
env_key
and put the export in your shell profile. If a key has been shared in a chat, pasted into a ticket, or checked into a repo, rotate it at the provider’s dashboard and treat the old one as burned.Two refinements: if you would rather not put secrets in ~/.zshrc
either, a secrets manager that exports them on shell start works, and for first-party xAI access grok login
stores a session token in ~/.grok/auth.json
so you don’t need an API key at all for the default provider.
Verify it #
Three checks, in order:
grok models
lists your entries with the full id (e.g.glm-5.2
, notglm-5
). If it shows a truncated id, you hit the dot gotcha - quote the section name. -
grok -m glm-5.2 -p "reply with one word: pong"
returnspong
. This confirms endpoint, auth, and the chat-completions path end to end. - If you get an auth error, the env var is not set in the shell you launched grok from -
source ~/.zshrc
or open a new terminal.
The custom Ollama Cloud models - glm-5.2, deepseek-v4-pro, kimi-k2.7-code, and the rest - appearing in grok’s model list alongside the built-in defaults, ready to switch to with /model or Ctrl+M.
The bigger picture #
This works on the released grok
binary and on the open-source grok-build fork alike - they share ~/.grok/config.toml
. The provider layer is intentionally provider-agnostic: three wire protocols, generalized Bearer
/ x-api-key
auth, and arbitrary extra headers. Adding a new provider is a config entry, not a code change. New wire protocols - something that is not Chat Completions, Responses, or Messages - would need a backend variant plus a stream transform in the sampler crate, but for the vast majority of providers, including every one on this page, you never touch Rust.
For model-level detail on the candidates above - throughput, context, pricing - see GLM 5.2, DeepSeek V4 Pro, DeepSeek V4 Flash, Kimi K2, and Nemotron 3 Ultra in the catalog.