![Kilo Code Local Setup: 12 Steps, 60 Min [2026]](https://shattered.io/wp-content/uploads/2026/10/kilo-code-local-setup-ollama-vs-code-2026-gen.webp)
Kilo Code, the open-source AI coding assistant for VS Code, JetBrains and the terminal, can run against a model on your own hardware instead of a paid API. Anaconda announced on July 15, 2026 that it had acquired Kilo Code, but individual use is still free and the clients are described as MIT-licensed. That combination makes Kilo Code a practical open-source AI coding assistant to pair with a local open-weight model.
Demand for this kind of tooling keeps climbing. In its 2025 developer survey, Stack Overflow reported that “31% of developers said they currently use AI agents, while 17% planned to use them.” Teams that cannot send proprietary code to a third-party API are exactly where a local Kilo Code setup earns its keep.
If you prefer a terminal-first agent, our walkthrough of OpenCode with Ollama covers that route. Kilo Code takes the editor-first route and adds permission rules, specialised agents and a shared config file across VS Code, JetBrains and the CLI. The project built below is a small Python library called textkit with seven deliberately failing tests, a preflight script that checks your local model, and a project config that decides what the agent may read, edit and run. The agent’s job is to turn the red suite green without touching the tests.
Don't miss new tech stories on Google
Add Shattered once in the Google app and our stories show up in your news suggestions.
Why Kilo Code Fits a Local Open-Source Coding Setup #
Kilo Code ships as a VS Code extension, a JetBrains plugin and a CLI. According to its own pricing material, it supports more than 60 providers and more than 500 models, including hosted frontier models, open-weight models, free models, bring-your-own-key setups and local inference. That breadth matters for a local build because you never bet the workflow on one runtime. If your 30B model stalls on a hard refactor, you change one line of config and the same agent, rules and history carry over to a hosted model.
The current VS Code extension is built on the Kilo CLI, and every client reads the same JSONC config files. Kilo’s docs put the global file at ~/.config/kilo/kilo.jsonc and the project file at kilo.jsonc or .kilo/kilo.jsonc. The stable release tagged on GitHub as v7.8.12 is one of the October 2026 7.8.x builds, and newer ones have shipped since. The v7.8.12 release changed the default Kilo Gateway route for chat, model listing, embeddings, autocomplete and transcription to ai-gateway.kilo.ai, with a KILO_AI_GATEWAY_URL override for anyone who proxies traffic.
Local does not mean fully offline. Kilo’s own documentation notes that account services, connected tools and some application functions can still reach hosted services even when inference stays on your machine. Treat the local model as a privacy and cost lever for the code that leaves your laptop, and verify the rest with a firewall rule if you need a strict air gap.
| Plan or service | Price | What it covers |
|---|---|---|
| Individual (VS Code, JetBrains, CLI) | Free | The clients themselves. Inference is billed separately or runs locally |
| Local models (Ollama, LM Studio) | $0 in software fees | You pay in hardware and electricity instead |
| Kilo Gateway | Provider list rates, no inference markup | Pay-as-you-go access to 500+ models through one key |
| Credit top-ups by card | 5% processing fee | Applies to card purchases of credits, not to seats or non-card payments |
| Teams | $15 per user per month | Shared agent modes, analytics, centralized billing, shared BYOK. Inference and cloud compute extra |
| Enterprise | Custom pricing | SSO, SCIM, audit logs, private gateway, SLAs |
Expect a gap between local and hosted quality. Kilo’s Ollama guide warns that locally runnable models are generally much smaller than cloud-hosted ones, get stuck in loops more often, call tools incorrectly more often and need more prompt trial and error. Plan for that, keep tasks small, and the setup below stays productive. For a broader look at which open-weight models deserve a spot on your disk, see our ranking of open-source LLMs by input price.
Prerequisites and Tested Versions #
Everything below was assembled against Kilo’s official documentation as of October 23, 2026. The Python project, the preflight script, the config file and the test results were run and checked on a Linux machine. The editor, extension and Ollama steps follow the vendor docs, so pin your own versions once they work.
| Component | Version or requirement | Role |
|---|---|---|
| Visual Studio Code | Latest stable | Editor that hosts the extension |
| Kilo Code extension | 7.8.x line (v7.8.12 stable on GitHub, newer builds since) | The agent |
| Kilo CLI (optional) | npm package @kilocode/cli | Terminal client sharing the same config |
| Ollama | Latest release | Local model runtime on port 11434 |
| Model | qwen3-coder:30b (Kilo’s documented recommendation), devstral:24b as the alternative | Local coding model |
| Python | 3.12 (tested on 3.12.3) | Lab project |
| pytest | 9.x (tested on 9.1.1) | Test runner |
| Node.js | 20 or newer | Only for the CLI install |
| Hardware | GPU with 24 GB VRAM or more, or a Mac with 32 GB unified memory or more | Per Kilo’s Ollama guide |
The hardware line comes straight from Kilo’s Ollama documentation, which says the models it discusses need a GPU with a large amount of VRAM (24 GB or more) or a MacBook with a large amount of unified memory (32 GB or more) to run at decent speed. Smaller machines can still handle side features such as Enhance Prompt and commit-message generation, since the docs say smaller models may suffice there.
Microsoft is also reshaping the editor. A VS Code blog post dated October 15, 2026 previews a modernized interface with Dark 2026 and Light 2026 themes, and users can still switch back to the classic workbench. The Kilo steps below describe the sidebar and the gear-icon settings panel, so pick whichever UI you like.
Create a working folder for the whole exercise and keep it outside any repository that holds client code. The agent will read and edit files here, so a clean sandbox keeps experiments contained and makes checkpoints easy to reason about.
Phase 1, Steps 1 to 3: Install Ollama, Pull the Model, Fix the Context Window #
Step 1: install Ollama and start the daemon. On Linux, the official install script does the job. On macOS, use Homebrew or the app installer from the Ollama site. On Windows, run the installer. All three expose the same HTTP API on http://localhost:11434.
curl -fsSL https://ollama.com/install.sh | sh
brew install ollama
ollama --version
ollama serve
If ollama serve reports that the address is already in use, a background service started it for you. That is fine. Confirm with one request.
curl -s http://localhost:11434/api/tags
Step 2: pull the coding model. Kilo’s Ollama guide currently recommends qwen3-coder:30b for the Kilo Code agent, with devstral:24b as the alternative. If a tool call fails, rephrase the prompt or press the Enhance Prompt button.
ollama pull qwen3-coder:30b
ollama list
The download runs to many gigabytes, so start it on a wired connection. Newer open-weight coding models appear weekly, and the Advanced Tips section covers how to swap one in without touching the rest of the setup. The pull above gives you a known-good baseline first.
Step 3: bake a 32k context window into a derived model. This step prevents the most common local failure. Kilo’s docs say Ollama truncates prompts to a very short length by default and that you need at least 32k tokens for decent results. A Modelfile fixes the problem at the source, so every client that talks to this model name gets the same window.
cat > Modelfile <<'EOF'
FROM qwen3-coder:30b
PARAMETER num_ctx 32768
PARAMETER temperature 0.2
EOF
ollama create qwen3-coder-32k -f Modelfile
ollama run qwen3-coder-32k "Reply with the single word: ready"
ollama ps
The last command shows what is loaded and where. Look at the processor column. A value such as 100% GPU means the model fits in VRAM. A split between CPU and GPU means part of it spilled into system memory, and generation will crawl. If you see a split, lower num_ctx or switch to the smaller devstral:24b. Larger windows cost memory, so raise the number only when your hardware has headroom.
You can also set Context Window Size (num_ctx) in the Kilo provider settings, as the docs describe. The derived-model approach wins because the value travels with the model name and works the same way from the editor and the CLI.
Phase 2, Steps 4 and 5: Build the Lab Project and the Preflight Script #
Step 4: scaffold the project with a red test suite. A coding agent needs an objective pass or fail signal, and a local model needs one even more. Tests give the agent a loop it can run without your help. The library below has a slugify function with three planted defects: it never trims leading or trailing hyphens, truncation can leave a dangling hyphen, and empty results do not fall back to a default.
mkdir kilo-local-lab && cd kilo-local-lab
git init
python3 -m venv .venv
. .venv/bin/activate # Windows: .venv\Scripts\activate
pip install pytest
mkdir -p textkit tests scripts .kilo
printf '.venv/\n__pycache__/\n.pytest_cache/\n.env\n' > .gitignore
Copy the Modelfile from Step 3 into this folder so the model recipe lives with the code. Then create the four source files exactly as shown. The package exports slugify, a tiny __main__ module lets you try it from the shell, and the tests encode the specification.
from .slugify import slugify
__all__ = ["slugify"]
php
import re
import unicodedata
def slugify(text: str, max_length: int = 60) -> str:
"""Return a lowercase ASCII slug.
Rules: words are joined by single hyphens, no leading or trailing
hyphen, at most max_length characters, and "untitled" when nothing
usable is left.
"""
normalized = unicodedata.normalize("NFKD", text)
ascii_text = normalized.encode("ascii", "ignore").decode("ascii")
slug = re.sub(r"[^a-z0-9]+", "-", ascii_text.lower())
return slug[:max_length]
php
import sys
from .slugify import slugify
def main(argv: list[str]) -> int:
if not argv:
print("usage: python -m textkit <text...>", file=sys.stderr)
return 2
print(slugify(" ".join(argv)))
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
python
import pytest
from textkit import slugify
def test_basic_phrase():
assert slugify("Hello, World!") == "hello-world"
def test_strips_surrounding_whitespace():
assert slugify(" Cafe au lait ") == "cafe-au-lait"
def test_folds_accents():
assert slugify("Crème brûlée") == "creme-brulee"
def test_collapses_repeated_separators():
assert slugify("one --- two ___ three") == "one-two-three"
def test_truncation_never_ends_with_hyphen():
assert slugify("alpha beta gamma", max_length=11) == "alpha-beta"
def test_truncation_respects_limit():
assert len(slugify("word " * 50, max_length=20)) <= 20
@pytest.mark.parametrize("text", ["", " ", "!!!", "---"])
def test_empty_input_returns_untitled(text):
assert slugify(text) == "untitled"
Run the suite once to confirm the starting state. Seven tests should fail and three should pass. The accent folding and separator collapsing already work, so those tests stay green.
python -m pytest -q
FAILED tests/test_slugify.py::test_basic_phrase - AssertionError: assert 'hel...
FAILED tests/test_slugify.py::test_strips_surrounding_whitespace - AssertionE...
FAILED tests/test_slugify.py::test_truncation_never_ends_with_hyphen - Assert...
FAILED tests/test_slugify.py::test_empty_input_returns_untitled[] - Assertion...
FAILED tests/test_slugify.py::test_empty_input_returns_untitled[ ] - Assertion...
FAILED tests/test_slugify.py::test_empty_input_returns_untitled[!!!] - Assertion...
FAILED tests/test_slugify.py::test_empty_input_returns_untitled[---] - Assertion...
7 failed, 3 passed in 0.04s
Commit this state with git add -A && git commit -m red-baseline. A clean commit gives you an instant rollback if the agent wanders, and Kilo’s own checkpoints sit on top of that safety net.
Step 5: write the preflight script. Most local-agent complaints trace back to one of five conditions: the daemon is down, the model is missing, the context window is too small, the model cannot emit tool calls, or generation is too slow to be usable. The script below tests all five before you open the editor. It uses only the Python standard library and talks to three documented Ollama endpoints: /api/tags, /api/show and /api/chat, plus /api/generate for timing.
#!/usr/bin/env python3
"""Preflight for a local Kilo Code stack running on Ollama.
Checks: daemon reachable, model pulled, native context length, tool-call
support, and generation speed. Exit code 0 means every hard check passed.
"""
import argparse
import json
import os
import sys
import urllib.error
import urllib.request
BASE = os.environ.get("OLLAMA_BASE_URL", "http://localhost:11434").rstrip("/")
MODEL = os.environ.get("KILO_LOCAL_MODEL", "qwen3-coder:30b")
MIN_CTX = 32768
TOOLS = [{
"type": "function",
"function": {
"name": "read_file",
"description": "Read a file from the workspace and return its text.",
"parameters": {
"type": "object",
"properties": {"path": {"type": "string"}},
"required": ["path"],
},
},
}]
def call(path, payload=None, timeout=600):
data = json.dumps(payload).encode() if payload is not None else None
req = urllib.request.Request(
BASE + path, data=data, headers={"Content-Type": "application/json"}
)
with urllib.request.urlopen(req, timeout=timeout) as resp:
return json.load(resp)
def report(name, status, detail=""):
print(f"[{status:<4}] {name}" + (f": {detail}" if detail else ""))
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--num-ctx", type=int, default=MIN_CTX)
parser.add_argument("--min-tps", type=float, default=8.0)
args = parser.parse_args()
hard_fail = False
try:
tags = call("/api/tags", timeout=10)
except (urllib.error.URLError, TimeoutError) as exc:
report("daemon", "FAIL", f"{BASE} unreachable ({exc}). Run: ollama serve")
return 2
names = [m["name"] for m in tags.get("models", [])]
report("daemon", "OK", f"{BASE}, {len(names)} model(s) pulled")
if MODEL not in names:
report("model", "FAIL", f"{MODEL} not pulled. Run: ollama pull {MODEL}")
return 3
report("model", "OK", MODEL)
info = call("/api/show", {"model": MODEL}, timeout=30).get("model_info", {})
native = next((v for k, v in info.items() if k.endswith(".context_length")), None)
if native is None:
report("context", "WARN", "model does not report a context length")
elif native < MIN_CTX:
report("context", "FAIL", f"native window {native} is below {MIN_CTX}")
hard_fail = True
else:
report("context", "OK", f"native window {native}, requesting num_ctx={args.num_ctx}")
chat = call("/api/chat", {
"model": MODEL,
"stream": False,
"messages": [{"role": "user", "content": "Read the file README.md using the tool."}],
"tools": TOOLS,
"options": {"num_ctx": args.num_ctx, "temperature": 0},
})
if chat.get("message", {}).get("tool_calls"):
report("tools", "OK", "model emitted a structured tool call")
else:
report("tools", "WARN", "no tool call returned; expect agent loops. Try another model")
gen = call("/api/generate", {
"model": MODEL,
"stream": False,
"prompt": "Write a Python function that reverses a string.",
"options": {"num_ctx": args.num_ctx, "num_predict": 128, "temperature": 0},
})
tps = gen["eval_count"] / (gen["eval_duration"] / 1e9)
load_s = gen.get("load_duration", 0) / 1e9
status = "OK" if tps >= args.min_tps else "WARN"
report("speed", status, f"{tps:.1f} tokens/s, cold load {load_s:.1f}s")
return 1 if hard_fail else 0
if __name__ == "__main__":
sys.exit(main())
Run it with the derived model name from Step 3. The model name and base URL come from environment variables, so the same script works against a remote Ollama host or a different model later. The next block shows illustrative output, since speed and load times depend entirely on your hardware.
export KILO_LOCAL_MODEL=qwen3-coder-32k
python scripts/check_stack.py
[OK ] daemon: http://localhost:11434, 1 model(s) pulled
[OK ] model: qwen3-coder-32k
[OK ] context: native window 262144, requesting num_ctx=32768
[OK ] tools: model emitted a structured tool call
[OK ] speed: 40.0 tokens/s, cold load 1.4s
The failure paths print an actionable fix and a distinct exit code. With the daemon stopped, the script prints [FAIL] daemon with the unreachable URL, suggests ollama serve and exits with code 2. With a model that was never pulled, it prints the exact ollama pull command and exits with code 3. A context window below 32768 returns code 1. A missing tool call only warns, because some models behave differently with Kilo’s real tool schema, but treat the warning seriously. Our self-hosted Qwen Code assistant guide uses a different agent and the same principle of testing the model before blaming the editor.
Phase 3, Steps 6 and 7: Install Kilo Code in VS Code and the CLI #
Step 6: install the extension. Open the Extensions view with Ctrl+Shift+X (Cmd+Shift+X on macOS), search for Kilo Code, and pick Install Pre-Release Version from the dropdown next to the Install button. That wording looks alarming, but Kilo’s docs explain that the pre-release label is only a Marketplace distribution channel. The extension is described as stable and recommended for all users, and the current extension is built on the Kilo CLI.
Prefer the command line? Kilo’s docs list this one-liner, followed by a check that the extension registered.
code --install-extension kilocode.kilo-code
code --list-extensions | grep -i kilo
VS Code forks that cannot reach the official Marketplace, such as VSCodium, can install from the Open VSX Registry. Search for Kilo Code there, install, and reload the editor if it asks. After installation, a Kilo Code icon appears in the Primary Side Bar. If the icon is missing, confirm the extension is enabled for the current workspace.
Step 7 (optional): install the CLI. The CLI shares the same engine and config files as the extension, which makes it the right tool for scripts, SSH sessions and headless runs. It installs from npm.
npm install -g @kilocode/cli
kilo --version
cd kilo-local-lab
kilo
Inside the TUI, run the /connect command to add provider credentials interactively. Local Ollama needs no key, so you can skip that and configure the provider file in the next step. Two commands are handy afterward. kilo models ollama lists the models Kilo can see for a provider, and kilo upgrade moves you to the latest build. Terminal agents change quickly, as our report on how fast Claude Code, Codex CLI and OpenCode ship updates shows, so check the version before you debug a strange behavior.
One compatibility note for people coming from OpenCode. Kilo’s settings docs warn that Kilo no longer falls back to configuration stored in .opencode directories. If you used that agent before, move the global config into ~/.config/kilo/ and any project config into ./.kilo/.
Phase 4, Step 8: Connect Kilo Code to the Local Model #
Step 8: add the Ollama provider. There are two routes, and both write to the same JSONC file, so pick whichever feels natural. Route A is the settings panel. Route B is editing config directly, which is better for a project you want teammates to clone.
Route A, the settings panel. Click the gear icon in the Kilo Code sidebar and open the Providers tab. Add Ollama. No API key is needed because Ollama runs locally. Leave the base URL at the default http://localhost:11434 unless Ollama runs on another host. Then choose your model from the picker. Kilo’s docs note that the extension stores these choices in kilo.json, so the UI and the file stay in sync.
Route B, the project file. Create .kilo/kilo.jsonc in the project root and register the derived model. The docs show this pattern for any Ollama model that does not appear in the picker automatically. The key under models must match the name Ollama serves, and the provider prefix in the top-level model field is ollama/.
{
"$schema": "https://app.kilo.ai/config.json",
"model": "ollama/qwen3-coder-32k",
"provider": {
"ollama": {
"models": {
"qwen3-coder-32k": {
"name": "Qwen3 Coder 30B (32k, local)",
"tool_call": true,
"limit": { "context": 32768, "output": 8192 }
}
}
}
}
}
Two details trip people up here. First, the tool_call flag tells Kilo the model can emit structured tool calls, so set it only for models that pass the preflight tool check. Second, the limit.context value tells Kilo how much history it may keep, and it should match the num_ctx you baked into the Modelfile. If the two disagree, Kilo may send prompts that Ollama silently truncates.
Kilo’s docs also draw a hard line between local Ollama and Ollama Cloud. The Ollama provider connects to your local daemon started with ollama serve, while Ollama Cloud shows up in the Kilo Gateway and bring-your-own-key flows as a hosted path and never touches http://localhost:11434. Choose the local entry if privacy is the point. Model choice will keep moving, because open-weight launches such as the Beam and Mistral releases arrive faster than any guide can track. The config above makes a swap a two-line change.
Now run a smoke test. Open the Kilo chat, switch to the Ask agent so nothing can be edited, and send a read-only question. A healthy local setup answers within seconds once the model is loaded.
Which files in this project define public functions, and what does each one return?
The first request after a cold start is slow because Ollama must load the weights into memory. Later requests reuse the loaded model until the keep-alive timer expires. If the first answer takes longer than a minute and your GPU is idle, jump to the troubleshooting table before changing anything else.
Phase 5, Step 9: Lock Down Permissions and Project Rules #
Step 9: write the permission policy and the AGENTS.md file. Out of the box, Kilo auto-approves most tools. Only shell commands, access to directories outside the workspace and reads of sensitive files prompt for confirmation, according to the quickstart docs. That default suits a hosted model you trust. A small local model that loops or misreads a tool schema deserves tighter rails, because a looping agent with write access can do a lot of damage in ten minutes.
The permission system uses three actions: allow, ask and deny. Rules are evaluated in order and the last matching rule wins, so put broad fallbacks first and exceptions after them. Reverse that order and the catch-all silently overrides your exceptions, which is the most common permission bug. Here is the complete project file, including the provider block from Step 8.
{
"$schema": "https://app.kilo.ai/config.json",
// Default model: the derived 32k-context build from Step 3
"model": "ollama/qwen3-coder-32k",
"provider": {
"ollama": {
"models": {
"qwen3-coder-32k": {
"name": "Qwen3 Coder 30B (32k, local)",
"tool_call": true,
"limit": { "context": 32768, "output": 8192 }
}
}
}
},
// Only the local provider shows up in the model picker
"enabled_providers": ["ollama"],
// Last matching rule wins, so broad fallbacks go first
"permission": {
"read": { "*": "allow", "*.env": "deny", "*.env.*": "deny", ".venv/**": "deny" },
"edit": { "*": "allow", ".kilo/**": "ask", "*.lock": "deny" },
"bash": {
"*": "ask",
"python -m pytest *": "allow",
"python scripts/check_stack.py *": "allow",
"git status *": "allow",
"git diff *": "allow"
}
},
"watcher": { "ignore": [".venv/**", "__pycache__/**", ".pytest_cache/**"] }
}
Walk through the policy. Reads are allowed everywhere except environment files and the virtual environment. Edits are allowed except lock files, and edits to the .kilo folder itself require a prompt so the agent cannot rewrite its own guardrails unnoticed. Shell commands default to ask, with the test runner, the preflight script and read-only git commands allowed outright. The enabled_providers list hides every provider except Ollama, so a stray click cannot route your code to a hosted model. The watcher.ignore block keeps noisy folders out of change detection.
Kilo also treats .env and .env.* reads as sensitive. Broad read approvals do not bypass the built-in prompt for those files, and .env.example counts as safe documentation. If you have an old .kilocodeignore file, it still works. The docs say a migrator converts its patterns into deny rules on the read and edit tools automatically, and 27 directories plus 11 file patterns are always skipped.
Now the human-readable half: AGENTS.md at the project root. This open-standard file tells the agent how to work in your repository, and Kilo loads it at the start of each task. Kilo write-protects it, so the agent needs your explicit approval to change it. Changes take effect in new tasks, so start a fresh session after editing.
Small Python 3.12 library (`textkit`) used to test a local coding agent.
## Commands
- Run tests: `python -m pytest -q`
- Preflight the local model: `python scripts/check_stack.py`
## Code style
- Standard library only. Do not add dependencies.
- Type hints on every public function.
- Keep functions under 25 lines.
## Rules for the agent
- Run the test suite after every change and report the pass/fail counts.
- Never edit files under `tests/` unless the task says so. If a test looks wrong, say why and stop.
- Never read or print `.env` files.
- Prefer the smallest diff that makes the tests pass.
Precedence matters when instructions collide. In the new platform, a per-agent prompt in config ranks first, project instructions in kilo.jsonc come second, the project AGENTS.md third, and global instructions after that. Keep the hard rules, such as never touching tests, in AGENTS.md where teammates can read them, and enforce the dangerous ones again in permissions where a model cannot talk its way past them.
Commit the config with git add .kilo AGENTS.md after confirming it holds no secrets. Kilo’s docs warn against checking in API keys under provider.*.options.apiKey and recommend environment variables for credentials. A local Ollama setup needs no key, which makes this the easiest config you will ever commit.
Phase 6, Step 10: Run the First Real Task and Read the Diff #
Step 10: hand the agent the red test suite. Open a new Kilo task, switch to the Code agent, and paste a prompt that names the goal, the constraint and the proof you expect. Vague prompts hurt small local models far more than they hurt hosted frontier models, so spell out the stop condition.
Run python -m pytest -q and read the failures. Fix textkit/slugify.py so every
test passes. Do not edit anything under tests/. When the suite is green, show me
the final diff and the pass count.
Watch the sequence. A healthy run goes like this: the agent runs the test command (allowed by your policy, so no prompt appears), reads slugify.py, proposes an edit, applies it, and reruns the suite. Each tool call shows its name and arguments in the chat, and you can approve or reject any prompted action. If the agent reaches for a shell command outside your allow list, Kilo asks first, which is the policy from Step 9 working as intended.
The fix is small. Three changes cover all seven failures: strip hyphens from both ends before truncating, strip again after truncating, and fall back to a default when nothing is left. Your model may phrase it differently, and that is fine as long as the tests pass and the diff stays small. Compare its work to this reference.
- slug = re.sub(r"[^a-z0-9]+", "-", ascii_text.lower())
- return slug[:max_length]
+ slug = re.sub(r"[^a-z0-9]+", "-", ascii_text.lower()).strip("-")
+ slug = slug[:max_length].rstrip("-")
+ return slug or "untitled"
After the edit, the run should end with a clean suite. This is the actual output of the reference fix on the lab project.
$ python -m pytest -q
.......... [100%]
10 passed in 0.02s
Local 30B-class models sometimes fumble the first attempt, so judge the process as well as the result. Count how many tool calls the agent needed, whether it repeated itself, and whether it respected the rule about the tests/ folder. The permission policy lets edits through everywhere except lock files and the .kilo folder, so the no-test-edits rule lives in AGENTS.md alone. For a hard guarantee, add a deny entry for tests/** to the edit block after the catch-all, run the task, and remove the entry before Step 11.
If the agent starts looping, stop it. Press the Enhance Prompt button, shorten the instruction, or split the task into one failing test at a time. Kilo’s checkpoints and your Step 4 git commit both let you roll back. When your model consistently loops on tasks this small, try a newer coding model, such as Recursive Flint, once your runtime lists it, and rerun the preflight before blaming the editor.
Phase 7, Step 11: Use the Plan, Debug and Ask Agents on a Second Task #
Step 11: extend the project with a CLI flag using three agents in sequence. Kilo’s VS Code docs describe specialised agents named Code, Ask, Plan and Debug, and the CLI docs list Architect, Ask, Debug and Orchestrator modes. Switching agents changes the system prompt and the tools available, which suits a small model because each agent asks for one narrow behavior.
Start with Plan. The agent should reason about the change without editing anything, which keeps a weak model from improvising. If your local model produces thin plans, this is the one place where borrowing a hosted frontier model for a single message pays off. Our comparison of Claude Opus 5.5 and GPT-6 Astra on Terminal-Bench shows how the hosted leaders stack up on terminal-style agent work. Remember that doing so sends the prompt, and any files it reads, to that provider.
Plan only, do not edit files. I want python -m textkit to accept a --max-length
option that is passed to slugify. List the files to change, the argparse
design, and two tests for tests/test_cli.py.
Review the plan against the existing code. Then switch to Code and approve the implementation. Because this task asks for new tests, the AGENTS.md rule about tests/ allows it. The reference implementation uses argparse and fits in a dozen lines.
import argparse
import sys
from .slugify import slugify
def main(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog="textkit")
parser.add_argument("text", nargs="+")
parser.add_argument("--max-length", type=int, default=60)
args = parser.parse_args(argv)
print(slugify(" ".join(args.text), max_length=args.max_length))
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
python
import subprocess
import sys
def run(*args):
return subprocess.run(
[sys.executable, "-m", "textkit", *args], capture_output=True, text=True
)
def test_cli_prints_slug():
assert run("Hello", "World").stdout.strip() == "hello-world"
def test_cli_respects_max_length():
result = run("alpha beta gamma", "--max-length", "11")
assert result.stdout.strip() == "alpha-beta"
With both files in place, the suite grows from ten tests to twelve. Run it, then try the tool by hand.
$ python -m pytest -q
............ [100%]
12 passed in 0.13s
$ python -m textkit "alpha beta gamma" --max-length 11
alpha-beta
If something fails, switch to Debug and paste the traceback. The agent reads the error, forms a hypothesis and checks it before editing, which beats asking the Code agent to guess. Finish with Ask for a read-only review, for example a request to list edge cases the new tests miss. Ask cannot edit files, so you can run it with full confidence even on a flaky model.
The same workflow runs from the terminal. The CLI’s run command takes a message and executes it non-interactively, which makes it handy for scripted checks.
kilo run "Run python -m pytest -q and summarize any failures in three lines"
Phase 8, Step 12: Verify the Stack, Measure It, and Review the Finished Project #
Step 12: run one command that proves everything works. The verify script chains the preflight check and the test suite, and it stops at the first failure thanks to set -eu. Run it before every session and in CI-like contexts where you rely on the local model.
#!/usr/bin/env sh
set -eu
python scripts/check_stack.py
python -m pytest -q
sh scripts/verify.sh
A green run prints the five preflight lines followed by the pytest summary. A red run tells you which layer broke, so you debug the runtime first and the agent second. That ordering saves hours, because most local-agent failures are runtime failures wearing an agent costume.
Next, build a baseline so you can compare models later. The CLI includes kilo stats for token usage and cost statistics, and the preflight script already reports generation speed. Log the same few signals every time you change a model, a quantization or a context size.
| Metric | How to measure it | Healthy sign |
|---|---|---|
| Generation speed | The speed line from scripts/check_stack.py | Stays above the –min-tps threshold you set for your hardware |
| Cold load time | Same script, load_duration field | Short enough that you do not restart the daemon between tasks |
| Tool-call reliability | Tools line of the preflight plus failed calls in the chat | Structured calls on the first try, few repeats |
| Time to green | Wall clock from prompt to a passing pytest run | Trends down as your prompts and AGENTS.md improve |
| Loop count | Times you interrupted or rephrased the task | Zero or one on small tasks |
| GPU residency | The processor column in ollama ps | Fully on GPU or unified memory, no CPU split |
The finished project is small on purpose. Every file has a job, and none of them depends on a hosted service.
kilo-local-lab/
├── .gitignore
├── AGENTS.md # rules the agent reads at the start of each task
├── .kilo/
│ └── kilo.jsonc # provider, model, permissions, watcher ignores
├── Modelfile # derived qwen3-coder-32k model (Step 3)
├── scripts/
│ ├── check_stack.py # preflight: daemon, model, context, tools, speed
│ └── verify.sh # preflight + pytest in one command
├── textkit/
│ ├── __init__.py
│ ├── __main__.py # CLI with --max-length (Step 11)
│ └── slugify.py # fixed by the agent (Step 10)
└── tests/
├── test_cli.py # written during Step 11
└── test_slugify.py # the original red suite
Commit the final state with a message that names the model and context size you used. When you upgrade the model next month, that note tells you what to compare against.
Common Pitfalls That Break Local Kilo Code Setups #
Most failures repeat across machines. These eight account for the bulk of the complaints, and every one has a cheap fix.
- Leaving Ollama at its default context. Kilo’s docs say prompts get truncated to a very short length unless you raise the window, and 32k is the floor for decent results. The agent then forgets instructions mid-task and looks stupid. Bake
num_ctxinto a Modelfile as in Step 3. - Picking Ollama Cloud when you meant local. The hosted entry appears in the Gateway and bring-your-own-key flows and never connects to localhost. Your code leaves the machine and you may not notice. Use the plain Ollama provider and confirm with
ollama psthat a request loads a model. - Enabling a model without tool calling. Kilo agents work by calling tools to read, edit and run. A model that cannot emit structured calls will print pseudo-commands as text and stall. Run the preflight tools check before you set
tool_callto true. - Ordering permission rules backwards. The last matching rule wins. Put a catch-all ask rule after a specific allow and the allow never fires. Always write the broad fallback first.
- Committing secrets in kilo.jsonc. Kilo warns against checking in values such as
provider.*.options.apiKey. Use environment variables for any hosted provider you add later, and keep.envin.gitignore. - Over a small model with tools. Kilo’s own guidance for local models says simple prompts, short conversations and disabled MCP tools speed things up. Every extra tool description eats context that a 32k window cannot spare.
- Assuming local means private everywhere. Account services and some connected tools can still use hosted endpoints. Also remember that a laptop holding your model cache and source tree is a theft risk, so full-disk encryption deserves a look. OurVeraCrypt vs BitLocker comparison covers the options.
- Ignoring the request timeout. The default API timeout is 10 minutes. A slow CPU-offloaded model can exceed it on a big prompt, and the failure looks like a random network error. Check
ollama psfirst, then raise the timeout in settings only if the hardware is simply slow.
Troubleshooting: 12 Problems and Fixes #
Work top to bottom. The first rows cover the runtime, the middle rows cover the agent, and the last rows cover config and install quirks.
| Symptom | Likely cause | Fix |
|---|---|---|
| Connection refused on port 11434 | Daemon not running, or Ollama lives on another host or inside WSL or Docker | Run ollama serve, or set the provider base URL to the real host and port |
| Model missing from the picker | Custom or derived model not registered | Add it under provider.ollama.models. The key must match the name in ollama list |
| Agent repeats the same tool call | Small mixture-of-experts model misrouting tool calls | Interrupt, use Enhance Prompt, shorten the task, disable unused MCP tools, or try devstral:24b |
| Generation crawls | Model split across CPU and GPU | Check the processor column in ollama ps. Lower num_ctx or choose a smaller model |
| Request timed out | Default 10-minute API timeout hit by a slow model | Fix the speed problem first. Then raise API Request Timeout in settings |
| Agent forgets AGENTS.md rules | Context window truncated | Confirm num_ctx is 32768 or higher, start a new task, keep the file short |
| Shell commands always prompt | Default is ask and no allow rule matches, or a later catch-all overrides one | Add a specific allow rule after the wildcard. Remember last match wins |
| Agent cannot read .env | Built-in sensitive-file protection | Expected behavior. Put placeholders in .env.example, which can be allowed |
| Illegal instruction when starting the CLI | CPU without AVX support | Download the -baseline build from the Kilo GitHub releases and run that binary |
| Config edits are ignored | Wrong file or stale session. The .kilo/kilo.jsonc file takes priority over a root kilo.jsonc | Open Local Config from the settings header to see the live file, then start a new task |
| Extension not found in VSCodium or a fork | Official Marketplace blocked for that editor | Install Kilo Code from the Open VSX Registry |
| Hosted providers still appear | Providers auto-detected or built in | Set enabled_providers to ollama only, or list unwanted ones in disabled_providers |
When none of these match, capture logs before you ask for help. Kilo’s troubleshooting docs point to an extension troubleshooting guide for collecting console logs, and the CLI offers kilo debug tools. Include your Kilo version, the model tag, and the preflight output in any bug report, because that trio answers the first three questions a maintainer will ask. More developer tool guides live in our coding and dev tools section.
Advanced Tips for Faster, Safer Local Coding #
Swap runtimes: LM Studio and any OpenAI-compatible server
Ollama is the easiest runtime, not the only one. Kilo supports LM Studio through its built-in local server. Download a GGUF model, open the Local Server tab, select the model, press Start Server, then add LM Studio on the Providers tab. If the model does not show up in the picker, register it in config. The key must match the identifier LM Studio serves, and an optional id field maps a friendlier display name onto the API name.
{
"model": "lmstudio/my-custom-model",
"provider": {
"lmstudio": {
"models": {
"my-custom-model": { "name": "My Custom Model" }
}
}
}
}
For anything else that speaks the OpenAI Chat Completions format, such as a vLLM server on a GPU box in your closet, use the Custom provider option at the bottom of the Providers tab. Give it an ID, a display name, the OpenAI Compatible API type and the base URL. Kilo fetches the model list from the server’s models endpoint, and you can add headers if a proxy needs them. Never point this option at an Azure GPT-5 deployment, because Kilo’s docs say that service rejects a parameter generic providers send. Use the native Azure provider there.
Keep a local default and borrow hosted models on purpose
A hybrid setup works well. Keep the project file pointed at the local model, and switch models per task in the picker when something genuinely needs more power. You can also move a chore to a cheaper model. The docs show a compaction setting that picks the model used to summarize old context, and it can stay local.
{
"agent": {
"compaction": { "model": "ollama/devstral:24b" }
}
}
When you do use the hosted route, mind the money and the data. Kilo Gateway charges provider list prices with no inference markup, and card top-ups carry the 5% processing fee. Kilo announced that MiniMax M3.1 Flash was temporarily free inside Kilo, with a provider price of $0.15 per million input tokens and $0.60 per million output tokens, so check that the offer still runs before you rely on it. A setting named hide_prompt_training_models removes gateway models whose providers may train on your prompts, and it is worth turning on for client work. If you proxy gateway traffic yourself, the KILO_AI_GATEWAY_URL variable overrides the default endpoint.
Try newer open-weight coding models, with a test first
The open-weight field moves quickly. Recursive reportedly released Flint on October 21, 2026, a coding model of roughly 40 billion parameters in a mixture-of-experts design with about 2.5 billion active per token, under the Apache 2.0 license, and TokenPost describes it as built for local machines. Kilo’s Google documentation names Gemma 26B and 31B as open-weight models available for local use. Mistral was also reported to be planning to release Large 4 weights on October 27, 2026, though a model that large is a server-room job, not a laptop one.
Treat every new model as a candidate, not a replacement. Pull it, create a 32k derived build, set KILO_LOCAL_MODEL to its tag, and run the preflight script. Only models that pass the tools check and the speed check deserve a place in kilo.jsonc. Then rerun the Step 10 task and compare time to green against your baseline table. A model that wins on a public leaderboard can still lose on your repository.
Two smaller wins round out the list. Set the Ollama keep-alive environment variable, OLLAMA_KEEP_ALIVE, to a longer window so the model stays in memory between tasks and you skip cold loads. And for remote work, run kilo serve on the GPU machine and attach from your laptop with kilo attach, so the heavy lifting stays home. Anaconda also reportedly introduced a Kilo Desktop beta on October 6, 2026, with a managed local-model server, so watch that project if you want a point-and-click runtime.
Local Ollama, LM Studio, a Custom Server or Kilo Gateway: Which Path Fits #
The right path depends on where your code may travel and how much hardware you own. Speed and cost swing with hardware and model choice, so the comparison below stays qualitative.
| Path | Setup effort | Cost | Where prompts go | Best for |
|---|---|---|---|---|
| Ollama (local) | Low. Install, pull, set a Modelfile | Hardware and power only | Stays on your machine | Solo developers and private repositories |
| LM Studio (local) | Low. Desktop app with a GUI server | Hardware and power only | Stays on your machine | People who like picking GGUF files visually |
| Custom OpenAI-compatible server | Medium. You run vLLM or similar | Your server costs | Stays inside your network | Teams sharing one GPU box |
| Kilo Gateway | Lowest. One key, 500+ models | Provider list rates plus 5% on card credit purchases | Goes to the model provider through Kilo | Hard tasks and quick model comparisons |
| Bring your own provider key | Low. Paste a key | Your provider bill | Goes to that provider | Teams with existing vendor contracts |
Most developers land on a hybrid: local for everyday edits and private code, hosted for the occasional hard problem. Because Kilo keeps one agent, one config format and one permission model across all five paths, you can change your mind weekly without relearning anything. Revisit the choice whenever a new open-weight model passes your preflight with room to spare.
Frequently Asked Questions About Kilo Code Local Setup #
Is Kilo Code free to use with a local model?
Yes. The VS Code extension, JetBrains plugin and CLI are free for individuals, and local inference through Ollama or LM Studio adds no software fee. You pay for hardware and electricity. Paid options exist for hosted inference, Teams seats at $15 per user per month, and custom-priced Enterprise plans.
Does Kilo Code work fully offline?
Inference can run offline once the model is on disk. Kilo’s documentation cautions that account services, connected tools and some functions may still use hosted services, so test with the network disabled before you promise anyone an air-gapped workflow.
Which local model should I start with?
Kilo’s Ollama guide currently recommends qwen3-coder:30b for the Kilo Code agent and lists devstral:24b as an alternative. Start there for a known baseline, then test newer open-weight coding models with the preflight script and your own repository.
How much memory do I need?
The docs call for a GPU with 24 GB of VRAM or more, or a Mac with 32 GB of unified memory or more, to run the recommended models at decent speed. Raising the context window increases memory use, so a smaller window or model is the first fix if you run short.
How is Kilo Code different from OpenCode?
Both are open-source coding agents with provider choice. Kilo Code leads with a VS Code extension and a JetBrains plugin on top of a shared CLI engine, and it adds permission rules and named agents. Kilo no longer reads OpenCode config from .opencode folders, so migrate those files into .kilo if you switch.
Did the Anaconda acquisition change the license or price?
Anaconda announced the acquisition on July 15, 2026. Kilo’s current materials still describe free individual use and MIT-licensed VS Code, JetBrains and CLI clients, while the gateway and cloud back end are source-available under a separate license. Check the official pricing page before you commit a team.
Can I share this setup with my team?
Yes. Commit .kilo/kilo.jsonc and AGENTS.md without secrets, and teammates get the same provider, model and permission rules when they clone the repository. Each person still needs Ollama and the derived model on their own machine.