A coding agent that runs entirely on your own machine. No API keys, no cloud, works offline.
pi is the agent you talk to. It runs against a local llama.cpp server hosting Google Gemma 4 (the unsloth builds).
Pick a model by how much memory you have:
| Model | unsloth repo | Download | VRAM | Start | Max |
|---|---|---|---|---|---|
| Gemma 4 E4B | |||||
unsloth/gemma-4-E4B-it-GGUF |
65536
(64k)131072
(128k)unsloth/gemma-4-12b-it-GGUF
65536
(64k)262144
(256k)This gives you the llama
binary; you start the server with llama serve
. Check it installed with llama version
.
macOS — Homebrew:
brew install llama.cpp
Linux (any distro) — grab a prebuilt binary with installama.sh.
This script auto-detects your CPU and GPU (CUDA / ROCm / Vulkan) and drops llama
into ~/.local/bin
(it'll tell you if that's not on your PATH):
curl -fsSL https://angt.github.io/installama.sh | sh
Add an alias to start the server. It also saves the server's output to a timestamped log file, which helps if something goes wrong. Put this in your shell config — ~/.zshrc
on macOS, ~/.bashrc
on Linux:
alias llamagemma4b='mkdir -p ~/.llama-logs && llama serve -hf unsloth/gemma-4-E4B-it-GGUF:UD-Q4_K_XL -c 65536 -fa 1 --jinja --parallel 1 --cache-ram 0 --temp 1.0 --top-p 0.95 --top-k 64 --min-p 0 2>&1 | tee ~/.llama-logs/llama-$(date +%Y%m%d-%H%M%S).log'
alias llamagemma12b='mkdir -p ~/.llama-logs && llama serve -hf unsloth/gemma-4-12b-it-GGUF:UD-Q4_K_XL -c 65536 -fa 1 --jinja --parallel 1 --cache-ram 0 --temp 1.0 --top-p 0.95 --top-k 64 --min-p 0 --reasoning on 2>&1 | tee ~/.llama-logs/llama-$(date +%Y%m%d-%H%M%S).log'
Then start it:
source ~/.zshrc # or ~/.bashrc
llamagemma4b # first run downloads the model (~5 GB), then serves on :8080
| flag | what it does |
|---|---|
--cache-ram 0 |
|
Keeps memory steady. Without it, llama.cpp piles up cached copies of the conversation that grow every time you /new . This keeps just one in place and reuses it. |
|
-c 65536 |
|
| The context window — 64k tokens here. A good default; raise it if you have memory to spare (max 128k on E4B, 256k on the 12B). | |
-fa 1 |
|
| Flash attention — faster, and uses less memory. | |
--parallel 1 |
|
| One conversation slot, which keeps the cache simple. | |
--temp 1.0 --top-p 0.95 --top-k 64 --min-p 0 |
|
| Gemma 4's recommended sampling settings. | |
--reasoning on |
|
| Enables reasoning. Only needed by 12b which has reasoning off by default. |
Gemma 4 can read images, that adds about 1.2 GB of memory; add --no-mmproj
to the alias if you'd rather run text-only.
You don't need to set anything for thinking — Gemma 4 handles it, and pi controls it. Unlike some models, turning thinking on here doesn't slow things down.
curl -fsSL https://pi.dev/install.sh | sh
pi install npm:pi-llama-cpp # connects pi to the llama.cpp server
pi install npm:pi-smart-fetch # lets the agent read web pages
pi install git:github.com/joematthews/pi-smart-web-search # lets the agent search the web
— finds your local server automatically.pi-llama-cpp
— lets the agent read web pages.pi-smart-fetch
— lets it search the web, no API key needed.pi-smart-web-search
Together they let a small model look things up instead of guessing.
source ~/.zshrc && llamagemma4b
cd ~/your-project
pi
That's the whole setup — a private coding agent that runs on a modest laptop.
Give it a spin. These all need up-to-date info from the web, which is exactly where a small local model needs a hand:
What's the latest version of Node.js, and what's new in it?
Compare Bun and Deno for a new TypeScript API in 2026.
Scaffold a minimal Vite + React + TypeScript app in ./demo, then explain the structure.
Read package.json and tell me which dependencies are out of date.
Find the current recommended way to set up GitHub Actions for a Node project, then write the workflow file.
Watch what it does: it searches, opens the most useful results, reads them, and answers from what it read — instead of guessing from old training data.