Humble Pi — local agentic coding on minimal hardware

A developer released Pi, a coding agent that runs entirely on a local machine without API keys or cloud dependencies. Pi works with a local llama.cpp server hosting Google Gemma 4 models, enabling offline AI-assisted coding on modest hardware. The setup requires only a few commands to install and configure the agent with web search and fetch capabilities.

A coding agent that runs entirely on your own machine. No API keys, no cloud, works offline. pi https://www.npmjs.com/package/@earendil-works/pi-coding-agent is the agent you talk to. It runs against a local llama.cpp https://github.com/ggml-org/llama.cpp server hosting Google Gemma 4 the unsloth https://huggingface.co/unsloth builds . Pick a model by how much memory you have: | Model | unsloth repo | Download | VRAM | Start | Max | |---|---|---|---|---|---| | Gemma 4 E4B | unsloth/gemma-4-E4B-it-GGUF | 65536 64k 131072 128k unsloth/gemma-4-12b-it-GGUF 65536 64k 262144 256k This gives you the llama binary; you start the server with llama serve . Check it installed with llama version . macOS — Homebrew https://formulae.brew.sh/formula/llama.cpp : brew install llama.cpp Linux any distro — grab a prebuilt binary with installama.sh https://github.com/angt/installama.sh . This script auto-detects your CPU and GPU CUDA / ROCm / Vulkan and drops llama into ~/.local/bin it'll tell you if that's not on your PATH : curl -fsSL https://angt.github.io/installama.sh | sh Add an alias to start the server. It also saves the server's output to a timestamped log file, which helps if something goes wrong. Put this in your shell config — ~/.zshrc on macOS, ~/.bashrc on Linux: Gemma 4 E4B @ 64k for pi f16 KV, flash-attn, :8080, vision on alias llamagemma4b='mkdir -p ~/.llama-logs && llama serve -hf unsloth/gemma-4-E4B-it-GGUF:UD-Q4 K XL -c 65536 -fa 1 --jinja --parallel 1 --cache-ram 0 --temp 1.0 --top-p 0.95 --top-k 64 --min-p 0 2 &1 | tee ~/.llama-logs/llama-$ date +%Y%m%d-%H%M%S .log' Gemma 4 12B @ 64k for pi needs ~12GB — same flags, bigger model alias llamagemma12b='mkdir -p ~/.llama-logs && llama serve -hf unsloth/gemma-4-12b-it-GGUF:UD-Q4 K XL -c 65536 -fa 1 --jinja --parallel 1 --cache-ram 0 --temp 1.0 --top-p 0.95 --top-k 64 --min-p 0 --reasoning on 2 &1 | tee ~/.llama-logs/llama-$ date +%Y%m%d-%H%M%S .log' Then start it: source ~/.zshrc or ~/.bashrc llamagemma4b first run downloads the model ~5 GB , then serves on :8080 | flag | what it does | |---|---| --cache-ram 0 | Keeps memory steady. Without it, llama.cpp piles up cached copies of the conversation that grow every time you /new . This keeps just one in place and reuses it. | -c 65536 | The context window — 64k tokens here. A good default; raise it if you have memory to spare max 128k on E4B, 256k on the 12B . | -fa 1 | Flash attention — faster, and uses less memory. | --parallel 1 | One conversation slot, which keeps the cache simple. | --temp 1.0 --top-p 0.95 --top-k 64 --min-p 0 | Gemma 4's recommended sampling settings. | --reasoning on | Enables reasoning. Only needed by 12b which has reasoning off by default. | Gemma 4 can read images, that adds about 1.2 GB of memory; add --no-mmproj to the alias if you'd rather run text-only. You don't need to set anything for thinking — Gemma 4 handles it, and pi controls it. Unlike some models, turning thinking on here doesn't slow things down. install pi curl -fsSL https://pi.dev/install.sh | sh install extension packages pi install npm:pi-llama-cpp connects pi to the llama.cpp server pi install npm:pi-smart-fetch lets the agent read web pages pi install git:github.com/joematthews/pi-smart-web-search lets the agent search the web — finds your local server automatically. pi-llama-cpp — lets the agent read web pages. pi-smart-fetch — lets it search the web, no API key needed. pi-smart-web-search Together they let a small model look things up instead of guessing. terminal 1: start the model server source ~/.zshrc && llamagemma4b terminal 2: open your project and start coding cd ~/your-project pi That's the whole setup — a private coding agent that runs on a modest laptop. Give it a spin. These all need up-to-date info from the web, which is exactly where a small local model needs a hand: What's the latest version of Node.js, and what's new in it? Compare Bun and Deno for a new TypeScript API in 2026. Scaffold a minimal Vite + React + TypeScript app in ./demo, then explain the structure. Read package.json and tell me which dependencies are out of date. Find the current recommended way to set up GitHub Actions for a Node project, then write the workflow file. Watch what it does: it searches, opens the most useful results, reads them, and answers from what it read — instead of guessing from old training data.