# Humble Pi — local agentic coding on minimal hardware

> Source: <https://gist.github.com/joematthews/d02639bcbe0e0c1c404f5d3a64c3c06f>
> Published: 2026-06-16 03:11:22+00:00

A coding agent that runs entirely on your own machine. No API keys, no cloud, works offline.

[pi](https://www.npmjs.com/package/@earendil-works/pi-coding-agent) is the agent you talk to. It runs against a local [llama.cpp](https://github.com/ggml-org/llama.cpp) server hosting **Google Gemma 4** (the [unsloth](https://huggingface.co/unsloth) builds).

Pick a model by how much memory you have:

| Model | unsloth repo | Download | VRAM | Start | Max |
|---|---|---|---|---|---|
| Gemma 4 E4B |
`unsloth/gemma-4-E4B-it-GGUF` |

`65536`

(64k)`131072`

(128k)`unsloth/gemma-4-12b-it-GGUF`

`65536`

(64k)`262144`

(256k)This gives you the `llama`

binary; you start the server with `llama serve`

. Check it installed with `llama version`

.

**macOS** — [Homebrew](https://formulae.brew.sh/formula/llama.cpp):

```
brew install llama.cpp
```

**Linux (any distro)** — grab a prebuilt binary with [installama.sh](https://github.com/angt/installama.sh).

This script auto-detects your CPU and GPU (CUDA / ROCm / Vulkan) and drops `llama`

into `~/.local/bin`

(it'll tell you if that's not on your PATH):

```
curl -fsSL https://angt.github.io/installama.sh | sh
```

Add an alias to start the server. It also saves the server's output to a timestamped log file, which helps if something goes wrong. Put this in your shell config — `~/.zshrc`

on macOS, `~/.bashrc`

on Linux:

```
# Gemma 4 E4B @ 64k for pi (f16 KV, flash-attn, :8080, vision on)
alias llamagemma4b='mkdir -p ~/.llama-logs && llama serve -hf unsloth/gemma-4-E4B-it-GGUF:UD-Q4_K_XL -c 65536 -fa 1 --jinja --parallel 1 --cache-ram 0 --temp 1.0 --top-p 0.95 --top-k 64 --min-p 0 2>&1 | tee ~/.llama-logs/llama-$(date +%Y%m%d-%H%M%S).log'

# Gemma 4 12B @ 64k for pi (needs ~12GB) — same flags, bigger model
alias llamagemma12b='mkdir -p ~/.llama-logs && llama serve -hf unsloth/gemma-4-12b-it-GGUF:UD-Q4_K_XL -c 65536 -fa 1 --jinja --parallel 1 --cache-ram 0 --temp 1.0 --top-p 0.95 --top-k 64 --min-p 0 --reasoning on 2>&1 | tee ~/.llama-logs/llama-$(date +%Y%m%d-%H%M%S).log'
```

Then start it:

```
source ~/.zshrc      # or ~/.bashrc
llamagemma4b         # first run downloads the model (~5 GB), then serves on :8080
```

| flag | what it does |
|---|---|
`--cache-ram 0` |
Keeps memory steady. Without it, llama.cpp piles up cached copies of the conversation that grow every time you `/new` . This keeps just one in place and reuses it. |
`-c 65536` |
The context window — 64k tokens here. A good default; raise it if you have memory to spare (max 128k on E4B, 256k on the 12B). |
`-fa 1` |
Flash attention — faster, and uses less memory. |
`--parallel 1` |
One conversation slot, which keeps the cache simple. |
`--temp 1.0 --top-p 0.95 --top-k 64 --min-p 0` |
Gemma 4's recommended sampling settings. |
`--reasoning on` |
Enables reasoning. Only needed by 12b which has reasoning off by default. |

Gemma 4 can read images, that adds about 1.2 GB of memory; add `--no-mmproj`

to the alias if you'd rather run text-only.

You don't need to set anything for thinking — Gemma 4 handles it, and pi controls it. Unlike some models, turning thinking on here doesn't slow things down.

```
# install pi
curl -fsSL https://pi.dev/install.sh | sh

# install extension packages
pi install npm:pi-llama-cpp     # connects pi to the llama.cpp server
pi install npm:pi-smart-fetch   # lets the agent read web pages
pi install git:github.com/joematthews/pi-smart-web-search   # lets the agent search the web
```

— finds your local server automatically.`pi-llama-cpp`

— lets the agent read web pages.`pi-smart-fetch`

— lets it search the web, no API key needed.`pi-smart-web-search`

Together they let a small model look things up instead of guessing.

```
# terminal 1: start the model server
source ~/.zshrc && llamagemma4b

# terminal 2: open your project and start coding
cd ~/your-project
pi
```

That's the whole setup — a private coding agent that runs on a modest laptop.

Give it a spin. These all need up-to-date info from the web, which is exactly where a small local model needs a hand:

```
What's the latest version of Node.js, and what's new in it?
Compare Bun and Deno for a new TypeScript API in 2026.
Scaffold a minimal Vite + React + TypeScript app in ./demo, then explain the structure.
Read package.json and tell me which dependencies are out of date.
Find the current recommended way to set up GitHub Actions for a Node project, then write the workflow file.
```

Watch what it does: it searches, opens the most useful results, reads them, and answers from what it read — instead of guessing from old training data.