{"slug": "show-hn-collie-a-local-ai-harness-that-runs-the-browser-desktop-and-code", "title": "Show HN: Collie – a local AI harness that runs the browser, desktop and code", "summary": "Collie, a new local AI harness that runs on a user's machine, can drive the real browser, desktop, screen, and code while keeping everything private and verified. The tool, built by Collie HQ, features a coding agent with self-verifying repair loops, a Chrome extension for browser control, an ambient desktop wallpaper, screen recording, and a phone remote via an iOS companion app. Collie is terminal-first and supports editors through an open protocol, with a one-click Windows installer available on GitHub.", "body_md": "**A coding agent that lives on your computer — and can actually run it.**\n\nLocal and private. It reaches your real environment — your logged-in browser, your desktop,\nyour screen, your files — and proves its work by running it.\n\n[collie.run](https://collie.run) ·\n`collie -p \"fix the bug\"`\n\n·\n`collie web`\n\nMost coding agents live in a cloud tab or an editor pane and can only touch the files you hand\nthem. **Collie runs on your machine** — so it works the way you already do: it drives your *real*\nlogged-in browser, arranges your desktop, records your screen, takes tasks from your phone, and\nedits your code. Nothing leaves your computer unless you send it there; there's no account and no\ntelemetry.\n\nAnd it doesn't just *claim* to be done. When Collie fixes something it writes a reproduction that\nmust fail on the broken code, makes the smallest edit that flips it, and re-runs the assertion — a\nrun isn't \"done,\" it's **verified ✓**.\n\n**It's local, and it reaches your real world.** A cloud agent can read a repo. Collie can open the\nsite in the browser you're already logged into, click through the actual flow, watch what happens on\nyour screen, and change the code — all on one machine, all under your control. That's a different\nclass of task: not \"edit these files,\" but \"get this working, end to end.\"\n\n**The range is the proof.** Collie isn't a coding agent with a pile of unrelated features bolted on.\nThe breadth below — the desktop console, the browser control, the screen recorder, the phone remote —\nis there because **Collie's coding agent built all of it.** The features *are* the benchmark: a\nharness strong enough to ship its own desktop app and iOS companion is strong enough for your bug.\n\n| Capability | What it means | |\n|---|---|---|\n| 🧠 | Coding agent |\nSemantic code navigation, syntax-gated edits, and a self-verifying repair loop — the core, covered below. |\n| 🌐 | Your real browser |\nA Chrome extension lets Collie act in your logged-in browser — the real session, real cookies — so it can operate sites, not just scrape them. Every action is a fenced, CSRF-checked localhost call. |\n| 🖥️ | Living desktop |\n`collie web` powers an interactive ambient wallpaper: clock, weather, an app dock, projects, a music player (real audio + synced karaoke lyrics), and a command bar — all agent-manageable via one JSON config. When Collie is working, the wallpaper becomes a live star-map of your code. |\n| 🎬 | Screen recorder |\n`collie record` captures screen + camera + mic (Windows and macOS) — a built-in way to demo or document a run. |\n| 📱 | Phone remote |\nPair once by scanning a code; then tail runs and start new ones from your phone — on the same Wi-Fi (`--lan` ) or anywhere through a relay (`--remote` ), with the companion iOS app. |\n| 🔌 | Everywhere else |\nTerminal, browser GUI, VS Code, and any ACP editor (Zed/JetBrains/neovim) — one harness, every surface. |\n\nCollie is **terminal-first** and reaches editors through an open protocol, not a bespoke extension:\n\n| Surface | Command | Reaches |\n|---|---|---|\nTerminal |\n`collie` (TUI) · `collie -p \"task\"` |\nanywhere — SSH, CI, tmux |\nBrowser GUI |\n`collie web` |\nchat, the live verification gate, diffs, the star-map, the ambient desktop, settings |\niPhone |\n`collie web --lan` (same Wi-Fi) or `--remote` (anywhere, via the relay) + the companion app |\nscan the pair code once, then run from the phone |\nVS Code |\nthe bundled `vscode-collie` extension |\nCollie docked in a sidebar panel (manages its own server) |\nEditors (ACP) |\n`collie acp` |\nZed · JetBrains · neovim · VS Code — one adapter, every\n|\n\n**Streaming / CI**`collie run \"task\" --stream-json`\n\n**Windows — one click.** Download ** Collie-Setup.exe** from the\n\n[latest release](https://github.com/colliehq/collie/releases/latest)and double-click it. A small app-style installer lays down a self-contained runtime (Python + Collie + semantic memory, nothing to preinstall) and opens Collie in a native desktop window. On first launch you\n\n**pick a brain**— an existing Claude, Codex, or Grok login is detected and connects in one click; or paste an API key.\n\n**macOS / Linux — pip.** The core is stdlib-only, so the base install is tiny:\n\n```\npip install -e \".[local,dev]\"      # from a clone (PyPI publish is planned)\ncollie setup                       # optional deps, pre-download the memory model, pick a provider\ncollie                             # the terminal chat (TUI) opens\n```\n\nNo account, no telemetry, and the core has **zero third-party dependencies** — `mock`\n\nand `ollama`\n\nrun without any key, and memory works out of the box on BM25 keyword recall.\n\nOptional extras: `pip install \".[local,tui,search]\"`\n\n— `local`\n\n(semantic memory: granite-107m via\nonnxruntime, ~55MB, multilingual), `tui`\n\n(rich terminal chat), `search`\n\n(keyless web search), `acp`\n\n(editor protocol), `browser`\n\n(Playwright — only for `collie browser-bridge --browser`\n\n, a managed\nChromium with the extension preloaded, for CI or when you'd rather not use your own Chrome). Per-OS\nsetup — especially the real-browser bridge (`collie browser-bridge`\n\n+ `harness/browser_ext/`\n\n) — is in\n** docs/PLATFORMS.md**.\n\n```\ncollie                     # terminal chat (TUI); first run picks a provider\ncollie web                 # browser GUI — chat, live gate, diffs, star-map, ambient desktop\ncollie selftest            # $0 deterministic end-to-end (mock model, real tools + memory)\n\n# a real cheap model (provider key in env)\nDEEPSEEK_API_KEY=... collie -p \"fix the off-by-one in utils/timeparse.py\"\n\n# machine-readable / streaming\ncollie run \"fix the bug\" --json          # final result object (tokens, cost, verified)\ncollie run \"fix the bug\" --stream-json   # live NDJSON: tool · edit · repro-gate · receipt\n\n# fully local, no key\ncollie run \"summarize app.py\" --provider ollama --model qwen2.5-coder:7b\n\n# autonomous loop: iterate toward the goal, STOP the first turn an executed check goes green\ncollie loop --goal \"get the suite passing\" --until \"pytest -q\" --max 8\n\n# best-of-N with EXECUTION-based selection: run N isolated attempts, keep only what passes\ncollie pack \"fix the failing test\" -n 3 --check \"pytest -q\" --apply\n\ncollie acp                 # serve as an ACP agent (an editor spawns this over stdio)\n```\n\nProviders: `mock`\n\n, `ollama`\n\n, `anthropic`\n\n, `anthropic-oauth`\n\n, and OpenAI-compatible presets\n`deepseek`\n\n· `qwen`\n\n/`dashscope`\n\n· `openrouter`\n\n· `moonshot`\n\n· `groq`\n\n· `zhipu`\n\n· `openai`\n\n.\n\nEverything above rests on a small, honest harness. This is what's under it.\n\n```\n  locate   code_search \"parse_duration compound units\"   · 4 hits\n           › utils/timeparse.py:42  _parse  ············· 0.91\n\n  repro    wrote repro.py · assert parse_duration(\"1h30m\") == 5400\n           ✗ FAILING  › got 1800, want 5400              ← gate armed\n\n  edit     utils/timeparse.py  ································· +1 −1\n           43 │- total = SECONDS[unit] * int(val)\n           43 │+ total += SECONDS[unit] * int(val)\n\n  verify   python repro.py\n           ✓ PASSING  › parse_duration(\"1h30m\") == 5400   ← gate green\n\n  ✓ verified in 12.8s · Δ +1 −1 · 3,410 tok · $0.006\n```\n\nOther agents check \"did the test not error.\" Collie's gate is stronger: the reproduction carries an\n`assert actual == expected`\n\nderived from the issue, so a plausible-but-wrong edit fails *loudly* and\ndrives another repair round. This ** assert-verify** loop is the core of the harness — a wrong edit\nnever silently ships as \"done.\" The same idea scales up:\n\n`collie loop`\n\nstops when a real shell check\nexits 0, and `collie pack`\n\npicks the best of N attempts by what actually passes.\n\n```\n                      ┌──────────────── loop.Harness ────────────────┐\n   task ─────────────▶│  compose → complete → run tools → verify ✓   │\n                      └──┬──────────────┬──────────────┬─────────────┘\n     ┌───────────────────┘              │              └───────────────────┐\n     ▼                                  ▼                                  ▼\n ContextComposer                  ModelProvider                     ToolRegistry\n STABLE/CONTEXT/VOLATILE          OpenAI-compat · Anthropic ·       read/write/edit/bash/\n + token budgeter                 Ollama · subscription-OAuth       grep/glob + code_search\n     ▼                                  │                                  │\n memory.SqliteMemory                    ▼                            recorder.Recorder\n hybrid recall (BM25+dense+RRF)   emit → stream-json / ACP          runs.db (+ dashboard)\n```\n\n| Seam (abstract base) | shipped impl |\n|---|---|\n`ModelProvider` |\nOpenAICompat (DeepSeek/Qwen/GLM/OpenRouter…) · Anthropic · Ollama · subscription-OAuth |\n`ToolRegistry` |\nread/write/edit (syntax-gated) · bash · grep · glob · · `code_search` + `web_search` (keyless) · `web_fetch` · `plan` · browser · `undo` MCP (deferred tier + `load_tools` ) |\n`EmbeddingProvider` |\nOnnxEmbedding granite-107m (Apache, 55MB, multilingual) · bge-m3 / e5 · jina-v3 opt-in · BM25-only when no model |\n`SqliteMemory` |\nCORE + facts + FTS5 + cosine, hybrid RRF + optional rerank + consolidation |\n`ContextComposer` |\nSTABLE/CONTEXT/VOLATILE + auto-prefetch · a ~1K-token fixed prefix (kept deliberately lean) |\n\n** code_search** extracts the identifiers from a natural-language query and greps the repo (ripgrep,\nelse grep), ranking files by how many of your terms each contains — so the agent reasons about\n\n*where*to edit instead of grepping blind, with no model and no index to go stale.\n\n**is exact-match, whitespace-tolerant, and**\n\n`edit_file`\n\n**rejects any edit that would break Python syntax**. Untrusted web/page content is\n\n**fenced as data**(prompt-injection defense), and the browser bridge refuses any request missing its CSRF header. A token/cost\n\n**budget**(\n\n`COLLIE_MAX_COST`\n\n/ `COLLIE_MAX_TOTAL_TOKENS`\n\n) stops a run at a ceiling.One cross-platform Python codebase — **not** a per-OS fork. The handful of operations that genuinely\ndiffer (kill a process tree on a timeout, secure a token file, convert a path, choose a shell) are\nisolated in `harness/plat.py`\n\n, so the same wheel runs everywhere.\n\n| OS | Status | Notes |\n|---|---|---|\nLinux |\n✅ native | the primary target |\nmacOS |\n✅ native | POSIX; the browser bridge is simplest here (Chrome + Collie on one OS) |\nWindows |\n✅ one-click | the packaged installer; the agent prefers the file/search tools over `bash` |\nWSL2 |\n✅ | a Windows-Chrome ↔ WSL bridge uses the LAN IP + `wslpath` (handled for you) |\n\nCollie measures itself against other harnesses on the **same** task and model — you run it yourself;\nno numbers are asserted here:\n\n```\nDEEPSEEK_API_KEY=... python swe_run.py --n 5                 # SWE-bench Verified (needs Docker)\npython -m bench.multirun_eval                                # pass@1 / pass@k / Wilson CI / McNemar\npython -m bench.polyglot_eval --langs python,cpp,javascript  # Aider-Polyglot, multi-language\npython -m harness.cli compare --vs all                       # vs Claude Code / Aider / …\n```\n\n- The benchmark harness is version-tagged and reproducible. \"Progress is a number\" cuts both ways —\nCollie surfaces the levers that turn out\n**net-neutral**, not just the wins. - Token counts are real usage (the model's own\n`usage`\n\n, or`harness/apitap.py`\n\nmetering for CLIs that report none) — apples-to-apples, same source both sides. - Collie draws a personal Max/Pro subscription only through the first-party OAuth path\n(\n`anthropic-oauth`\n\n), the same mechanism the official CLI uses; it never scrapes or resells subscription tokens. Cheap API keys and local models are the default.\n\nMIT © 2026 — see [LICENSE](/colliehq/collie/blob/main/LICENSE).", "url": "https://wpnews.pro/news/show-hn-collie-a-local-ai-harness-that-runs-the-browser-desktop-and-code", "canonical_source": "https://github.com/colliehq/collie", "published_at": "2026-07-30 20:24:02+00:00", "updated_at": "2026-07-30 20:52:39.361962+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-tools"], "entities": ["Collie", "Collie HQ", "Chrome", "VS Code", "Zed", "JetBrains", "neovim", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/show-hn-collie-a-local-ai-harness-that-runs-the-browser-desktop-and-code", "markdown": "https://wpnews.pro/news/show-hn-collie-a-local-ai-harness-that-runs-the-browser-desktop-and-code.md", "text": "https://wpnews.pro/news/show-hn-collie-a-local-ai-harness-that-runs-the-browser-desktop-and-code.txt", "jsonld": "https://wpnews.pro/news/show-hn-collie-a-local-ai-harness-that-runs-the-browser-desktop-and-code.jsonld"}}