{"slug": "ducklab-dev-harness-that-built-itself-416-runs-176-local-models", "title": "Ducklab – dev harness that built itself: 416 runs, $176, local models", "summary": "Ducklab, an Apache-2.0 self-hosted software development harness built with a Go engine, CLI, and desktop app, has completed 416 runs at a total cost of $176 using local models first, and has developed itself through 111 accepted tasks. The harness, designed for multi-LLM use, ensures verdicts are based on exit codes rather than model opinions, and supports local models via llama.cpp and vLLM alongside OpenAI-compatible and Anthropic endpoints. It operates over MCP and records all decisions, with a gate that reproduces tests from clean checkouts before any commit is accepted.", "body_md": "A full-cycle software development harness that is **multi-LLM by default**\nand honest by construction.\n\n**In one block:** self-hosted development harness (Go engine + CLI + desktop,\nLinux first) · brief → requirements → spec → plan → build → review → release ·\nverdicts are exit codes, never model opinions · local models first (llama.cpp,\nvLLM) beside any OpenAI-compatible or Anthropic endpoint · operable by humans\nor by other agents over **MCP** with recorded, attributed decisions ·\nApache-2.0 · develops itself (the run records in `.ducklab/`\n\nare the\nreceipts). Agents: start at [ AGENTS.md](/jrullan/ducklab/blob/main/AGENTS.md) and\n\n[.](/jrullan/ducklab/blob/main/llms.txt)\n\n`llms.txt`\n\nYou give it a brief. It writes requirements, a spec and a plan; builds tasks with one model or several arguing; runs your project's real test gate; and stops for you before anything is committed. Every model call is logged. No model ever decides a verdict.\n\n[\n](/jrullan/ducklab/blob/main/docs/screenshots/council.gif)*A real council intake, recorded live and sped up: the architect streams the draft, a different model reviews it, the budget ticks in cents — and the run stops at your gate. Total cost of what you just watched: $0.07.*\n\nIt was built for **local models first** — the two that built most of it are a\nvLLM box on the LAN and a llama.cpp server on localhost, both priced at zero —\nand hosted models sit beside them in the same roster, measured by the same\nevidence.\n\nMost agentic coding tools assume one strong model and trust it. Ducklab\nassumes **several cheap models and trusts none of them**:\n\n**The gate decides, never a model.** A verdict is a command's exit code. A test-first run measures a green**baseline** before any test is written, the red**over the new test** after, and every accept**reproduces the gate from a clean checkout of the committed sha**— nothing lands that did not reproduce, and an accept whose reproduction fails takes its own commit back.** Decorrelation everywhere.**A different model reviews; a reviewer never learns who wrote the code (absent from the payload, not hidden in the UI); tournament judges choose blind; council critics read the draft, not each other.**Work is a contract.** A task's deliverables are the implementer's numbered checklist; it reports on each by number, the reviewer checks each against the diff, and an undelivered item summons the**rubber duck**— an advisor seat that wakes only on measured distress (brake refusals, failure streaks, red gates) and answers`none`\n\n, a note that sends the implementer straight back to work, or`stop`\n\n.**Seats are chosen on evidence.** Every duckling carries a scorecard — in-seat pass rate from your own runs, cost per run, coding index — and the roster board suggests seats from it, with the ranking criteria yours to reorder. Suggestions are rare and justified: pass rates rank by their Wilson lower bound, three runs minimum, locals never win on a $0 price.**Nothing is unbounded.** Turns, tokens, cost, wallclock, tool output, shell commands — every ceiling visible and liftable mid-run, on the record.**Your documentation is not bounded by the model's window.** Attach a wiki to a stage and a big seat reads it whole; a small seat gets each document digested to fit, the full text one`ref_read`\n\ncall away, and the gate names any document nobody opened. A 32k local model can be briefed by a quarter-million characters of reference material — the harness carries the working memory.\n\n[\n](/jrullan/ducklab/blob/main/docs/screenshots/runs.png)*The record does not round up: every run with its verdict, its cost, and whether its accept reproduced green from a clean checkout.*\n\nAnd the existence proof: **ducklab is developed inside ducklab.** The plan,\nthe bugs, the releases and 111 accepted tasks went through its own loop,\ndriven by the same local and hosted models it measures — most recent\nfeatures (the escalation suggestions, the acceptance receipts, the MCPB\nrelease packaging, the multimodal chat) were built by the duck, gated by a\nperson. Don't take the claim on faith:\n\n```\ngit clone https://github.com/jrullan/ducklab && cd ducklab\ngo build -o ducklab-cli ./cmd/ducklab\nfor r in .ducklab/runs/*/receipt.json; do ./ducklab-cli proof verify \"$r\"; done\n```\n\nReceipts ship with every accept since v0.7.0: the committed sha, the gate command, its exit code, and the clean-checkout reproduction verdict — facts a third party re-derives, never assessments.\n\n**v0.7.0**, moving fast — seven releases in the first three weeks. Seven\nstages, five modes, the roster board with evidence and suggestions,\nreference documents with automatic digestion, skills managed from the\ndesktop, a seated consultant you can chat with (images included, vision\nverified before they are sent), bugs with screenshot evidence, adopt\nsurveys with a deterministic coverage check at the gate, provider-aware\nqueueing that says why a run waits, escalation suggestions when a seat\nmeasurably hits its ceiling, **exportable acceptance receipts** with\n`ducklab proof verify`\n\n, releases, autopilot, a CLI, a desktop app, and an\n**MCP server** — in the [official MCP registry](https://registry.modelcontextprotocol.io)\nas `io.github.jrullan/ducklab`\n\n— that lets another model operate the whole\nloop with recorded, attributed decisions.\n\n[ docs/status.md](/jrullan/ducklab/blob/main/docs/status.md) tracks all acceptance criteria and does\nnot round up. Where code and spec differ, the difference is recorded in\n\n[.](/jrullan/ducklab/blob/main/docs/decisions)\n\n`docs/decisions/`\n\nNeeds Go 1.25+, Node 22+ for the desktop, and git.\n\nThe CLI and engine are pure Go. The desktop is a Wails v3 app and needs the GTK/WebKit development packages:\n\n```\nsudo apt install libgtk-3-dev libwebkit2gtk-4.1-dev   # Debian/Ubuntu names\nmake desktop && make install\n```\n\nOn Ubuntu 24.04+ the desktop also needs an AppArmor profile — see\n[decision 0003](/jrullan/ducklab/blob/main/docs/decisions/0003-apparmor-userns.md) and\n`packaging/apparmor/`\n\n.\n\n```\nxcode-select --install    # the desktop build links against WebKit\nbrew install go node\nmake desktop && make install\n```\n\nHonesty note: ducklab is developed and exercised daily on Linux. The CLI and\nengine compile-check for `darwin/arm64`\n\non every `make cross`\n\n, but no desktop\nbuild has been verified on a Mac yet — the first person to try it is the\ntest, and `make install`\n\ngives you the CLI and engine either way. Please\nreport whatever breaks.\n\n`make install`\n\ninstalls to `~/.local/bin`\n\n— make sure it is on your `PATH`\n\n.\nIt warns when the desktop binary predates `frontend/src`\n\n, because it will\nhappily install a stale one.\n\nTo exercise the frontend against the lightweight fake engine, run the engine and Vite in separate terminals, then open the browser with its connection details:\n\n```\ngo run ./cmd/fake-engine --port 8787 --token fake-token\nnpm run dev --prefix frontend\n# open http://localhost:5173/?engine=http://127.0.0.1:8787&token=fake-token\n```\n\nThe `engine`\n\nand `token`\n\nquery parameters are available only in Vite dev\nbuilds. They can also be supplied as `VITE_DUCKLAB_ENGINE`\n\nand\n`VITE_DUCKLAB_TOKEN`\n\nenvironment variables. The desktop shell continues to use\nits injected `window.ducklab`\n\nconnection.\n\n| What it is | |\n|---|---|\n`ducklab-engine` |\nThe daemon. Owns every run. Binds 127.0.0.1 only, bearer token rotated each start. |\n`ducklab` |\nThe CLI client. Holds no state; it asks the engine. |\n`ducklab-desktop` |\nThe desktop app. Also a client, also holds no state. Starts (or adopts) the engine itself. |\n\nProvider keys come from the engine's environment at call time — export them before it starts, or launch the desktop through a wrapper that loads them from your keyring. The app tells you when the engine it adopted is missing a key this app has, with the restart button beside the words.\n\nFrom the desktop: **Projects → New project**, then **Cycle → Draft it**. From\na terminal:\n\n```\ncd ~/dev/myproject\ngit init                                    # ducklab needs a git repo\nducklab project init --name MyProject       # auto-starts the engine if none is running\n\nducklab intake --from brief.txt             # brief        → requirements\nducklab spec                                # requirements → spec\nducklab plan                                # spec         → milestones and tasks\n\nducklab run T-001                           # build it\nducklab run accept r-20260729-...           # commit it\n\nducklab review T-001                        # read the commit\nducklab release plan --bump minor           # what shipped\n```\n\nEach stage writes a `.proposed`\n\nfile first and waits for you. `accept`\n\npromotes it; `reject`\n\nrestores exactly what the run wrote and nothing else;\n\"request changes\" sends any draft — spec, plan, release notes — back with\nyour note. Nothing is committed without you (or without the autonomy level\nyou explicitly granted).\n\n**Reference documents** ride any stage: `--ref ~/wiki/product/`\n\n(or the\nattach door in the desktop) loads files or whole directories as background\nfor the architect — grounded by two rules the prompt states outright: the\napproved requirements own the scope, and where a reference and the code\ndisagree, the code is the truth. When the corpus outgrows the seat's\ncontext, each document is digested once (cached by content hash), the full\ntext stays reachable through the `ref_read`\n\ntool, and the proposal card\nlists any document no seat ever opened.\n\n**Adopting an existing codebase** works the same way: intake reads the code\nand writes as-built requirements, the spec marks its sections `as-built`\n\n, and\nthe plan stays deliberately empty — new work then enters through bug reports\nand plan amendments, which is how ducklab itself is developed.\n\nYour project declares its own truth in `.ducklab/project.toml`\n\n: the gate\n(`[verify]`\n\n— with `link_deps`\n\nand `setup`\n\nfor what a clean checkout needs),\nhow the app launches (`[run]`\n\nwith a preflight), and how the project's own\nbinaries are rebuilt (`[install]`\n\n) so the whole loop runs without leaving\nducklab.\n\nGate and shell process trees always receive `DUCKLAB_RUN_ID`\n\nand `DUCKLAB_PROJECT_ID`\n\n. For example, excercise-tracker can use `DATABASE_URL=test_db_${DUCKLAB_RUN_ID}`\n\nin `[verify].tests`\n\n, and a compose preflight can use `${DUCKLAB_PROJECT_ID}`\n\nas its per-run project name. Ducklab guarantees identity only; provisioning and teardown remain the project's.\n\n```\nducklab provider set openrouter --url https://openrouter.ai/api/v1 \\\n                                --key-env OPENROUTER_API_KEY\nducklab duckling set pato-sonnet --provider openrouter \\\n                                 --model anthropic/claude-sonnet-4.5 \\\n                                 --roles reviewer,judge --context 200000 \\\n                                 --cost-in 3.0 --cost-out 15.0\nducklab duckling test pato-sonnet --prompt \"say OK\"\n```\n\n`--key-env`\n\nis the **name** of an environment variable, never a key. No key\nis written to config, sent over the API, or kept in shell history.\n\n[\n](/jrullan/ducklab/blob/main/docs/screenshots/roster.png)*Seats are argued with evidence: pass rates from your own runs, cost per run, coding index — suggestions justified, never imposed.*\n\nThe desktop's **Roster** view is where seats are assigned: drag from the\nFlock onto a mode's seat, globally or per project, with each duckling's\nevidence on the card and the engine's suggestions beside the seats. Coding /\nintelligence / agentic indices come from OpenRouter's benchmarks endpoint\nwhen a duckling lives there; your own runs supply the rest.\n\n[\n](/jrullan/ducklab/blob/main/docs/screenshots/council-run.png)*The same machinery on real work: a council revising ducklab's own spec, 4.5M tokens in, paused once on a budget it asked to lift.*\n\n`ducklab run T-001 --mode <mode>`\n\n| Mode | What it does |\n|---|---|\n`solo` |\nOne duckling. The yardstick everything else is measured against. |\n`pair` |\nImplementer and reviewer, decorrelated. Between them the advisor — the rubber duck. |\n`tournament` |\nContestants build the same task in isolated worktrees; a judge picks, blind. |\n`split` |\nAn architect decomposes; subtasks run in parallel; integration is file copies, no model involved. |\n`council` |\nSeveral models on one document, for intake, spec, plan and review. One drafts, the others critique blind, the first revises. |\n\nThese are load-bearing, not preferences.\n\n**A model never decides a verdict.** A gate is a command's exit code.**A green candidate is applied byte-for-byte.** Nothing is re-generated after it passed.**A reviewer never learns who wrote the code.****Nothing lands that did not reproduce** from a clean checkout.**A reject undoes what the run wrote, and nobody else's work.****Nothing is unbounded.****Secrets never touch project state.****The engine is loopback-only.** There is no remote mode.\n\nA skill is a directory with a `SKILL.md`\n\n— under `.ducklab/skills/`\n\nfor one\nproject, or in the machine-wide skills directory to serve every project\n(project shadows global on a name collision). The documentation-only form\nhas no script and is the default: a recipe a model reads and follows. The\narchitect reads survey guides before an adopt (`skill_list`\n\nis in its\nprompt), the consultant reads them in chat, and only the implementer can\n`skill_run`\n\nan executable one.\n\nSkills are administered from the desktop (**gear → Skills**): list with\nscope badges and validation problems, read, edit the whole `SKILL.md`\n\n,\nrun with arguments, delete. A skill a duckling writes during a run shows\nthere greyed `pending acceptance`\n\nuntil its run is accepted — proposing a\nskill goes through the same gate as proposing code.\n\n```\nducklab skill new house-style\nducklab skill run changelog-entry --arg summary=\"...\"\n```\n\nEvery project seats a **consultant** (a Common seat on the roster board):\nthe model behind the \"chat about this\" doors and the free-form chat in the\nguide rail. It reads the code, the runs, the boards and the skills — never\nwrites — and takes **images**: paste a screenshot of a broken view and ask.\nVision is verified, not assumed: a declared-vision seat is probed with a\nreal image request once, and a text-only seat refuses the paste with words\ninstead of hallucinating an answer.\n\n[\n](/jrullan/ducklab/blob/main/docs/screenshots/consultant-chat.png)*Asked to sell the product, the seated consultant read the repo and wrote this pitch itself. We kept it.*\n\n`ducklab mcp serve`\n\nexposes the whole loop over stdio as an MCP server: an\nexternal model reads each result, decides gates (with a required, recorded\nreason — decisions land as `approved_by: mcp:<client>`\n\n, never as \"human\"),\nanswers questions, files bugs, amends plans and starts work. The engine's\n`next`\n\nlists are the law: an operator cannot take an action a person could\nnot.\n\nSee [CONTRIBUTING.md](/jrullan/ducklab/blob/main/CONTRIBUTING.md) — how to build, how the tests guard\nthe architecture, how work flows through ducklab's own loop, and where to\nstart. The short version:\n\n```\nmake            # vet, test, build the frontend\ngo test ./...   # 38 packages\ncd frontend && npx vitest run\n```\n\n**License:** [Apache-2.0](/jrullan/ducklab/blob/main/LICENSE). Contributions are accepted under the\nsame terms (§5 of the license — no CLA). The Ducklab name and the duck are\nthe maintainer's (§6).\n\nThe code implements a written specification, in this repo:\n[ docs/spec/](/jrullan/ducklab/blob/main/docs/spec) (00-VISION through 08-DESKTOP-UI) is the\n\n**normative** layer — vision, invariants, protocol contracts, acceptance criteria. What the system IS today lives in\n\n`.ducklab/docs/`\n\n— the as-built\nrequirements, spec and plan the loop itself maintains, each version signed\nat a human gate. Where the two differ deliberately, the difference is\nrecorded in [; the diff between them is the roadmap, and the alignment stage computes it.](/jrullan/ducklab/blob/main/docs/decisions)\n\n`docs/decisions/`", "url": "https://wpnews.pro/news/ducklab-dev-harness-that-built-itself-416-runs-176-local-models", "canonical_source": "https://github.com/jrullan/ducklab", "published_at": "2026-08-23 14:55:02+00:00", "updated_at": "2026-08-23 15:14:17.458602+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "ai-tools", "machine-learning"], "entities": ["Ducklab", "llama.cpp", "vLLM", "OpenAI", "Anthropic", "MCP", "Apache-2.0"], "alternates": {"html": "https://wpnews.pro/news/ducklab-dev-harness-that-built-itself-416-runs-176-local-models", "markdown": "https://wpnews.pro/news/ducklab-dev-harness-that-built-itself-416-runs-176-local-models.md", "text": "https://wpnews.pro/news/ducklab-dev-harness-that-built-itself-416-runs-176-local-models.txt", "jsonld": "https://wpnews.pro/news/ducklab-dev-harness-that-built-itself-416-runs-176-local-models.jsonld"}}