{"slug": "why-the-model-shouldn-t-be-the-one-that-says-done", "title": "Why the model shouldn't be the one that says \"done\"", "summary": "A developer released Loopsmith, an MIT-licensed open-source tool at v1.0.0 that enforces a rule that the AI model never certifies its own completion. The Rust CLI pairs a plain-text job definition with a deterministic compiled gate that reads files rather than the model's claims, can revoke satisfied goals when required artifacts are deleted, refuses judges running on the same provider as the work, and bounds unattended loops with iteration, time and no-progress caps plus an append-only checkpointed ledger. The accompanying agent skill proposes changes to goals and success criteria for user approval rather than moving its own goalposts.", "body_md": "*Disclosure: this article was written with AI assistance and reviewed and edited by me, the tool's author. Loopsmith is my own open-source project.*\n\nI have a few jobs I redo every week and care about getting right: a competitor roundup, a lead list, a research brief. Handing them to an AI agent works up to a point. The part that kept failing for me was the end of the job, not the work itself.\n\nThe agent finishes, says \"done\", and I open the output to find a missing section, a citation that points nowhere, or a test it quietly edited so it would pass.\n\nSo I built a small tool around one rule. **The model never gets to say \"done\".** I call it [loopsmith](https://github.com/bitphill/loopsmith): a skill that creates self-evolving, loop-based, purpose-specific skills, backed by a Rust CLI. It's MIT-licensed, at v1.0.0, and I'm sharing it here to find out whether the idea holds up for other people.\n\n*20-second demo: [watch it with sound](https://github.com/bitphill/loopsmith/blob/main/assets/demo/loopsmith-demo.mp4).*\n\nMost agent loops end in one of two ways. The model decides it's finished, or a counter runs out. The first means the thing doing the work also grades the work. The second means you get whatever it had when the budget ended.\n\nA model asking itself \"is this done?\" is not a check. It's an opinion from the party with the most reason to be agreeable. The design rule that follows is simple: whatever certifies completion must not be the thing doing the work.\n\nYou describe a job once, in one plain text file (YAML or Markdown). The file says what you want, what proves it, and every way the run is allowed to end. loopsmith then:\n\nThat checker is the **gate**, and it's compiled Rust code. It reads files, not the model's description of them. In the README's words, `goal_satisfied` is written by the deterministic gate and by nothing else.\n\nThe gate can also **revoke**. If you delete a required artifact later, a goal that was satisfied flips back to unsatisfied. A system that can only mark things complete is a burndown chart with extra steps, and I didn't want that.\n\nA judge running on the same provider as the work it grades is refused, not just warned about. A model shouldn't mark its own homework.\n\nAn unattended loop needs more than a good definition of done. It needs to be bounded. loopsmith has ceilings on:\n\nplus a no-progress cap for the loop that is still busy but no longer changing anything. Every iteration is checkpointed and written to an append-only ledger, so a run that dies at 4am resumes instead of restarting. You can ask the ledger what happened afterwards:\n\n```\nloopsmith run ledger <config> <run-id>\n```\n\nThe other half of the project is a skill, installed into a coding agent, that creates self-evolving, loop-based, purpose-specific skills. You describe a purpose, and it helps you shape a loop around it: one skill per job, with the gate that proves the job is done.\n\n```\nnpx skills add bitphill/loopsmith\n```\n\nThe skills drive the `loopsmith` binary, so install the binary too.\n\nIt means the loop improves from work that has to be redone. When the gate fails a goal, the work goes back to the AI with what failed, and the loop learns from that redo. It also trials candidate skills and scores them against gate outcomes, and it keeps scratchpad notes between iterations. A candidate with too few trials is recorded and ignored, so one lucky run isn't treated as evidence.\n\nWhat it does not do is move its own goalposts. It never changes its own goals, checks or config without your approval. Changes to goals, validations, success criteria and the skills a config uses arrive as proposals, and you apply them yourself:\n\n```\nloopsmith run proposals <config> <run-id>\n```\n\nA system that can move its own goalposts can't certify that it met them, so I didn't build one.\n\nPick whichever package manager you already have. All of them put a `loopsmith` on your `PATH`.\n\n```\n# crates.io, builds from source\ncargo install loopsmith\n\n# npm, downloads a prebuilt binary\nnpm install -g @bitphill/loopsmith\n\n# PyPI, downloads a prebuilt binary on first run\npip install loopsmith-cli\n\n# Homebrew\nbrew install bitphill/loopsmith/loopsmith\n```\n\nOn Homebrew 6, third-party taps need a one-time `brew trust bitphill/loopsmith` first. There is also `./install.sh` (Linux, macOS, BSD) and `install.bat` (Windows) in the repo, and the `npx skills add` line above for the agent skills.\n\nThe npm and PyPI names differ from the command because plain `loopsmith` was already taken on both by unrelated projects.\n\nYou don't have to hand-write the config. All three front ends build the same file:\n\n```\nloopsmith --guided   # question-and-answer wizard in the terminal, works over SSH\nloopsmith --web      # local web UI, bound to 127.0.0.1 only\nloopsmith loop new -p ~/loops/my-loop\n```\n\nThere's also a local MCP server, so an agent can drive it directly.\n\nFifteen working example loops ship with the binary. Loading one is the fastest way to see what a validation or a stop gate looks like. A few of them:\n\nI'd rather be plain about this:\n\nIf you try it, I'd like to hear about:\n\nIssues and PRs are welcome. There's a `CONTRIBUTING.md` in the repo.\n\nThanks for reading.", "url": "https://wpnews.pro/news/why-the-model-shouldn-t-be-the-one-that-says-done", "canonical_source": "https://dev.to/bitphill/why-the-model-shouldnt-be-the-one-that-says-done-ibh", "published_at": "2026-11-05 23:36:37+00:00", "updated_at": "2026-11-05 23:54:55.718566+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-safety"], "entities": ["Loopsmith", "Rust", "GitHub", "crates.io", "npm"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/why-the-model-shouldn-t-be-the-one-that-says-done", "markdown": "https://wpnews.pro/news/why-the-model-shouldn-t-be-the-one-that-says-done.md", "text": "https://wpnews.pro/news/why-the-model-shouldn-t-be-the-one-that-says-done.txt", "jsonld": "https://wpnews.pro/news/why-the-model-shouldn-t-be-the-one-that-says-done.jsonld"}}