AutoCraft: Harness for maintaining GitHub repositories with a large backlog AutoCraft, a model harness built for the storytold craft apps, maintains GitHub repositories with large backlogs by mirroring every issue, PR and comment into a local database each minute and running a pipeline of model steps over them using Jev classifiers plus Claude Haiku, Sonnet and Opus. The models act only through a fixed set of GitHub actions (comments, labels, branches, PRs, merges) behind one write module and safe-merge-style checks, running automatically where rules deem them safe and otherwise proposing actions for a maintainer to accept, with each issue and PR getting its own Opus-maintainer conversation. AutoCraft can track repositories owned by other GitHub organizations or users via `autocraft-server repos track owner/name` without source edits, and the pipeline is designed around maintainers' judgement as the limited resource rather than token cost. AutoCraft is a model harness for maintaining GitHub repositories with a large backlog. It mirrors every issue and PR into a local database and runs a pipeline of model steps over them. Jev classifiers and Claude Haiku, Sonnet and Opus classify, prioritise, find duplicates, ask maintainers for decisions, review, write fixes, verify and merge. The models' tools are a fixed set of GitHub actions comments, labels, branches, PRs, merges , all behind one write module and safe-merge -style checks. Actions run automatically where the rules say they are safe. Otherwise a model proposes and a person accepts. Each issue and PR has its own conversation between Opus and a maintainer, with action cards the maintainer can run. The pipeline is built around maintainers' judgement, which is the limited resource, not around token cost. Originally written for the storytold https://github.com/storytold craft apps, AutoCraft can also track repositories owned by other GitHub organizations or users via autocraft-server repos track owner/name without requiring source edits. See Track repos and sync track-repos-and-sync for naming and GitHub App sync limitations. Stages run in this order. Details, limits and rules for each are in docs/pipeline.md https://github.com/storytold/autocraft/blob/main/docs/pipeline.md . | Stage | What it does | Model | Stops for a person | |---|---|---|---| | Sync | Mirrors every issue, PR and comment of the tracked repos into the local database each minute, reads the CI state of each default branch every 2 minutes, and makes a vector of each item's title and body for similar-item search. | Local embedding model | no | | Duplicates | Finds items that describe the same bug or request as another open item, or one closed in the last year, and compares each group. | Jev, Haiku, Opus, Sonnet | yes: duplicate? decision | | Classify | Gives each open item one type label and two flags security, needs decision . Jev goes first, Haiku gives a second opinion, and a referee settles disagreements outside the scope labels. | Jev, Haiku, Sonnet, Opus | yes: scope disagreements and labels the referees were unsure of through Decide | | Priority | Gives each open item a level from P0 to P4, then orders the candidates of later stages by a work rank. | Jev, Haiku | no | | Decide | Asks an operator for scope calls, prepares every open decision with a summary, a pick and drafted answers, and answers questions about an item in its conversation. | Haiku, Opus | yes: an operator answers | | Label | Writes each settled type label and flag to GitHub. A label a person set is never changed. | none | yes: per-repo approval | | Review | Opus reviews every wanted PR at its current head, reading the repo, and gives a verdict with the complete fix list. It writes nothing to GitHub until the final review is posted. | Opus | yes: hold, close and merge-outside-rule verdicts | | Fix | Writes a patch for a bugfix issue where the fix is clear, tests it in the sandbox and opens a PR. | Sonnet, Opus, Haiku | yes: per-repo approval | | Fix-up | For a take-with-fixes review, applies the listed small fixes to the contributor's branch, or posts them as a comment when the branch cannot be pushed to. For a wanted PR whose gate check fails, runs the formatter, reruns an infrastructure failure once, or has a model patch the code. For a red default branch in an approved repo, reruns a flaky failure once and asks an operator about reporting an infrastructure failure. | Sonnet, Opus, Haiku | yes: per-repo approval | | Verify | Records upstream CI as the verification of taken PRs in repos that have it. A repo without CI, and a bugfix that changes a test file, get a model-planned build in a build environment; a model summarises failures. With the Screenshots feature on, a UI pull request is also rendered at the base and at the head, and a model compares the images; a clear regression holds the merge. | Sonnet, Haiku, Opus | yes: failures and errors | | Merge queue | Merges PRs that pass the rules in integration batches up to 50 PRs, checked by upstream CI on the batch PR through safe-merge, and asks for a Merge? decision on PRs outside the rules. Before a PR the rule would merge goes into a batch, and before a pre-approved merge is handed over to the queue, Opus reviews its head commit once more, adversarially and independently of the first review; a defect it verifies becomes a fix-up or a Merge? question for a pre-approved merge, a Merge? question . Approves held CI runs of outside contributors where the repo allows it. | Opus | yes: merge? decision for PRs outside the rules | | Act | Makes the comments and closes that follow an operator's answer, from fixed templates. | none | yes: per-repo approval | | Post-merge review | Opus reviews each PR the pipeline merged a second time, reading what landed with the pipeline's own commits and reviews of the PR, and lists defects and minor findings. Each defect that no open item covers becomes one follow-up issue in repos approved for commenting, except on a security fix whose advisory is not published yet its findings are only shown ; minor findings are only shown. | Opus | yes: per-repo approval | The image is the dashboard's home page mock data . Each column is a stage with its counts, and the chips above it list what waits for a person. - Untrusted code runs only in a separate build environment a worker as another OS user, with rootless Podman , never where GitHub or Claude credentials are. - Model calls that use tools run inside a bubblewrap sandbox. They get Read , Grep and Glob on read-only checkouts of the public tracked repos, a Haiku explorer subagent, and read-only item tools search, item, similar items, PR diff over a snapshot of the public items. Other calls run without tools. - Only src/github/write.ts writes to GitHub. Every write is recorded in an audit table, and a test fails if another file writes. - Every GitHub write needs an operator's approval for that repo. Approvals are given on the dashboard and cannot be set from the CLI. - The dashboard needs GitHub sign-in. Roles are operator answers, approvals, switches, accounts , maintainer read and comment and viewer read only . - Contributors' own agents reach the Inbox through an MCP server on the dashboard POST /mcp with a personal token. They act with their user's role, and what they send back is feedback, the same as the dashboard's Feedback box: an operator still answers. MCP decision answering and undo are currently disabled answering.enabled in src/dash/mcp/index.ts , including when GitHub App sign-in is configured. Answer decisions on the dashboard. - A pipeline switch on/off turns the whole pipeline off, a per-repo pipeline switch limits it to chosen repos, and each step has a daily limit. See Trust boundary https://github.com/storytold/autocraft/blob/main/docs/pipeline.md trust-boundary . Items is one filterable list; a row's details open inline under it. Needs you shows the decision queue beside the selected decision, with its Evidence, Discussion and item timeline. The item page shows the AutoCraft side and the GitHub side. The UI design https://github.com/storytold/autocraft/blob/main/docs/ui-design.md describes the layouts and filters. | Needed | Used for | |---|---| | Node 24 or later engines in package.json , pnpm | Runs the TypeScript directly; SQLite through the built-in node:sqlite | | gh , logged in | Sync gh api graphql and every GitHub write | | git | Clones, diffs and pushes | | claude Claude Code CLI | All Claude calls | | bwrap bubblewrap, /usr/bin/bwrap | Sandbox for tool-using calls. Without it, those calls run without tools; AUTOCRAFT NO SANDBOX=1 forces that | | jev optional | CLI for TypeSafe's Jev model, used by the classify, priority and duplicate steps jev --compact ask . Without it, switched off autocraft-server accounts jev off , or when its account is out of credit or its key is rejected, Haiku does Jev's work through the Claude accounts more Claude usage | | Podman and a second OS user | Build environments for Verify and Fix optional | | sqlite3 , curl | deploy/backup.sh and deploy/hook.sh | git clone https://github.com/storytold/autocraft.git cd autocraft pnpm install pnpm -C web install && pnpm -C web build writes web/dist node src/cli.ts lists every command autocraft is the bin of package.json ; node src/cli.ts