Show HN: I forked an agent stack and measured myself against it, losses included Toolbay Stack v0.2.0, a free MIT-licensed agent stack forked from an unnamed upstream, claims to reduce context cost by 83% across 53 paired skills and includes 145 engines each with its own test, with a backtest that reproduces all measurements. The stack's 20 free skills are available via npx, and it is designed to fail loudly rather than silently, as demonstrated by a seeded failure where the upstream's guard exits 0 when missing while Toolbay exits 1. Toolbay Stack · v0.2.0 An agent stack that tells you when it failed. 60 skills for Claude Code. 145 engines, and every single one carries its own test — an engine without one fails the suite rather than shipping. Free, MIT, no dependencies. Most agent tooling fails silently: a guard that allows everything when it is missing, a check that exits 0 having done nothing, a save that writes somewhere you will never look. This is built around refusing that one behaviour, and the refusals are measured rather than claimed — npm run backtest reproduces every number on this page. Get it Toolbay Stack is public and MIT licensed. Clone it and run it — there is no account, no telemetry, and no dependency to install. git clone https://github.com/sriptcollector/toolbay-stack.git It is not on npm yet, so there is no npx one-liner. Clone is the install path. Verified 2026-08-18: curl -s -o /dev/null -w '%{http code}' https://github.com/sriptcollector/toolbay-stack → 200 — public gh api repos/sriptcollector/toolbay-stack --jq .license.spdx id → MIT npm view toolbay-stack → E404 — not on npm yet What does run today, on the same machine, with no account and no telemetry, is the 20 free MIT-licensed skills on the marketplace. That command is verified end to end, not asserted. 20 free skills, no account npx toolbay add pr-reviewerInstalls a real Claude Code skill that reviews your diff before you push. See all 20 /free . Measurement 1 Context cost A SKILL.md is injected into the conversation every time its skill is invoked, so its size is a cost you pay per invocation. The backtest reads both stacks off disk and pairs them by skill name. | Skill | Toolbay | gstack | Smaller | |---|---|---|---| | /spec | 9.2 KB | 124 KB | 93% | | /review | 8.8 KB | 103.3 KB | 91% | | /qa | 8.1 KB | 81.2 KB | 90% | | /ship | 8.4 KB | 79.2 KB | 89% | | /retro | 10.5 KB | 90 KB | 88% | | /design-review | 14.3 KB | 102.5 KB | 86% | | 53 paired | 557.4 KB | 3193.2 KB | 83% | One session investigate → review → ship → context-restore Of gstack’s 290.4 KB in that four-skill session, 134KB is lines an earlier skill in the same session already injected — an auto-generated preamble stamped into every skill and paid for again on every invocation. Toolbay Stack’s repeated share is 1.9 KB. Bytes are measured; tokens are an estimate at 4 bytes/token. Files a skill reads while it runs are not counted, on either side. Three skills are bigger here The tool prints this and so do we. These are the guard skills, and the extra bytes are the documented failure modes and the list of what the guard does not cover. Trimming them to win a byte count would be gaming the number the measurement exists to report. - /careful — 6.1 KB here vs 2.5 KB upstream - /freeze — 5.7 KB here vs 3.1 KB upstream - /unfreeze — 1.7 KB here vs 1.5 KB upstream Measurement 2 Correctness on seeded failures The backtest builds a fixture with a known defect, runs the equivalent command from both stacks against it, and records CAUGHT or MISSED per side. It fails closed: a scenario whose fixture did not build, whose upstream is not installed, or which timed out is scored for nobody. The defects, with the file and the line Open them yourself. Paths are inside a gstack install at ~/.claude/skills/ . - 01 A safety guard that allows everything when it is missing gstack/investigate/SKILL.md:26 -x "$S" && bash "$S" || exit 0 This is the PreToolUse hook that is supposed to hold the edit boundary during a debugging session. If the guard script is not on the machine, the || exit 0 fires — and exit 0 with no output is how a PreToolUse hook spells ALLOW. The protection does not fail loudly, it evaporates silently. Read directly from the file, and reproduced: backtest scenario guard-not-installed, where gstack exits 0 with no output at all and Toolbay Stack exits 1 so the failure is visible. - 02 A failing test suite that reports success gstack/ship/sections/tests.md:170 bin/test-lane 2 &1 | tee /tmp/ship tests.txt & The exit status of a shell pipeline is the status of its LAST command, and the last command here is tee, which essentially always succeeds. The test suite's own exit code is discarded before anything can read it. This is the step that decides whether a change is safe to land. Read directly from the file in gstack 1.60.1.0. - 03 An expiry that cannot be parsed means never expires gstack/ios-qa/daemon/src/allowlist.ts:56 if Number.isFinite exp && exp < now continue; exp comes from Date.parse on the entry's expires at. A malformed date gives NaN, Number.isFinite NaN is false, so the continue never runs and the entry is returned as valid. A device grant with a garbled expiry is not rejected and is not expired — it is permanent. Read directly from the file in gstack 1.60.1.0. - 04 A destructive command waved straight through backtest scenario careful-block-device dd if=/dev/zero of=/dev/sda A command that destroys a filesystem. gstack's guard allowed it silently — exit 0, output "{}". Its sibling scenario careful-quoted-sql does the same with psql -c "DROP TABLE users;": the pattern match stops at the first escaped quote, so the guard never sees the DROP. npm run backtest --only careful-block-device raw log: .toolbay/backtest/raw/careful-block-device.gstack.log - 05 A corrupt row that silently vanishes from your learnings backtest scenario learn-corrupt-row LEARNINGS: 3 loaded over a store with 4 rows on disk Given a learnings store with four rows, one of them unparseable, gstack exits 0 and reports three. The fourth is gone and nothing says so. Toolbay Stack names the damaged row and exits 3. npm run backtest --only learn-corrupt-row raw log: .toolbay/backtest/raw/learn-corrupt-row.gstack.log - 06 Saved context that reports NO CHECKPOINTS over files that exist backtest scenario restore-slug-unresolvable resolved CHECKPOINT DIR= /projects//checkpoints On a machine where the slug helper is not at the path the skill hard-codes, the directory resolves with an empty middle segment, and the skill prints NO CHECKPOINTS at exit 0 over saved context that is sitting on disk. An empty answer that looks like a real answer. npm run backtest --only restore-slug-unresolvable raw log: .toolbay/backtest/raw/restore-slug-unresolvable.gstack.log The other direction What gstack does better This section is the reason to believe the rest of the page. If you want the original, use the original — it is upstream, it is very good, and it is where most of these ideas came from. Four scenarios where upstream is correct too Scored as ties. These are not wins for us. - freeze-boundary-posix freeze boundary enforced with POSIX-style absolute paths — both allowed the edit inside and denied the one outside - careful-rm-rf rm -rf on a real directory — both warned first - careful-safe-artifact rm -rf node modules — both correctly stayed silent - freeze-traversal-escape an edit that leaves the boundary through .. — both denied it One command both stacks wave straight through aws s3 rm s3://bucket --recursive Neither stack's pattern list covers it. Both allowed it silently exit 0, output "{}" . This is a real hole in Toolbay Stack, not just in upstream. A capability upstream has that this fork never ported learn-cross-project-trust-gate Cross-project learnings with a trust allowlist. A learning written by another project, without trusted: true, is correctly refused by gstack. Toolbay Stack has no cross-project learnings reader at all, so there is nothing to gate. Skills that exist only upstream: none, as of 2026-08-17 This section used to name six. The last four — benchmark-models, landing-report, setup-gbrain, sync-gbrain — landed on 2026-08-17, so every one of upstream's 53 skills now has a counterpart here. The heading is generated from the list rather than written, so it will say a number again the moment upstream ships something this fork has not matched. Seven exist only here backtest, memory, modes, router, suggest, toolbay, video and are not compared, because only one stack has them. Counts: 60 skills here, 53 upstream, 53 paired. Parity in the skill list is not parity in behaviour — the capability named above is still missing. Two results you should not quote at us One scenario freeze-boundary-windows-paths is Windows-only: gstack denied an edit that was inside the boundary because it reads a drive-letter path as relative. It does NOT reproduce on macOS or Linux, where Claude Code sends POSIX paths and upstream's guard is correct — see freeze-boundary-posix, which upstream passes. The backtest prints this caveat itself. A third scenario ios-qa-expiry-unparseable is INCONCLUSIVE: the upstream half only runs on macOS and this was measured on win32. It is scored for nobody. Measurement 3 The hook that runs before every tool call The careful guard is a PreToolUse hook, so its latency is added to every single tool call in a session. Median over 12 runs: Most of each number is interpreter startup: 59ms of the Toolbay hook is bare node, 37ms of the gstack hook is bare bash. The gap is real but it is not 7x of our own code. On top of the fork What is actually ours Modes A stance layer. Set SHIP, CRAFT, WILD or LOCK per project and every other skill changes what it does — /review in SHIP flags blockers only, the same /review in LOCK flags everything and rejects any claim without a citation. The four modes live in one hand-edited modes.json and the code knows none of their names, so a stance is a thing you edit rather than argue with. Everything fails closed: no mode, an unknown name, a corrupt or empty config all land on a restrained built-in stance and print the reason. Memory Claude's own memory keeps facts. This keeps corrections — it writes at the moment you say "no, not like that", and refuses to store one without the reason, because the reason is the only part that transfers. A newer entry never silently wins: a contradiction is surfaced with both sides and neither is applied until you say which is true. Every rule knows its scope, and when that is unclear it goes narrow. Off until you run enable twice. Suggest Reads the project's actual state and says what is worth doing next, with the evidence. Names the command, never runs it, and says "nothing worth suggesting" when that is true. The active mode changes what gets offered, and every suppressed suggestion is printed with the weight that dropped it — but no mode at any weight can suppress a credential about to be committed. Backtest The reason you are reading real numbers instead of adjectives. It seeds fixtures with a known defect, runs the equivalent command from BOTH stacks, and records CAUGHT or MISSED per side. It fails closed: a scenario whose fixture did not build, whose upstream is missing, or which timed out is scored for nobody. You can point it at your own setup and prove your own hooks fire. Cursor support The guards port to Cursor, and the installer refuses to install a skill whose guard Cursor cannot actually enforce, naming the missing capability. What is not covered is printed at the top of every installed copy rather than left for you to find: Tab autocomplete does not go through the freeze boundary, and your own typing is never guarded. Why this lives on Toolbay The skill that reaches the marketplace Every agent stack dead-ends the same way. gstack’s routing rule ends with “If nothing matches, answer directly.” its SKILL.md , line 531 — quoted exactly, because a page like this one does not get to paraphrase inside quotation marks . The toolbay skill is the one thing a fork can add that upstream structurally cannot: somewhere to go. Blocked on a capability that does not exist locally, it searches the live catalogue on this site. Finished something reusable, it can evaluate and list it without you opening a form. The rails on it are not optional. Search results are paid listings you would be buying, and are always shown with the price. An empty result stays empty rather than being padded with something loosely related. Revenue figures are ranges with a stated basis, never a single number and never a forecast. It is offered once, and declined means dropped. And the mode can switch it off outright — LOCK refuses find and call before a token is read or a packet is sent. There is no bypass flag. Check it yourself Reproduce every number on this page These are the commands the figures above came from. They read the upstream install, never write inside it, and log every process they start — argv, cwd, env overrides, exit status, stdout, stderr, duration — under .toolbay/backtest/raw/ . npm run backtest every scenario, both stacks node tools/backtest.mjs --list what the scenarios are node tools/backtest.mjs --family context node tools/backtest.mjs --gstack