Showcase: Parametric CAD Agent

A single-file Claude Code agent built on Opus 5 scored 0.9505 on the gNucleus Parametric CAD Bench, with 89 of 100 parts scoring a perfect 1.0 and running 37% cheaper than the prior top row on the public leaderboard. The agent, called freecad-engineer, turns natural-language part specifications into parametric FreeCAD PartDesign scripts and was developed by Helix, which ran the full lifecycle of spec reconstruction, build-verify, evaluate, and diagnose; humans contributed about thirty messages over three hours. The agent never sees the reference model, the grader, or its score, working from its own measurements with a six-attempt budget per task.

Showcase: Parametric CAD Agent Helix ran the full lifecycle, spec reconstruction, build-verify, evaluate, diagnose, behind a single-file Claude Code agent that scored 0.9505 on the gNucleus Parametric CAD Bench, 37% cheaper than the prior top row. Humans typed about thirty messages. 0.9505. Eighty-nine of a hundred parts scored a perfect 1.0. Thirty-seven percent cheaper than the top row on the public board. And the agent that did it is one markdown file, three hundred and five lines, no fine-tuning, no code between the model and its tools. Helix ran the lifecycle that got it there. It reconstructed the spec from what already existed, reviewed the design adversarially, judged the trajectories against real traces, and diagnosed the failures, each stage leaving an artifact the next one read. Our part was about thirty messages and three hours at the keyboard: the goal, a first draft of the playbook, and saying yes. What it does freecad-engineer turns a natural-language part specification into a parametric FreeCAD PartDesign script, and proves the part is right before it stops. Stock Claude Code on Opus 5, briefed by one file appended to every task prompt. Everything the agent knows beyond the base model is in that file. The window is the whole agent. A spec goes in, a parametric feature tree comes out, and the four tiles in between are everything the definition file adds to the base model. What the picture can’t show is why each of them is there. Predict first, then measure. The point of deriving the volume by hand before writing code is not the arithmetic. It’s that the number becomes a gate the agent can’t argue with. A keyway modeled as a closed pocket instead of an open slot, a bore that stops short, a pattern with the wrong count: every one of those moves the volume, and the agent has to explain or fix any disagreement above 0.1 percent before it’s allowed to finish. A calibrated gear builder, in the prompt. Involute geometry is the kind of thing a model gets slightly wrong in a hundred different ways, and nearly a third of the benchmark is gears. So instead of asking the model to derive the tooth profile each time, the definition ships a tested function and tells the agent to copy it and pad it. That one function is the difference between gears scoring 1.0 and gears scoring whatever a fresh derivation lands on. Read the self-check as a floor. The agent can run the benchmark’s public validator on its own output, but the validator is generic. It can anchor a length it can measure directly, and it can’t see standards-derived gear values, so a perfect gear reports several parameters as missing. The rule is that a missing direct parameter is a real defect and a missing derived one, with the volume matching, is expected and gets recorded. Without that rule the agent bends correct geometry to satisfy a checker that can’t see it. One drafting convention per part family. The references were drawn a particular way, and a spec alone doesn’t say how. The keyway in the picture is the pattern: the naive read is a box cut into the shaft, the convention is an open slot whose ceiling is the shaft surface, and the two differ in volume by exactly the amount the analytic gate catches. Sixteen families have a rule like that. What the agent never sees is the reference model, the grader, and its score. It gets its own measurements and nothing else. The loop per task is plan, write one script, execute, measure, gate, repair, with a budget of six attempts. The median task takes one or two attempts and thirteen shell calls, with nobody watching. Two of the parts it produced, loaded straight from the files it saved. Drag to rotate, scroll to zoom. How it was built The human row is the whole of what a person contributed. Everything else, Helix did on its own: Spec reconstruction. Helix’s spec stage enumerated what already existed, wrote the spec back from it, and cross-checked the two. Nine gaps, including a fallback path the spec promised that existed nowhere in the playbook. Build-verify. Helix’s reviewer read the playbook, the runner, the scorers, one full task trace, the validator source, and the two patches to the benchmark’s Harbor runner. Verdict: steer, eleven ranked findings, three of them score-losing. All eleven went into the playbook before the first full run, so the 0.9471 that run scored already had them in. Evaluate. Eight full trajectories, each re-scored with the benchmark’s own scorer and then judged against six conduct criteria, forty-eight cells. Gate passed, 45 of 48. All eight trials scored a perfect 1.0, and the judges still found an 8x cost spread across them: the expensive one had misread its self-check and spent twenty-two shell calls grepping for a gap that wasn’t there. They also raised a question a score never could. Two trials stayed inside the rule that fences off the grader and the reference, then installed the public copy of the grading library and read its source to learn what gets counted. Legal as written, and now a criterion. The revisions that came out of these verdicts are the measured difference between the first full run and the last: 0.9471 to 0.9505. Diagnose. A single-shot analyzer went over the eleven trials that came in under 1.0 and returned six findings with why-chains and evidence. One of them cuts against the first tile above. The volume gate derives its expected value from the agent’s own reading of the spec, then checks a script built from the same reading. It proves the script matches the interpretation, not that the interpretation is right, and on the failing trials it agreed with itself to sixteen decimal places while the graded volume was off by up to 47 percent. The proposed remedy, a second derivation under the next most plausible reading of the spec, is not applied yet. Two more of the six are below. Two things Helix found that we would have missed The first playbook told the agent to iterate until its self-check came back clean. That instruction would have had it distort gears that already matched the reference to the last digit, because the self-check can’t see standards-derived values and reports eight of seventeen gear parameters as missing. Helix’s reviewer caught it by reading the validator source rather than its output: the reclassification step only ever promotes a finding, never demotes one, so the graded result is always at least the self-check. That is the whole basis for the floor rule above. The evaluate stage then checked for it explicitly, and both trials that hit the situation refused to touch the geometry, in writing. The second one came out of diagnose. On one task, the benchmark’s own live verifier scored a geometrically exact flange 0 out of 5. Helix reran the same official scorer against the same file. It returned 1.0. The scorer itself flaked, and its stdout for that trial was empty, so the harness had thrown away the evidence needed to explain its own zero. Read naively, that zero is an agent defect and a reason to change a correct part. Benchmarks need audits too, and Helix caught one on its own output, not ours. The headline number above still counts that task as a zero. Why CAD We didn’t pick CAD because it’s easy. We picked it because six benchmarks for this exact problem landed in about a year, and because the split inside them, editing at 80 to 90 percent against generation from drawings at 2 to 10, is the clearest picture anywhere of what a harness can reach and what only the model can. The Parametric CAD Bench sits on the checkable side of that line: its specs are text with named parameters, which is the only reason an analytic volume gate works at all. The whole argument is in CAD Benchmarks: A Proving Ground for Harness Design https://mutagent.io/blog/cad-benchmarks-harness-design/ . Our entry isn’t on the gNucleus leaderboard. The approach was calibrated against the benchmark, we said so in the submission, and we don’t claim a spot. What the run validates is the loop: point it at a target, and the score goes up on autopilot. Whether that’s the right thing to measure, what one pass costs against doing the same work by hand in Claude Code, and where held-out discipline belongs, is its own post: Meta-Evaluation: Which Tool Builds the Better Agent https://mutagent.io/blog/meta-evaluation-helix-vs-claude-code/ . Full audit trail, spec, build report, evaluator scorecard, diagnostics findings: github.com/mutagent-io/examples/tree/main/showcase/freecad-engineer https://github.com/mutagent-io/examples/tree/main/showcase/freecad-engineer .