# Stable IDs Are All You Need | barkup-bench > Source: > Published: 2026-07-08 13:05:35.819311+00:00 Over the past week we ran seven pre-registered studies, more than 13,000 scored model runs, against one question: what is the most reliable way to let an LLM agent edit structured data? Typed trees, specifically: page layouts, document templates, anything with a schema, nesting, and rules. We tested four models from three vendors, trees from 5 to 1,000 nodes, and editing sessions up to twelve consecutive edits. The whole thing compresses to one sentence: **give every node a stable id, and never make the model reproduce anything it is not changing.** Here is how we got there, including the part where we had to correct ourselves in public. ## The question we started with The [original essay](/blog/ast-as-html) argued that an HTML dialect plus whole-tree rewrite was the right interface for agent editing: readable, familiar, one coherent artifact validated in one shot. We [benchmarked it](/blog/barkup-bench-results) against an equal-strictness JSON twin, granular mutation tools, and patch formats, with every prompt and seed committed before the first scored call. The format claim died immediately, and cleanly. Modern models write strict HTML and strict JSON essentially perfectly (first-pass validity was at least 99.3% in every arm of every model). Format never decided anything. That was our first null result, and far from the last. ## The bug that was better than a finding The early headline said whole-tree rewrite beat granular tools by 33 points on multi-turn tasks. It was dramatic, it confirmed our thesis, and it was wrong. [Our harness had built conversation histories with the AI SDK's response.messages](/blog/tool-history-footgun), which silently omits the model's own tool calls. In every multi-turn tools conversation, the model could not see what it had just done, including the insert that returned the id it was asked to edit next. With correct history, the gap vanished. Rewrite vs tools became 91.9% vs 93.9% (a slight tools edge). The "small models are bad at tools" result vanished with it: gemini-flash went from 3.8% to 71.3% on multi-turn edits, haiku from 28.7% to 98.8%, purely from history construction. The frontier models had masked the bug because they barely need history to re-derive state. We re-ran everything affected, published the correction across every post, and filed the issue upstream ([vercel/ai#16840](https://github.com/vercel/ai/issues/16840)). Two lessons survived that are worth more than the original claim: audit your history construction, and evaluate on a cheap model, because it will expose plumbing failures your frontier evals hide. ## Positions lie, ids don't One early result did survive everything: RFC 6902 JSON Patch, the standard that addresses nodes by positional paths like `/children/3/children/7` , collapsed as trees grew. 70% success at 150 nodes, roughly 10% at 1,000. Index arithmetic is the wrong job for a language model. So we pre-registered a patch dialect where every operation names its target by stable id and placements anchor to sibling ids, no indexes anywhere. [It recovered the entire collapse](/blog/barkup-0-2-anchored-patches), matched whole-tree rewrite on accuracy, and was the cheapest condition measured. Then [the size extension](/blog/we-found-the-crossover) made it decisive: above roughly 300 nodes, whole-tree rewrite becomes a frontier-only technique (gemini-flash solved 0 of 15 tasks at 1,000 nodes; sonnet needed a streaming transport and ten minutes per rewrite), while anchored patches held 87 to 100% for both tiers at every size. Per solved 1,000-node edit: about $0.26 and four seconds, vs $0.88 and ten minutes. ## The model doesn't need to see the tree Every study so far had put the full tree in the prompt. Two follow-up studies asked whether it must. We replaced the full serialization with a focused view: the path from the root to the nodes the edit concerns, their child lists in order, and everything else collapsed to labeled placeholders or omitted with a count. Accuracy did not move. Statistically indistinguishable from full input for both models at every size, and sonnet on the most aggressive view went 45 of 45. Input tokens fell 96 to 98% at 1,000 nodes, and the view barely grows with tree size at all (about 1,300 tokens at 300 nodes, about 1,500 at 1,000): it scales with tree depth, not node count. Document size stops mattering to your token bill. Rendering the view as HTML vs JSON changed nothing on accuracy, one more format null, though the HTML views were 9 to 24% terser. ## Sessions drift, and the fix is free The last study was the production shape: twelve consecutive edits against one evolving tree in a single conversation. Shown the tree once and then asked to keep patching it, sonnet decayed to 83.8% success by the final third of the session, and the failures were exactly what you would guess: placements computed against a mental child list that three earlier edits had reshuffled. Only 8 of 20 sessions ended with a byte-perfect tree. Attaching a fresh minimal view to every single turn erased the drift (239 of 240 steps, 19 of 20 sessions intact) and cost four times less, because the once-shown tree rides along in conversation history every turn anyway. Best and cheapest turned out to be the same policy. Whole-tree rewrite as a session protocol failed structurally: the cheap model silently corrupted state (every one of its 44 failures was a valid tree with wrong content), and one frontier session deterministically ran out of context window at step 11, because twelve accumulated rewrites of a 350-node tree simply do not fit. ## The checklist for app builders **Give every node a stable id and guarantee nothing ever renumbers it.** Every winning result in seven studies rests on this one property.**Have the model reply with id-anchored operations**, applied and validated by your code. Never positional indexes. Never reproduction of unchanged content.** Send focused context, not the whole document.**A view of the relevant region with honest placeholders matches full-input accuracy at 2 to 4% of the cost.** In multi-turn editing, refresh the model's picture every turn.**It is cheaper than not doing it.** Validate at the boundary and return structured errors verbatim.**The correction loop recovered most first-pass failures everywhere.** Audit your conversation-history plumbing, and eval on a cheap model.**Our worst failure came from the SDK, not the model, and frontier models hid it.** Whole-tree rewrite is fine below about 200 nodes and for from-scratch builds.**Do not scale it, and do not chain it. ## What shipped, and what we still don't know The validated pieces live in `@kevinpeckham/barkup` [0.3](/blog/barkup-0-3-focused-views): the anchored-patch applier and the focused-view renderer, each shipped with conformance vectors generated by the benchmark itself. Honest limits: the view studies are an oracle bound (our instructions name their target ids, so finding the right node from a vague request is untested); the later studies ran on two models rather than four; and everything ran on one grammar family. Those are the next questions, not footnotes we are hiding. Everything is reproducible from the [benchmark repo](https://github.com/kevinpeckham/barkup-bench): pre-registrations committed before every scored run, seeded corpora, raw analyses, the correction, and the audit trail. The series in order: [the original argument](/blog/ast-as-html), [the benchmark](/blog/barkup-bench-results), [the footgun](/blog/tool-history-footgun), [anchored patches](/blog/barkup-0-2-anchored-patches), [the crossover](/blog/we-found-the-crossover), and [the barkup 0.3 release](/blog/barkup-0-3-focused-views).