Four lessons from building IWE's block-editing language for LLM writers: state the blast radius, make identity a constraint, fail toward the recoverable mistake, and treat error messages as the documentation agents actually read.
While benchmarking IWE as memory for AI agents (the full story is in 📝 The Benchmark That Built the Tools), we watched LLM curators maintain a markdown knowledge base across hundreds of writes, and the failure statistics kept pointing at one design flaw. Adding a single fact to a page meant re-emitting the whole document — and an agent re-emitting a full document under pressure will eventually misplace a frontmatter block. In our first benchmark round, 9% of the store's event pages came out structurally malformed, and curation cost grew with the store, because every small edit paid for the whole file.
The fix became a language: block operations. IWE's query language could already select and shape documents; we extended it to address the structural nodes inside them — headers, paragraphs, list items, code blocks, tables — with one grammar used everywhere:
find --matches '(?i)todo'
greps every block in the store and answers with key › section path › text
lines. find --blocks PRED
returns matching blocks as structured data.$append
a line under a section, $replaceText
one line, $insertAfter
a block — each operator carrying its own selection predicate, validated fully before anything is written.Designing this for agents forced decisions that a human-oriented tool would never make. Four are worth recording.
Each operation can carry an expect
guard — "this edit should touch exactly one target" — and the operation fails loudly, listing what it actually selected, if reality disagrees. We then went a step further and split enforcement by surface: on the CLI, guards are opt-in (--strict
), because a human with git behind them shouldn't pay ceremony; over MCP — the agent surface — strict is always on, because the population most likely to skip the guard is exactly the one that needs it.
The workflow this creates is locate, count, pin, mutate: find your targets, learn the count, write it into the edit, and the store refuses to run an edit whose shape surprises you.
A document's key is a primary key: creating at an existing key is a conflict error, not a silent second copy. Let the agent set the key and derive it from metadata — the entity's name, the session's date — and a whole failure class disappears structurally: the same event cannot be recorded twice, a crashed ingest retries idempotently (skip-if-exists is an explicit mode, not a fallback you hope for), and cross-links become computable — a timeline line can point at 2024-05-21
without a lookup, before that page even exists. We learned this from its absence: when keys were minted from title wording, every retry and rephrasing was a fresh identity, and the store quietly accumulated near-duplicates that no prompt admonition prevented. Like expect
, it converts a discipline ("search before you write") into an engine guarantee.
The guarantee is exactly as strong as the derivation is canonical, and we got a live demonstration of the boundary: the first automated run under the new scheme created a relationship page twice — joanna-and-nate
and nate-and-joanna
— because a pair of people has two spellings and the prompt hadn't pinned one. Both keys were unique; the identity wasn't. The fix is a canonicalization rule (names in alphabetical order), the same reason compound database keys fix a column order. A uniqueness constraint dedups identities, not intentions — the derivation has to leave the writer no choices.
That lesson is now a command. iwe create
makes identity explicit on every surface: content mode requires the key — it is the document's identity — and a collision fails by default, with --if-exists
turning the retry policy into an argument (skip
for idempotent ingest, suffix
or override
only when you name them). Template mode goes further and takes the derivation out of the writer's hands entirely: a configured key_template
— journal/{{today}}
, people/{{slug}}
— mints the key from metadata, so the canonicalization rule lives in project config where the agent cannot re-decide it. And --strict
refuses to write a document that violates its bound schema — one more prompt admonition converted into an engine guarantee.
Our first draft gave update operators subtree semantics — replace a header, get its whole section replaced. Hands-on testing showed this is a data-loss trap: an agent that "renames" a header by replacing it silently destroys everything beneath. The shipped semantics follow text-editing intuition instead: a header selection is the heading line; its contents re-attach and re-level, exactly as if you'd deleted the line in an editor. Destroying a whole section requires naming the section.
The failure modes now point in the safe direction — the recoverable mistake (leftover visible content) instead of the silent one (a vanished subtree).
A guard violation prints the count, every selected target, and the fix:
error: $replaceText expects 1 block, selected 2
jon › "He is wrapping up his business plan and actively searching for investors…"
jon-actively-searching-for-investors-27-may-2023 › "Jon is actively searching for…"
hint: narrow with $within or $matches, or raise expect
We watched agents recover from these errors in a single turn, using nothing but the message. When the strict layer names the two specific guards you're missing, a small model can drive a mutation language it has never seen.
The first store curated entirely through block operations had zero structural defects — the malformed-frontmatter class that hit 9% of pages in round one is unrepresentable when metadata travels through a targeted $set
. Across hundreds of answering sessions where the agent held a mutation-capable tool, the store diff showed zero stray writes: strict mode held in the field, not just in tests.
And when we swapped the frontier-model curator for Haiku — a small model at a fraction of the cost — the structural hygiene did not move: zero malformed pages, in every run, in every store since. Structural quality had become a property of the tool workflow, not of the model driving it. That result is what let the benchmark's final configuration put the cheap model on the writing side entirely.
Guards beat instructions: an expect
clause or a key-collision error catches what a prompt admonition never will. And enforcement should live at the surface, not the grammar — humans get freedom, agents get strictness, and the language means the same thing everywhere.
The benchmark that forced these designs — losing to grep, retracting our own best number, and ending with a $4.50 curator reading at 96% of a hand-built ceiling — is written up in 📝 The Benchmark That Built the Tools. IWE is open source at iwe.md.