Built 27 Aug 2026 · written up 14 Oct 2026
Yesterday I asked:
Can technically safe product knowledge survive contact with an actual user?
Not particularly. 😅
One model, one fixture, same as yesterday: every number below comes from anthropic-opus-5 against a single application we wrote ourselves, with patterns we knew to look for.
Wrong-feature claims accepted 0
Unsupported claims accepted 0
Model restraint 30/30
Verified claim yield 95%
Nice.
Then we asked:
Would any of this actually help a human?
The usefulness scores below were not produced by people. A separate model, OpenAI GPT-5.6 Sol, scored the twenty-one features against a fixed rubric. It is not the model under test, so this is not self-scoring, but it is not a human review either, and no amount of it becomes one.
1 / 3 median usefulness
24% rated genuinely useful
Nothing was dangerously wrong.
Nothing was completely useless either.
It was worse in a much more software-engineering kind of way:
almost everything was mediocre.
We assumed the remaining problem would be missing capabilities.
The data disagreed.
1.43 features with a verified capability
1.14 features without one
Barely a difference.
That was how we read it at the time. A later round, with the language problem below fixed, put the same split at 2.00 against 1.07. The capability signal was real; we were reading it through bad presentation.
Meanwhile:
14 / 21 outputs were too technical.
Thirteen of twenty-one were flagged as missing a capability.
Then we found the smoking gun.
Ten features exposed this sentence directly to the user:
"No capability, route or permission has been verified for this feature."
Every single one was rated too technical.
10/10.
Fair enough. 😂
That's not product copy.
That's a debugger having an existential crisis in front of the customer.
Another feature had excellent evidence:
✓ POST endpoint ✓ Create permission
✓ Submit path ✓ Button
And our deterministic renderer proudly produced:
"You can create a new invoices create form and submit the invoices create form form…"
Technically grounded.
Linguistically deceased.
That exposed another boundary.
We had already separated:
what the application does from what the model thinks it does
Now we needed to separate:
what we know from what a human actually needs to hear
So the architecture gained another layer:
Product Model → what do we know?
Guidance Compiler → what is worth telling the user? ← new
Renderer → how should we say it?
No verification rules loosened.
No extra authority for the model.
We were finally admitting something painfully obvious:
Technically correct data structures do not automatically know how to talk to humans.
Shocking discovery for software developers, I know.
Statewave Guide — Day 4.
Next question:
Can we turn verified product truth into human guidance without inventing anything new?
This day in the repo: PR #4 · Previous: Day 3 — True is not the same as relevant
Statewave Guide is built on Statewave — an open-source, self-hosted memory runtime for AI agents. Governance and provenance, not just retrieval. Apache 2.0.
→ statewave.ai · github.com/smaramwbc/statewave
Every post in this series: The Statewave Guide Journey Index