The problem: marketing promises and the real product drift apart silently. AI growth agents make it worse: they write promises at scale, and they learn from the numbers those promises produce.
Who: solo founders who let an AI agent write and post their marketing, and judge what works from tracking tags.
When it breaks:
- At publish time, the link lies. Every App Store button on my app's site, scribo.cc, overwrites the campaign tag with
ct=site-en-hero. Every install from my Threads posts would have been credited to the website. No error, just a wrong number, and my agent would have concluded Threads doesn't work and stopped posting there. - After every release, the product moves on. Scribo 2.0 (October 1) turned a handwriting vocabulary app into a content learning app. Old @scribo.voca posts still promise a handwriting vocabulary notebook and keep bringing people in, who then find a different product and may leave.
The fix: the agent drafts a post and turns every promise in it into a check. Kane CLI checks each one in a real browser before the post ships (the link's tag, every sentence against the page, every promised feature), and again after every release (Re-verify for campaigns, Audit past posts for old posts). Failed promises are repaired from what Kane saw. Only a campaign that passes can publish.
Proof:
- A real post went out through the gate: @scribo.voca/post/De4-yPnATF5 . The button unlocked only because Re-verifyKR-85 passed that exact version (
a9480fade25c1e68). - The link lie, caught: Kane read
ct=site-en-heroon the button, FAIL (KR-63). The agent rebuilt the link from the URL Kane saw, PASS (KR-68). The same in all five full loops. - The kanehack recorder logged KR-46 (Kane-designed tests, 2/2),KR-77 andKR-85 (Re-verify) as PASSED (VERIFIED source). Every run is inRUNS.md .
- Supporting example: feature claims on TestMu AI's free JSON tool. I seeded a false "convert JSON to YAML" claim to show the repair: FAIL (KR-100), the agent swapped in Minify, PASS (KR-102).
Before my agent learns from a post, Kane proves the funnel tells the truth.
bun install && bun run dev
You need:
.envwithOPENAI_API_KEYandOPENAI_MODEL(I usegpt-5.4-mini, picked by timing the models this key can list).kane-cli(0.10.0-rc.3) signed in, andkanehackpaired. Every Kane call goes throughkanehack report -- kane-cli β¦.- Optional, once per app:
bun run explore https://your.appmaps the site with Kane Explore intoexplore/<host>.json; drafts for that app are grounded in the map. scribo.cc is already mapped.
bun test runs the parser and evaluator tests against real Kane output saved in fixtures/ (no credits).
The left panel also lists recorded runs. Clicking one replays it on the same timeline without spending credits, labeled as recorded.
A Verified card has a Publish to Threads button. It stays locked unless the latest Kane result (the last attempt, or the last Re-verify) is a PASS for this exact campaign version (a hash of the copy, link and claims), and the campaign isn't already published. Publishing takes a click and then a confirm that shows the Threads account, the exact text and the Kane result it rests on; the server checks the gate again and needs the confirmed version.
PUBLISH_MODEdefaults to dry run: the confirm checks the Threads account and records the attempt, and nothing is posted. SetPUBLISH_MODE=livein.env(and restart) to post for real.- One real post has gone out this way: @scribo.voca/post/De4-yPnATF5 , from campaign
c-2026-10-24T19-41-41Z, gated on Re-verify 2 (KR-85 PASSED) for versiona9480fade25c1e68. - Posting goes through the existing Hermes script,
~/.hermes/scripts/threads_api.py, withTHREADS_PROFILE(defaultscribo, i.e. @scribo.voca;defaultuses the script's default token file). The script reads its own token file; Omni Verify never reads or stores a token. - The published post URL is stored with the campaign (
runs/c-*.json,published) and shown on the card.
Marketing and the product drift apart silently. Case 1 above: the link breaks the data. Case 2: the product changes and old posts keep promising the old one. Scribo 2.0 (Oct 1) turned a handwriting vocabulary app into a content learning app, but @scribo.voca's September posts still say "μμΌλ‘ μ μ΄μ μ μ₯νλ λ¨μ΄μ₯" (a vocabulary notebook you fill in by hand), and they keep bringing people in. Page 3, /audit.html:
posts (Korean) β OpenAI: each product promise, Korean words copied exactly + English gloss
β Kane reads scribo.cc/ko and scribo.cc once each: scroll once, then per promise store true/false + the sentence it relied on
β code judges: either page says it = PASS, both say no = FAIL, missing = UNVERIFIED
β per post with a FAIL, the agent proposes one action: a Korean rewrite that only promises what the site says today, or unpin/archive
β Re-audit replays Kane's recorded tests (_test.md) after the next release
Nothing on Threads is edited, deleted or posted; the proposals are for a person.
Audit a-β¦20-46-49Z, Kane KR-125 (scribo.cc/ko) and KR-126 (scribo.cc), both pages agreeing on every promise:
| Post | Promise | Kane, today's site | Proposed action |
|---|---|---|---|
| 2026-09-18 DddN67NlHjn | μμΌλ‘ μ μ΄μ μ μ₯νλ λ¨μ΄μ₯ (a notebook where you save words by hand) | FAIL | Rewrite: hold a word to see its meaning and save it; iPhone version is out |
| μμ΄ν° λ²μ μΆμ (an iPhone version) | PASS | ||
| 2026-09-22 DdnaDR8GxkH | 리뷰 κΈ°λ₯ (a review feature) | PASS: "μ μ₯ν λ¨μ΄λ μμ λμ―€ λ€μ λμμ." | Rewrite: saved words come back in quizzes |
| μ λ³Έ λ¨μ΄λ₯Ό μμμ λμμ€ (unseen saved words resurface) | PASS, same sentence | ||
| μκΈμ¨λ‘ νλ² λ μ¨λ³΄λ©΄μ λ³΅μ΅ (review by writing words again by hand) | FAIL : the site only says "On iPad, write your answers with Apple Pencil", and Kane did not count that | ||
| 2026-10-05 DeI1ZVXj8Po | Scribo μκΈμ¨ λ¨μ΄μ₯ (Scribo, the handwriting vocabulary notebook) | FAIL | Unpin / archive: the whole post is about the old product |
Where the posts came from. The read-only API listing (threads_api.py --profile scribo my-posts) needed an approval the build session couldn't give, so Kane read the posts instead: the public profile is behind a login wall (KR-121, Kane stuck on the dialog), but the three post pages are public (KR-123, text stored exactly). They're in fixtures/scribo-posts.json, labeled with that source; GET /api/audit/posts?refresh=1 runs the API listing when the server is started by a person. Dates are as Threads showed them in US Pacific time (one day behind KST).
Rewriting old posts isn't enough: the writing agent must never repeat a retired promise. So Kane's verdicts become the agent's memory, memory/product-facts.json:
- Confirmed : promises Kane found on today's site, with the exact sentence it relied on, the page, the run and the date. From the audit's PASSes and the launch gate's sentence checks (6 today).
- Retired : promises Kane found the site no longer makes, with Kane's FAIL evidence, the run, the date and the old post (3 today: μκΈμ¨ λ¨μ΄μ₯ / the handwriting vocabulary notebook, review by writing words again by hand, Scribo μκΈμ¨ λ¨μ΄μ₯).
- Unverified : Kane gave no answer. Neither confirmed nor retired.
The audit and Re-audit merge their verdicts in when Kane finishes: a PASS confirms, a FAIL retires, and an UNVERIFIED never overrides either. A verified campaign adds the sentences Kane found. The drafting agent gets the memory in its prompt, may only state confirmed facts, and every draft is checked before it's accepted: an exact-wording guard for retired promises, then one model check for rewordings and unconfirmed sentences. A draft that slips is refused and redrafted once with the reason. The Draft step shows "Grounded in: N confirmed facts, M retired promises excluded".
Proof (OpenAI only, no Kane). Goal: "Get Korean learners who used to journal vocabulary by hand to try Scribo again", a goal that invites the old promise. The draft (runs/c-2026-10-24T21-05-53Z.json, replayable as DRAFT in the campaign page's recorded runs) only states confirmed facts:
Learn a language from the videos, articles and PDFs you actually care about. Words you saved come back in your quiz just before you'd forget them. https://scribo.cc/?ct=threads-oct24
No handwriting notebook, memory check clean, nothing refused. To be honest about what that proves: two control drafts with memory turned off didn't promise handwriting either, because the agent already drafts from today's page text. What memory adds is the hard stop. Given two planted drafts, the check refused both: "Miss your handwriting vocabulary notebook?" (caught by the exact-wording guard and the model) and the reworded "Write each new word by hand again, and Scribo saves it in your notebook for review." (caught by the model alone), while a clean sentence passed.
Scribo (https://scribo.cc) is my own app. Every App Store button on the site has a hard-coded tracking tag (ct=site-en-hero, ct=site-en-nav, ct=site-en-get), and the site never reads the incoming URL's query. So a Threads post that links to https://scribo.cc?ct=threads-oct24 loses its tag at the button, and App Store Connect would credit the install to "site-en-hero". Attribution breaks silently: the post looks right, the click works, the numbers would be wrong. (I haven't checked App Store Connect numbers; this is what the tags say would happen.)
The demo loop, from the UI (runs/c-2026-10-24T19-26-21Z.json, 136 s):
| Step | What happened | Kane run |
|---|---|---|
| Draft | Post links to https://scribo.cc?ct=threads-oct24 ; 3 claims: 2 sentences of copy + the tracking tag |
|
| Kane run 1 | Both sentences found on the page. Kane read the main button's href before clicking: β¦id6794793715?pt=126975132&ct=site-en-hero&mt=8 .FAIL : the campaign'sct=threads-oct24 does not survive |
KR-63, Test Manager |
| Repair | The agent read Kane's href and rebuilt the link from it: same App Store URL, pt andmt kept,ct set tothreads-oct24 |
|
| Kane run 2 | Both sentences found again; the post's link carries ct=threads-oct24 and Kane landed on App Store id6794793715.PASS |
KR-68, Test Manager |
| Result | Verified for launch, with a diff from the first draft |
The same FAIL β repair β PASS happened in all five full loops run today (runs/c-*.json). The first FAIL, before the loop existed, is KR-6 (Test Manager).
input (app URL, goal, audience)
β draft (OpenAI, structured JSON, grounded in the Kane Explore map + page text)
post_copy, cta_url (with ct=), claims[] β assertions
β compile: all assertions β one plain-English Kane objective
β kanehack report -- kane-cli run "<objective>" --url β¦ --agent --headless --assertion-failure-mode continue --name <campaign>-a<n>
β parse NDJSON: steps stream to the page over SSE; verdicts come from run_end (context.memory)
β PASS: Verified for launch
FAIL / UNVERIFIED: repair (OpenAI) gets the failing assertions + what Kane saw + Kane's summary,
returns a revised campaign + one-line reason β Kane again (max 3 attempts)
β Re-verify (button on the Verified card):
kanehack report -- kane-cli testmd run .testmuai/tests/<campaign>-a<n>_test.md --url β¦ --agent --headless
replays Kane's own recording of the passing attempt; the values its checks stored are read from the
replay's evidence pack and judged by the same evaluator
- Kane extracts, the code judges. The objective asks Kane tostore what it sees (the button's href, the landing URL, true/false per sentence). The evaluator compares those values to the assertions, so the verdict is the same check on every run and cites the Kane step it came from.
- Two assertion types.
param_preserved: the link that hands the visitor to the App Store carries the campaign's ownct.text_visible: each sentence of the post is backed by a phrase Kane finds on the app's page. A draft that leaves any sentence without a claim is rejected before Kane runs. - Repairs are constrained. The agent may fix the link (only from URLs Kane observed), or rewrite/drop a sentence Kane didn't find. It may not change the campaign tag or drop the attribution claim, and every repair is re-validated. In a dry run the model first copied Kane's href verbatim (still
site-en-hero); the validator rejected it, and the agent gets one retry with the reason. - No Kane result means no PASS. A missing
run_endor a value Kane didn't report is UNVERIFIED, which is never treated as a pass. - The commit history records each time a Kane result changed what the agent or the design did.
| Feature | How | Evidence |
|---|---|---|
kane-cli run --agent |
Every verification attempt; NDJSON parsed live | runs/ ,fixtures/kane-sample.ndjson (Step 0 schema) |
kane-cli explore |
Read-only local exploration of scribo.cc, 12 leases: 7 screens, 7 flows verified, 20 findings (mostly accessibility). Showed every App Store button links to apps.apple.com/app/apple-store/{slug}?ct&mt&pt , i.e. sets its ownct |
explore/scribo.cc.json ,explore/scribo.cc.ndjson ,explore/scribo.cc-steps/ |
| Assurance loop | The verified campaign became a requirement ( requirements/campaign-threads-oct24.md ).context ingest β use case β owner review βdesign tests (5 criteria, 3 scenarios, 2 tests, 2 gaps) β owner review βmaintain evolve βtestrun run (2/2 passed,KR-46 ) βcover |
evidence/assurance/ ,.testmuai/tests/ |
| Coverage | 75% : 3 of 4 criteria proven by real runs | evidence/assurance/11-cover.ndjson |
--name +testmd run replay |
Every attempt is a named run, so Kane records it as a _test.md . The passing one is the campaign's regression test;Re-verify replays it (about 10 s, no authoring) and re-judges every claim from the values the replay computed. First replay: all 3 claims PASS,KR-77 PASSED (VERIFIED source) |
.testmuai/tests/c-*_test.md ,runs/*-rv*.json ,docs/reverify.png |
Why 25% is unmeasurable. The fourth criterion is "installs from the Threads post are credited to threads-oct24". Kane itself recorded it as an unmeasurable gap: install credit shows up in App Store Connect, not in any page a browser can open. What Kane can prove is that the hand-off link carries the right ct.
Owner decisions in the loop. Kane proposed a criterion "the App Store hand-off retains ct after opening". The App Store strips the query string from the address bar after load (Step 0: Kane's final URL has no query), so it can't be observed. It was rejected and archived with that reason, and maintain evolve had Kane supersede the test that asserted it (t-1 β t-3). Two design --force runs before that answered nothing_to_change; the archive was what made the redesign happen.
The build brief said scribo.cc shows "Try Pro free for 7 days" in the pricing section, below the fold. It does: <b>Try Pro free for 7 days</b> is in the HTML. Asking Kane different ways (all real runs, saved in fixtures/):
| Phrasing | Result |
|---|---|
| "Verify the text 'Try Pro free for 7 days' is visible" (KR-51) | FAIL , a false negative: Kane's generated check read the wrong element |
| Scroll first, then verify (KR-54) | Trial PASS, but the run got stuck on an absent price and never judged the third check |
--analyzer-only --condition β¦ (KR-55) |
[null, false, true] : null on the below-fold trial |
| "Scroll down once, then store whether the text 'β¦' appears as 'claim_n'" (KR-58) | true / false / true, all correct, no stuck run |
Omni Verify compiles to the last phrasing. Trusting the first one would have made the repair agent delete a true sentence.
Text checks prove a sentence is on a page, not that the feature works. For posts about a tool, every sentence that promises a feature is performed by Kane in real Chrome. Target: TestMu AI's free online tools (JSON Prettify). UI at /features.html.
tool URL β page text (the tool's own section) β OpenAI drafts the post; per sentence: control, action (exact sample input), expected result
β one Kane run per claim: store whether the control exists, do the action only if it does, store the result and whether it matches
β code judges: control missing or result wrong = FAIL, nothing stored = UNVERIFIED (re-run unchanged), both true = PASS
β FAIL: the agent rewrites or drops that sentence (passing ones must stay word for word) β Kane runs only the new claim
Stretch mode (on by default in the UI, labeled) tells the agent to add one format conversion the page doesn't show, so the gate has a false promise to catch.
| Run | What happened |
|---|---|
| f-β¦20-04-05Z | Prettify PASS (KR-96), Validate PASS (KR-98, Kane read "Valid!"), "Convert JSON to YAML" FAIL (KR-100,Test Manager ): Kane clicked the button, the output was still JSON. The agent swapped in Minify βPASS (KR-102,Test Manager ). Verified for launch. |
| f-β¦19-53-00Z | "Converts the same input into CSV" FAIL : Kane found no CSV control (KR-90). The two true claims came back UNVERIFIED: an "if it does not exist, stop" branch made Kane stop before storing the result. The branch was removed, and the run stopped because the repair was asked to rewrite UNVERIFIED sentences too. Now only FAILs go to repair. |
| f-β¦19-58-22Z | Validate UNVERIFIED (KR-93), re-run unchanged β PASS (KR-95). Its stretch sentence ("Clear the fieldβ¦") was a real feature. Its KR-94 PASS stored "none" as the result. That now counts as UNVERIFIED, and the stretch prompt now asks for a format conversion. |
Seen on the real tool: JSON Prettify's tab-indented output puts spaces around colons ("a" : 1). Kane's output is in each run file.
-
Phrase-level backing. A sentence is backed by a 3 to 12 word phrase copied from the page. The phrase can prove only part of a longer sentence (in the demo run, "rewrites them for the learner you are today" backs a longer sentence that is on the page in full, but the check covers only the phrase).
-
It stops at the App Store. Kane checks the link and that it lands on the right listing. It does not install the app or follow the funnel inside it.
-
Verification is not measurement. A PASS means the funnel keeps the post's promises today. It does not mean installs went up, or that App Store Connect credited them (see the unmeasurable 25%).
-
Recorder bug H-0007. kane-cli 0.10.0-rc.3 sends a
coverageevent beforerun_end, so the kanehack recorder never seesrun_endas the completion event and marks passingkane-cli runcalls as ERRORED (UNVERIFIED). Reported as help ticket H-0007. My own record of every run (time, Kane's own status, credits, Test Manager link) is inRUNS.md .testrunis unaffected: KR-46 shows PASSED. -
Pre-launch post. Nothing is published before the gate passes, so Kane's designed tests open
/post/<id>, a preview page Omni Verify serves, labeled "not published". -
Re-verify has not caught a regression yet. The replays so far passed; the checks are re-computed during the replay (their timestamps in the pack are the replay's), but a FAIL needs the site to change. A replay whose recordings no longer match the page heals by re-authoring, which costs more and is slower.
-
The test files of failed attempts stay in
.testmuai/tests/too (Kane writes every named run); only the passing one is linked as the regression test. -
Scope: one app (Scribo), one channel, App Store links only. Feature claims: tested on one TestMu AI tool (JSON Prettify).
-
Feature claims trust Kane's true/false. The code judges from Kane's stored values, and "does the result match" is Kane's own judgment (the result text is kept next to it). A stretch claim may name an existing control (the YAML claim used Json Prettify), so it fails on a wrong result instead of a missing control.
-
docs/verified-card.png : the Verified card with every assertion and the diff from the first draft.
-
docs/live-run.png : the whole timeline of the demo run: draft, Kane run 1 FAIL, repair, Kane run 2 PASS, verdict.
-
docs/reverify.png : Re-verify replaying Kane's saved test and re-judging the claims.
| What | Where |
|---|---|
| Every Kane run: time, recorder label, Kane status, credits, Test Manager link | RUNS.md |
| Each campaign run: drafts, objectives, verdicts, repairs | runs/c-*.json (+ raw Kane NDJSON per attempt inruns/*.ndjson ) |
| Sealed Kane evidence packs, unedited (they are sealed, so they keep local paths) | evidence/loop/ (demo run, FAIL and PASS),evidence/assurance/packs/ (KR-46). View withkane-cli evidence serve <pack> |
| Assurance loop streams, verdict files, designed tests (before/after evolve), coverage | evidence/assurance/ |
| Kane Explore map and stream | explore/ |
| Real Kane output used by the tests | fixtures/ |
Built at Kane CLI Hack Day, 2026-10-24.
AI can write the promise. Kane checks if the product keeps it.