cd /news/ai-agents/show-hn-ghostbench-do-ai-models-reco… · home › topics › ai-agents › article
[ARTICLE · art-144365] src=github.com ↗ pub= topic=ai-agents verified=true sentiment=· neutral

Show HN: GhostBench – do AI models recommend dead SaaS? I tested my own thesis

A developer released GhostBench, a benchmark that tested whether AI models recommend shut-down SaaS vendors, finding that 19 of 20 dead products' domains still return HTTP 200 and that Claude Haiku 4.5 presented 8 of 20 dead vendors as sign-up-today viable. The accompanying kyv convention publishes Ed25519-signed, offline-verifiable company attestations at /.well-known/attestations.json, anchored in a daily Merkle-root transparency log that began 2026-09-30, with the reference issuer's own attestation reading mrr_usd: "<1000" and continuity 0 months. The project argues that signed, continuously renewed aliveness (alive_since plus continuity) is the signal separating a live vendor from a ghost, and is released as an MIT-licensed v0 draft with an MCP server exposing verify(domain).

read3 min views1 publishedOct 3, 2026
Show HN: GhostBench – do AI models recommend dead SaaS? I tested my own thesis
Image: Michielbdejong (auto-discovered)

Signed, banded, machine-verifiable operational facts about a company, published at its own /.well-known/attestations.json. Ed25519-signed, offline-verifiable, anchored in a daily transparency log. Facts only — no rankings, no scores. Absence never means negative.

Payments for AI agents have a foundation (x402). Agent identity has one (ANS). Nobody answers the remaining question: is the company on the other end of the pipe real — and continuously real? This repo is a small convention for that layer, plus a reference implementation and the benchmark that motivated its design.

We collected 20 verifiably shut-down SaaS products (shutdown-evidence URL per entry) and 6 alive controls, then asked models about each as if picking a vendor (kyv/ghostbench/, re-runnable):

  • 19 of 20 dead products' domains still return HTTP 200 — parked pages, farewell pages, redirects. The web will not tell you a company died.
  • A frontier model (Claude Opus 4.8) flagged 20/20 dead with no tools — famous shutdowns are in the weights. (Documented bias: requiring verifiable death skews the set famous; real exposure is the long tail and post-cutoff shutdowns.)
  • The small/fast model class that actually runs inside agent loops (Claude Haiku 4.5) presented 8/20 dead vendors as sign-up-today viable — and a facts-only verification result rescued 3 whilenewly breaking 2 : it readhttps_reachable: true as proof of life. Weak facts can be worse than no facts.

The signal that separates a live vendor from a ghost is signed, continuously renewed aliveness — alive_since + continuity, where a missed day in the log is a visible gap. That is Level 1 of this convention.

Level Content Cost to publish
0 domain + public key one DNS TXT or one file, ~5 min
1 aliveness + continuity (no financials) near zero
2 banded financials (e.g. "grr_pct": ">=85" ) bands only, never raw
3 retention depth (cohort bands)

Witness states: observed (mechanical facts a verifier gathers — the default) / self (domain holder signs; the log witnesses continuity) / issuer (a third party verified read-only payment data). Multiple issuers are welcome — the spec and the log are the root of trust, not any single key. Full draft: SPEC.md.

npm install         # or pnpm install

npx tsx kyv/preflight.ts example.com

npx tsx kyv/mcp-server.ts

cat kyv/openai-tool.json

npx tsx kyv/cli.ts keygen
npx tsx kyv/cli.ts sign your-attestation.json

npx tsx kyv/ghostbench/run.ts --model claude-opus-4-8

npm test

The reference issuer's own attestation (public/.well-known/attestations.json) reads mrr_usd: "<1000", continuity 0 months — the transparency log started 2026-09-30 and history cannot be backfilled. A verification convention that starts by flattering its author isn't one.

Transparency log: daily Merkle roots + OpenTimestamps receipts live in kyv-attest/log.

  • Bands only, never raw values — non-conforming documents are reported invalid and never rendered.
  • null / absence NEVER means negative. Absence is non-adoption, not a signal.
  • No rankings, no scores, no leaderboards — a credential layer must judge nothing.
  • Verifier consumers: observed ≠attested ;https_reachable only means the domain served a response — parked and farewell pages do too.

v0 draft. Feedback wanted on the band vocabulary (SPEC §7) and on what an agent runtime would need before calling verify(domain) by default. MIT.

── more in #ai-agents 4 stories · sorted by recency
── more on @ghostbench 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/show-hn-ghostbench-d…] indexed:0 read:3min 2026-10-03 · —