{"slug": "show-hn-ghostbench-do-ai-models-recommend-dead-saas-i-tested-my-own-thesis", "title": "Show HN: GhostBench – do AI models recommend dead SaaS? I tested my own thesis", "summary": "A developer released GhostBench, a benchmark that tested whether AI models recommend shut-down SaaS vendors, finding that 19 of 20 dead products' domains still return HTTP 200 and that Claude Haiku 4.5 presented 8 of 20 dead vendors as sign-up-today viable. The accompanying kyv convention publishes Ed25519-signed, offline-verifiable company attestations at /.well-known/attestations.json, anchored in a daily Merkle-root transparency log that began 2026-09-30, with the reference issuer's own attestation reading mrr_usd: \"<1000\" and continuity 0 months. The project argues that signed, continuously renewed aliveness (alive_since plus continuity) is the signal separating a live vendor from a ghost, and is released as an MIT-licensed v0 draft with an MCP server exposing verify(domain).", "body_md": "Signed, banded, machine-verifiable operational facts about a company, published at its own\n`/.well-known/attestations.json`. Ed25519-signed, offline-verifiable, anchored in a daily\ntransparency log. **Facts only — no rankings, no scores. Absence never means negative.**\n\nPayments for AI agents have a foundation (x402). Agent identity has one (ANS). Nobody answers\nthe remaining question: *is the company on the other end of the pipe real — and continuously\nreal?* This repo is a small convention for that layer, plus a reference implementation and the\nbenchmark that motivated its design.\n\nWe collected 20 verifiably shut-down SaaS products (shutdown-evidence URL per entry) and 6\nalive controls, then asked models about each as if picking a vendor\n(`kyv/ghostbench/`, re-runnable):\n\n- **19 of 20 dead products' domains still return HTTP 200** — parked pages, farewell pages,\nredirects. The web will not tell you a company died.\n- A frontier model (Claude Opus 4.8) flagged **20/20** dead with no tools — famous shutdowns\nare in the weights. (Documented bias: requiring verifiable death skews the set famous;\nreal exposure is the long tail and post-cutoff shutdowns.)\n- The small/fast model class that actually runs inside agent loops (Claude Haiku 4.5)\npresented **8/20** dead vendors as sign-up-today viable — and a facts-only verification\nresult rescued 3 while**newly breaking 2** : it read`https_reachable: true` as proof of\nlife. Weak facts can be worse than no facts.\n\nThe signal that separates a live vendor from a ghost is **signed, continuously renewed\naliveness** — `alive_since` + `continuity`, where a missed day in the log is a visible gap.\nThat is Level 1 of this convention.\n\n| Level | Content | Cost to publish | \n|---|---|---|\n| 0 | domain + public key | one DNS TXT or one file, ~5 min | \n| 1 | aliveness + continuity (no financials) | near zero | \n| 2 | banded financials (e.g. `\"grr_pct\": \">=85\"` ) | bands only, never raw | \n| 3 | retention depth (cohort bands) |  | \n\nWitness states: `observed` (mechanical facts a verifier gathers — the default) / `self`\n(domain holder signs; the log witnesses continuity) / `issuer` (a third party verified\nread-only payment data). **Multiple issuers are welcome — the spec and the log are the root\nof trust, not any single key.** Full draft: [SPEC.md](https://github.com/kyv-attest/kyv/blob/main/SPEC.md).\n\n```\nnpm install         # or pnpm install\n\n# verify a domain's published attestations + observed facts (offline except the fetches)\nnpx tsx kyv/preflight.ts example.com\n\n# run it as an MCP server exposing verify(domain)\nnpx tsx kyv/mcp-server.ts\n\n# OpenAI-style tool schema\ncat kyv/openai-tool.json\n\n# publish your own: generate a key, sign an attestation\nnpx tsx kyv/cli.ts keygen\nnpx tsx kyv/cli.ts sign your-attestation.json\n\n# re-run the benchmark (needs ANTHROPIC_API_KEY)\nnpx tsx kyv/ghostbench/run.ts --model claude-opus-4-8\n\nnpm test\n```\n\nThe reference issuer's own attestation (`public/.well-known/attestations.json`) reads\n`mrr_usd: \"<1000\"`, continuity 0 months — the transparency log started 2026-09-30 and history\ncannot be backfilled. A verification convention that starts by flattering its author isn't one.\n\nTransparency log: daily Merkle roots + OpenTimestamps receipts live in\n[`kyv-attest/log`](https://github.com/kyv-attest/log).\n\n- Bands only, never raw values — non-conforming documents are reported invalid and never rendered.\n- `null` / absence NEVER means negative. Absence is non-adoption, not a signal.\n- No rankings, no scores, no leaderboards — a credential layer must judge nothing.\n- Verifier consumers: `observed` ≠`attested` ;`https_reachable` only means the domain served\na response — parked and farewell pages do too.\n\nv0 draft. Feedback wanted on the band vocabulary (SPEC §7) and on what an agent runtime would\nneed before calling `verify(domain)` by default. MIT.", "url": "https://wpnews.pro/news/show-hn-ghostbench-do-ai-models-recommend-dead-saas-i-tested-my-own-thesis", "canonical_source": "https://github.com/kyv-attest/kyv", "published_at": "2026-10-03 09:05:14+00:00", "updated_at": "2026-10-03 09:36:26.526612+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-search", "agent-protocols", "structured-data"], "entities": ["GhostBench", "kyv", "Claude Opus 4.8", "Claude Haiku 4.5", "Anthropic", "x402", "ANS", "OpenTimestamps"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-ghostbench-do-ai-models-recommend-dead-saas-i-tested-my-own-thesis", "markdown": "https://wpnews.pro/news/show-hn-ghostbench-do-ai-models-recommend-dead-saas-i-tested-my-own-thesis.md", "text": "https://wpnews.pro/news/show-hn-ghostbench-do-ai-models-recommend-dead-saas-i-tested-my-own-thesis.txt", "jsonld": "https://wpnews.pro/news/show-hn-ghostbench-do-ai-models-recommend-dead-saas-i-tested-my-own-thesis.jsonld"}}