{"slug": "someone-just-published-open-knowledge-base", "title": "Someone just published open knowledge base", "summary": "Open Knowledge Base (open-kb), an open-source tool that maps a company's market by searching for the job each product does rather than the company name, was published on GitHub. The tool, which requires Node 20+, a Bright Data account, and an OpenRouter key, exports a folder of markdown files with entities, relations, and segments, and includes an llms.txt file for agent access. In a bake-off of five configurations, the DeepSeek flash default was the cheapest at $0.00065 per entity, compared to $0.0008 for the next best and $0.002 for the dearest config that beat it on volume.", "body_md": "Point it at your domain and it maps the whole market — competitors,\nsubstitutes, integrations, buyers, and where the market argues. It never\nsearches the company's name: it works out the job each product does and\nsearches for the job. Every claim carries a URL and a literal quote from a page\nthe run fetched, and the map exports as a folder of markdown your agent can\nwalk, with `llms.txt`\n\nat the door.\n\nThe launch film — watch it with sound\n\nNode 20+, a [Bright Data](https://brightdata.com) account with a SERP and an\nUnlocker zone, and an [OpenRouter](https://openrouter.ai) key.\n\n```\ngit clone https://github.com/mo-root/open-kb.git && cd open-kb && pnpm install\ncp .env.example .env      # four keys, named in the file\npnpm sweep yourcompany.com\nruns/sweep-yourcompany-com-<stamp>.json   every entity, edge and span\n\nruns/exports/kb-yourcompany-com/          pnpm run export <run>\n├── entities/     one file per company, with its quotes\n├── relations/    one file per relation found: adjacent, competitor, substitute …\n├── segments/     each market, and who is in it\n└── llms.txt      the door an agent comes in through\n```\n\n**Why those two accounts.** Bright Data because a map is only as good as what\ncomes back: across the 17 sweeps in `runs/`\n\n, 11–15% of hosts stayed unreadable\nafter a direct fetch and one Web Unlocker escalation, and those stay on the map\nwearing the reason rather than vanishing from it. OpenRouter because the\njudgement does not need a frontier model: in the bake-off — five configs, one\ncompany, ten queries each — the DeepSeek flash default came in cheapest per\nentity at $0.00065, against $0.0008 for the next best and $0.002 for the dearest\nconfig that beat it on volume. Two configs did find roughly 2.4x more entities\nfor 3.5x and 7.7x the money; the default wins on cost per entity and on the\nrecall probe, at the same description grounding. That is why the cheap model is\nthe default and the spend caps are sized for it — not a claim that it finds the\nmost.\n\nOne judgement each, every answer in a schema. Each is a markdown prompt in\n[ prompts/](/mo-root/open-kb/blob/main/prompts) — changing how the engine thinks is a text edit. Six run\non every sweep:\n\n| agent | owns | runs |\n|---|---|---|\nunderstand |\nwhat the company sells, and which products share a market | once per run |\ncatalog |\na product's de-branded queries — the job, never the name | once per product |\nassess |\nwiden, deepen or stop, racing the search | up to eight times |\nclassify |\nwhat a host is, with its page in hand | once per host on the default path |\nlink |\nhow two entities relate | 40 pairs a call |\norphan |\na relation for an entity no pair reached | 20 at a time |\n\nThree more run BY DEFAULT since 2026-08-22 — each survived an A/B on a stored\nrun and each has an opt-OUT environment variable: **listicle-harvest** mines\nthe vendor names a roundup already printed (`OPENKB_LISTICLE_HARVEST=0`\n\nturns\nit off — found Windsurf, Zed, Tabnine, Codeium, Aider and Continue on\ncursor.com with zero direct SERP hits, and 18 real vendors on grundfos.com);\n**triage** skips hosts from search metadata before a fetch is spent\n(`OPENKB_TRIAGE=0`\n\n— 4.6% of hosts skipped, pooled over the 28 runs that\nrecord it, 1,283 of 28,182; per-market it ranges 0.3% to 13.3%);\n**second-look** re-asks **classify** itself against a deeper page for hosts\nleft `unknown`\n\n(`OPENKB_SECOND_LOOK=0`\n\n— 716 asked and 324 rescued over those\nsame runs; a third of the asks never get a page at all). Two more stay opt-in, one environment variable each: **discover** and\n**group** replace the single understand call (`OPENKB_DISCOVERY=agent`\n\n);\n**drop-confirm** gives every settled `none`\n\na second batched opinion\n(`OPENKB_DROP_CONFIRM=1`\n\n) — its own A/B did not survive, rescuing 0 of 12, 0 of\n27 and 5 of 29 across three runs.\n\n``` php\nflowchart TD\n    D[domain] --> U[\"understand<br/>reads the site\"]\n    U --> C[\"catalog<br/>per product, in parallel\"]\n    C --> Q[\"query queue\"]\n    Q --> W[\"SERP worker pool<br/><i>every hit tagged with its query</i>\"]\n    W --> A{\"assess\"}\n    A -->|\"widen: more queries\"| Q\n    A -->|\"deepen: 2 pages to 4\"| Q\n    A -->|\"enough\"| H[\"listicle-harvest<br/><i>default on</i>\"]\n    H --> T[\"triage<br/><i>default on — skip before fetching</i>\"]\n    T --> K[\"classify<br/>page in hand\"]\n    K --> S[\"second-look · drop-confirm<br/><i>second-look default on · drop-confirm opt-in</i>\"]\n    S --> L[\"link\"] --> O[\"orphan\"] --> M[\"the map\"]\n```\n\nOne liberty in the drawing: assess is not a gate the workers wait at — it\nraces the pool, and both widening and deepening land mid-flight.\n\nAgentic where the answer is a judgement, code where the answer is a guarantee:\n\n| guarantee | held by |\n|---|---|\n| A citation exists only if its quote is a literal substring of bytes this run stored |\n`core/src/evidence.ts` |\n\n`core/src/judge.ts`\n\n`competitor`\n\nand `substitute`\n\nneed that host's own readable page — a listicle nominates, it never convicts. `adjacent`\n\n, the softest placement and the most common one, carries no such bar`core/src/verdict.ts`\n\n`core/src/spend-cap.ts`\n\nA model having a bad day writes a weak query or misreads a host. It cannot fabricate a citation or blind a market.\n\nThe sweep buys breadth in one pass; the swarm buys depth. A lead agent writes missions onto a priced board, six lanes claim and work them with search and page tools, and every mission reserves its allowance before any work starts. A finish the scorecard objects to comes back refused — work clears a refusal, restating the objection does not.\n\n```\npnpm swarm yourcompany.com 5                          # depth, with a ceiling\npnpm swarm yourcompany.com 5 --from-sweep runs/<run>  # interrogate a sweep's map\n```\n\n[ARCHITECTURE.md](/mo-root/open-kb/blob/main/ARCHITECTURE.md) covers both engines phase by phase;\n[DEPLOY.md](/mo-root/open-kb/blob/main/DEPLOY.md) covers putting it on a host.\n\n```\nset -a && source .env && set +a   # the CLI reads keys from the shell\n\npnpm sweep yourcompany.com        # breadth: the map\npnpm sweep yourcompany.com --quick  # a bounded first look: seals the search early, no paid link pass\npnpm swarm yourcompany.com 5      # depth, with a ceiling\npnpm run export <run> vault  # the map as a folder of markdown\npnpm run diff a.json b.json  # what moved between two runs of one anchor\npnpm run audit <run>         # deal a review packet, score it symmetrically\npnpm test                    # the full suite, offline, no network, no keys\npnpm check                   # CI's gate: three guards, tsc, five test projects\n\ncd packages/web && pnpm dev  # the app, http://localhost:3210\n```\n\nAn [Agent Skill](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview)\nat [ skills/mapping-markets](/mo-root/open-kb/blob/main/skills/mapping-markets) teaches Claude Code and\nother agents to run maps, read them, and tune the query doctrine:\n\n```\nnpx skills add mo-root/open-kb/skills/mapping-markets\nopen-kb/\n├── packages/\n│   ├── core/        pure logic: evidence mint, query families, span accounting\n│   ├── providers/   Bright Data SERP + Unlocker, OpenRouter wiring\n│   ├── sweep/       the breadth engine, one file\n│   ├── swarm/       the depth engine: a lead, a funded board, six lanes\n│   └── web/         Next.js: live run surface and the map\n├── prompts/         every judgement, as editable markdown\n└── skills/          the Agent Skill\n```\n\n**Bright Data** (SERP API, Web Unlocker) for searches that do not get blocked ·\n**OpenRouter** via AI SDK 7, answers typed with Zod · **Next.js 16** for the\napp · **Supabase** optional locally, required on Vercel.\n\nMIT. Use it, fork it, ship it.\n\nBuilt on Bright Data's web infrastructure. Not affiliated with, endorsed by, or sponsored by Bright Data.", "url": "https://wpnews.pro/news/someone-just-published-open-knowledge-base", "canonical_source": "https://github.com/mo-root/open-kb", "published_at": "2026-08-26 06:11:57+00:00", "updated_at": "2026-08-26 06:45:07.372131+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "artificial-intelligence"], "entities": ["Open Knowledge Base", "GitHub", "Bright Data", "OpenRouter", "DeepSeek", "Windsurf", "Zed", "Tabnine"], "alternates": {"html": "https://wpnews.pro/news/someone-just-published-open-knowledge-base", "markdown": "https://wpnews.pro/news/someone-just-published-open-knowledge-base.md", "text": "https://wpnews.pro/news/someone-just-published-open-knowledge-base.txt", "jsonld": "https://wpnews.pro/news/someone-just-published-open-knowledge-base.jsonld"}}