{"slug": "convergent-evidence", "title": "Convergent evidence", "summary": "A developer's in-house AI orchestration project evaluated nine external tools over ten days and adopted none as dependencies, finding that most duplicated existing capabilities. Two candidates—a stranger's routing config and an academic paper—offered novel ideas that were incorporated as policy changes rather than installs. The developer argues this pattern indicates the underlying routing problem is well-understood, with independent solutions converging on the same core principles.", "body_md": "Companion to [Route, don't guess](https://thekilted.dev/route-dont-guess/): nine tools crossed the desk during the ten-day build. Not one became a dependency. The reasons why are the actual argument for building it in-house.\n\nBuilding something in-house invites an obvious question: did you check whether someone already shipped it? A companion project, the one that tracks and evaluates AI tooling for the whole machine, kept an eye out for exactly that, every time an orchestration-shaped tool crossed its desk. The bar it held every candidate to was narrow on purpose: does it add a capability the orchestrator doesn't already have?\n\n**Almost nothing cleared it. That's less interesting as a verdict than as a pattern.**\n\nSix candidates were orchestrators or multi-agent harnesses solving the same problem: which model does which task, and how do you trust what comes back.\n\nThe second cluster doesn't route anything. These are desktop shells for running a fan-out once you've decided on one, checked for an unrelated reason rather than because they compete with the routing policy.\n\nTwo inputs cleared the bar, both as ideas rather than installs, arriving from opposite directions and landing on the same shape.\n\nA stranger's public routing config contributed one genuinely new distinction: taste as an axis separate from difficulty. A component name or a marketing hook is trivial to compute, but still wants the highest-taste model available. A difficulty-only rule would route that down, wrongly. The axis is now a permanent exception in the routing policy. The same source's \"manager never runs high reasoning effort\" rule was weighed and rejected; it conflicts directly with keeping the terminal review and hardest verification stages deliberately expensive.\n\nAnd an academic paper on filesystem-as-agent-architecture (numbered stage folders, a per-stage context file, human gates between them) turned out to overlap about 80% with the four-file working-notes system already in use. Independent convergence, this time from arXiv instead of a product pitch.\n\n**Neither produced a dependency. Both produced a stronger version of something that already existed.**\n\nFable Foreman is worth lining up in full, precisely because it's the closest external thing to the in-house build.\n\n| Dimension | Fable Foreman | The in-house orchestrator |\n|---|---|---|\n| Routing basis | Capability class: frontier / workhorse / fast | Verifiability + iteration count, down a cost ladder |\n| Model naming | Aliases resolved at runtime | Same. Tier by role, never a hardcoded model ID |\n| Verification | \"Trusts no one\": blind verify against the real workload | Same posture. Never trust a subagent's \"done\" without evidence |\n| Budget awareness | Budget-aware dispatch, degradation rule | Append-only ledger plus a budget halt |\n| Design/review routing | Not specified | Never routes down. A standing, explicit rule |\n| Trust surface | Four days old, spawns full agent fleets, content gated behind a CTA | In-house, auditable, every piece proven separately |\n\nThe market kept independently rediscovering the same routing model: a stranger's config file, an academic paper, three viral reels, and a four-day-old repo that maps onto our own build dimension for dimension. None became a dependency, because the same five patterns were already decomposed into separable, auditable pieces before any of them showed up.\n\nA single rejected tool proves nothing; every project rejects things. What's worth noticing is the shape of nine candidates checked and not one dependency adopted. Not one offered a capability the routing layer didn't already have, and the two that offered something real turned out to be ideas rather than products. That's what you'd expect if the underlying problem (route by verifiability, verify blindly, cap the budget, keep a human in the loop) is well-understood enough that independent people keep arriving at the same four or five pieces.\n\nThe discipline wasn't refusing to look. It was refusing to trade five pieces we can audit for one box we can't. This doesn't claim there were no more tools worth checking. Only that everything that crossed the desk was checked against the same bar, and the bar held.\n\n*Originally published at thekilted.dev/convergent-evidence.*", "url": "https://wpnews.pro/news/convergent-evidence", "canonical_source": "https://dev.to/thekilteddev/convergent-evidence-3ohk", "published_at": "2026-08-10 18:57:13+00:00", "updated_at": "2026-08-10 19:20:25.544332+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["Fable Foreman", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/convergent-evidence", "markdown": "https://wpnews.pro/news/convergent-evidence.md", "text": "https://wpnews.pro/news/convergent-evidence.txt", "jsonld": "https://wpnews.pro/news/convergent-evidence.jsonld"}}