{"slug": "kaiban-distributed-evaluation", "title": "Kaiban Distributed, Evaluation", "summary": "Kaiban Distributed, a new TypeScript/Node.js runtime, runs each AI agent as an isolated OS process communicating via message queues, enabling independent scaling, failure isolation, and human oversight through a live Kanban board. The open-source tool, built on KaibanJS, targets use cases from solo content pipelines to enterprise support surges, with a roadmap toward real-time data processing at scale.", "body_md": "Kaiban Distributed is a genuinely useful piece of software. This article checks its capabilities directly against the source code and separates three things cleanly:what ships today, what’s on the roadmap, and what’s a future vision.The aim isn’t to sell it or to criticize it — it’s to tell you exactly what you’d be getting, in plain language for non-technical readers and in verified detail for engineers. Where a description and the code differ, the code is treated as the source of truth.\n\nImagine a team of capable assistants — one researches, one writes, one edits, one checks the numbers. In most “AI agent” tools today, that whole team lives inside **one program on one computer**. It works, but it’s fragile: if one assistant trips, the whole team can fall over, and scaling up means a bigger, pricier machine.\n\nKaiban Distributed does it the way a *real* company does. **Each assistant is its own independent worker**, in its own room, with its own inbox. They don’t interrupt each other — they **pass notes** (messages). If one worker faints, the others keep going and a manager restarts it. If work doubles, you just **add more copies of the busy worker**. And above it all is a **live board (Kanban)** where a human can watch every task move *To-Do → Doing → Done* and step in on the hard calls.\n\nIn one breath: **an AI workforce you can watch, trust, scale, and control — one that fails safely instead of all at once.**\n\n🧑💻 **Personal / solo** — *“a reliable little assembly line on my machine.”* Run a research → draft → edit team on your own laptop with Docker to produce blog posts, newsletters, or reports, and watch it on a local board. This is what ships today (the *blog-team* and *global-research* examples). **Available now.**\n\n👥 **Small team / startup** — *“our first AI back-office, without a platform team.”* A few agents handling support drafts, lead research, content, or data enrichment — each independently deployable and scalable, with a human approving tricky cases and cost caps preventing surprise bills. You get serious plumbing (queues, retries, dead-letter recovery, audit logs) without building it. **Realistic today; you assemble the specific workflow.**\n\n🏢 **Enterprise** — *“AI that fits how we already run software.”* Because agents ride the **same message buses enterprises already run** (Kafka or Redis), they drop onto existing infrastructure. Natural fits: **support-surge handling** (add triage agents when tickets spike, scale back when calm), **document-heavy back office** (each invoice/claim/contract is a task, sensitive ones route to a human, everything logged), and **regulated review** (finance/insurance/legal, where HITL and a tamper-evident audit trail are exactly what teams ask for). *These are patterns you build on the runtime, not pre-packaged products.*\n\n🌍 **Frontier / “north-star”** — *“a thinking layer over massive real-time data.”* At the far end, the same design *could* sit over huge live-data streams — city-scale sensor networks, IoT fleets, environmental early-warning systems. The message-bus foundation is the right shape for it, but to be clear this is **a direction, not a shipped capability**: there are no sensor swarms running on it today. It’s where the architecture *points*.\n\nThe rest of this article verifies the engineering behind those promises.\n\n**Want to run it yourself?** The [kaiban-distributed-examples](https://github.com/andreibesleaga/kaiban-distributed-examples) repo has runnable, self-contained demos — the same agent code running as independent, message-passing, individually-scalable services, with the live Kanban board, checkpoint/resume, and human-in-the-loop — so you can watch the distributed substrate work end-to-end instead of taking any of the above on faith.\n\nIt’s a TypeScript/Node.js runtime that takes [KaibanJS](https://github.com/kaiban-ai/KaibanJS) — a Kanban-styled, role-based multi-agent framework — and runs each agent as an **isolated OS process (an “actor”)** that communicates only by messages over a **pluggable bus** (Redis/BullMQ by default, or Apache Kafka). A gateway exposes HTTP + WebSocket surfaces, a React/Zustand Kanban board renders state in real time, and the whole thing is wired for observability, federation (A2A + MCP), FinOps controls, and a hardened supply chain.\n\nThe repo’s own one-line pitch is careful and fair:\n\n“One of the first open-source projects to combine Enterprise Messaging (Kafka/Redis), Distributed Actor-Model Isolation, AI Multi-Agent Orchestration, and Kanban Visualization into a single JavaScript-ecosystem runtime.”\n\n**Are the primitives new? No.** The actor model dates to 1973 (Hewitt); production actor runtimes (Erlang/OTP, Akka, Orleans, Ray, Cloudflare Durable Objects) long predate this. Agent orchestration isn’t novel either — LangGraph(.js), CrewAI, AutoGen, Mastra, and KaibanJS all exist.\n\n**Is the combination plausibly first-of-kind in its niche? Yes.** As of 2025–2026 the JS/TS multi-agent field is dominated by\n\n**Algorithmic.** Local-mailbox processing, vertical scaling each node and horizontal worker scaling are real. The system is a sequential checkpointed pipeline plus fire-and-forget state broadcast; the queueing-theory and CAP framing describe intent, not enforced mechanism.\n\n**Systems architecture.** Clean/Hexagonal layering is actually enforced (madge no-cycles gate, a Result monad, api-extractor drift checks). The gateway/worker split is coherent. The weak spots are the **single-active orchestrator** and the **permanent Redis dependency**.\n\n**Reliability.** Worker-tier fault containment is solid (isolation, 3× retry, DLQ + selective replay, sliding-window circuit breaker, chaos-tested). The ceiling is the orchestrator: crash-safe, not HA. At-least-once delivery makes **idempotency the developer’s job** — and since each duplicate can trigger a paid LLM call, non-idempotent handlers are a direct cost risk.\n\n**Security & governance.** Strong for the category: opt-in/fail-closed controls, semantic firewall, JWT on board + A2A, channel signing, JIT tokens, the action gate, hash-chained audit, data caps (64 KB message / 20 KB state result), OWASP/NIST posture. Caveat: **default-off means insecure-by-default if operators don’t enable it.**\n\n**Developer & operational experience.** Good DX: one-command example scripts, a zero-setup HTML viewer plus a full React board, thorough docs and ADRs. The operational cost is real — Redis (always), possibly Kafka, a gateway, an orchestrator, and N workers, with two messaging semantics to monitor. More moving parts than an in-process orchestrator, by design.\n\n**Product.** A clear point of view: *your AI agents should be a visible, governable workforce, not a black-box script* — the board + HITL + guardrails express it well. The gap: it’s a **runtime/toolkit, not a finished app** — no hosted control plane, no pre-built industry workflows, no click-to-deploy SaaS. Right shape for developers; wrong shape for a non-technical buyer expecting turnkey.\n\n**UI/UX.** The Kanban board is the standout: a non-engineer can watch tasks move, read the economics panel (tokens/cost/duration), follow the event log, and act on the HITL modal (Approve/Revise/Reject) — plus a zero-setup static viewer and a11y tests. Limit: the polished board is a **separate React app you run yourself**, and day-to-day use still means .env files, Docker, and terminal scripts. Excellent legibility; developer-grade onboarding.\n\n**Managerial.** Maps well to “manager, not operator”: delegate → observe → intervene on exceptions, with answerable questions on purpose, transparency, and control. The cost is headcount and cognition — you’re operating a small distributed system, and the single-maintainer upstream means your team owns more risk. Budget for a competent platform engineer.\n\n**Data & privacy.** Thoughtful defaults: a PII-name denylist, strict data caps, per-tenant actor isolation, a hash-chained audit log. Caveats: controls are opt-in, the PII filter is **name-based** (won’t catch sensitive values under innocuous keys), and prompt/response content still leaves your boundary to whichever model provider you configure. Good scaffolding; not a substitute for your own governance review.\n\n**Maturity & bus factor.** A **single-maintainer, “production-shaped” but not yet production-proven** project at a version 2.0. The discipline (coverage, mutation, chaos, signed releases) is well above solo-project norms and partly compensates — but there’s no independent production track record and a real bus-factor risk.\n\n**Innovation.** The novelty is a *new arrangement of proven parts* — actor model, enterprise buses, the Kanban metaphor, and A2A/MCP interop, fused into one JS-native agent runtime. That combination is a credible **first-of-its-niche**, and the forward-looking pieces (federation, policy-as-code, FinOps) sit exactly where the industry is heading in 2026. The contribution is **integration and engineering discipline**, done cleanly — which is a real contribution in itself.\n\n**Enterprise.** What you buy is *operability*: agents running as independent, restartable services on **infrastructure you already operate**, with distributed tracing, a tamper-evident audit trail, opt-in JWT/mTLS/signing, cost ceilings, an action gate, and a signed, SBOM-backed supply chain — a serious, compliance-aware posture. The caveats to design around: a **single-active orchestrator (SPOF)**, a **permanent Redis dependency**, and **default-off security**; the marquee scenarios are patterns you build, not turnkey products. Pilot on a contained workload, prove the controls, then expand.\n\n**Business.** In money-and-risk terms: **fail safely** (contained hiccup, not total outage), **scale elastically** (pay for the busy step, scale down after), **stay in control** (a human approves the sensitive calls), and **cap the spend** (no surprise LLM bill) — plus open standards to avoid lock-in. The counterweight: an **in-progress 2.0 beta, effectively from one maintainer, without an independent production track record yet.** The prudent move is a low-stakes pilot with a clear exit, not a mission-critical bet on day one.\n\nMost tasks don’t need any of this; a single-process orchestrator is simpler, and you should use one. But there’s a specific *intersection* of requirements where this combination is the thing that would otherwise cost you a multi-quarter platform build. If your problem sits at **all** of these at once, it’s arguably the only off-the-shelf JS/TS answer:\n\nFor those intersections, the verdict flips from “capable” to “uniquely well-fit.” Outside them, reach for something simpler — and that restraint is part of using this system well.\n\nA grounded map of what’s notable versus the 2025–2026 field, and what each thing is *for*.\n\nEach agent is an instance of a single AgentActor class (src/application/actor/AgentActor.ts): identity is an agentId, its mailbox is one queue/topic per actor (kaiban-agents-{id}), it processes one message at a time, and it touches the outside world only through an IMessagingDriver. Agents never call each other's methods. This is a faithful actor implementation.\n\n*One nuance worth stating:* “supervision” here means Docker Compose restart: unless-stopped / a Kubernetes Deployment — process supervision by the container platform, not an in-runtime actor supervisor tree like Erlang/Akka. Reasonable design; just good to know.\n\nA factory behind an IMessagingDriver interface swaps transport with one env var. **Two drivers are real: BullMQ/Redis (default) and Kafka.** A third, **AMQP, is an explicitly unimplemented stub** — the driver throws *\"AMQP driver is not implemented (roadmap)\"* (ADR-016). So the seam is pluggable and Redis/Kafka work today; AMQP is roadmap.\n\nThere are **five canonical channels**: kaiban-agents-{id} (mailboxes), kaiban-events-completed, kaiban-events-failed (DLQ), kaiban-state-events, and kaiban-hitl-decisions. Task routing uses the configured driver, but **state telemetry and HITL decisions are pinned to Redis Pub/Sub regardless of ****MESSAGING_DRIVER** — a deliberate, smart latency choice. The practical consequence to plan for: **\"Kafka mode\" doesn't remove Redis.** Even a Kafka deployment depends on Redis for state + HITL; there's no fully-Redis-free configuration.\n\nIt’s worth being precise here, since it’s easy to over-describe. The code (src/adapters/state/distributedMiddleware.ts) wraps a store's setState, strips PII-named keys, and **publishes the delta to Redis Pub/Sub** — no validation phase or conflict detection. The system's stated consistency posture is **AP + BASE: at-least-once delivery, ****taskId as the dedup key, every actor idempotent.** (The only \"optimistic concurrency\" reference in the repo is a single version ETag field in the API spec — a data field, not a state-sync engine.)\n\nSo the accurate mental model is **last-writer-wins broadcast**: if two producers publish overlapping state deltas, the later one wins. For a *view-model* feeding a Kanban board, that’s exactly right. Just don’t treat it as a consistency guarantee on shared business state — it isn’t designed to be one.\n\nVerified in AgentActor.executeWithRetries():\n\nThe code (src/shared/orchestrator.ts) is refreshingly plain about itself:\n\nSo the “no central bottleneck” property applies to the **worker tier** (competing consumers scale horizontally) — genuinely valuable — but **the orchestrator is a single point of failure**, mitigated by crash-safe checkpointing rather than HA. This is the most important operational fact about the system, and to the maintainer’s credit the code and ADRs state it clearly.\n\nThis is what credibly supports the interop story: a KaibanJS actor can genuinely share a board with a LangGraph/CrewAI agent reached over A2A.\n\nAmong the most differentiated code, and easy to overlook:\n\nBoth default **off** (opt-in, fail-closed) — a deliberate, security-conscious posture. A gate that blocks rogue behavior *before* billing and a tamper-evident audit trail are exactly what enterprises ask for.\n\nLet **A** = distinct agent roles, **R** = replicas per role, **N** = steps in a workflow, **W** = concurrent workflows, **C** = connected board clients, and **M** = state-update events. The design’s cost profile falls out cleanly from its shape — a competing-consumer worker tier under a single-active orchestrator, with a Redis Pub/Sub telemetry fan-out.\n\n**Time.** An actor’s per-message work is **O(1)** relative to the system: it dequeues exactly one message (Redis BRPOP/BullMQ pop and Kafka partition reads are amortized **O(1)**) and yields to the event loop, so the framework's algorithmic overhead is constant — the real wall-clock term is LLM inference latency, which dominates everything else and is not algorithmic. The orchestrator drives a workflow as a sequential **O(N)** dispatch-and-await chain, but its crash-safe checkpointing carries a subtlety: each completed step re-serializes the *entire* checkpoint and its taskId dedup scans all prior entries, so per-workflow bookkeeping is **O(N²)** in time and bytes written (negligible for the typical N≈3–10, but a real ceiling for very long workflows — where native incremental checkpointing would help). Throughput scales near-linearly with the worker tier: adding R replicas of a bottlenecked role cuts queue wait time roughly **O(L/R)** for offered load L (Little's Law), until a *serial* resource binds. Two such resources cap end-to-end speedup per Amdahl's Law — the **single-active orchestrator** (all W workflows' coordination funnels through one process, **O(W·N)** on one node) and **Redis**, whose Pub/Sub fan-out is **O(C)** per event and **O(M·C)** in aggregate. Practically, system throughput ≈ **min**(orchestrator coordination rate, Redis Pub/Sub throughput, worker service-rate × R, and the LLM provider's rate limits).\n\n**Space.** Each actor holds **O(1)** local state plus an **O(Q)** mailbox backlog; the hard 64 KB/message and 20 KB/state-result caps keep every payload **O(1)**, bounding blast radius. Redis footprint is the sum of in-flight tasks, completed jobs held until TTL, buffered state events, and orchestrator checkpoints — the last being **O(N)** per active workflow, i.e. **O(W·N)** across the fleet — which is exactly why TTLs and allkeys-lru eviction are load-bearing (uncapped completed-job accumulation is the documented OOM failure mode). Kafka trades this for **O(retention-window)** append-only log storage rather than live queue depth. The board client holds **O(A + tasks)** view state.\n\n**Net.** The worker tier is effectively embarrassingly parallel — near-linear horizontal scaling in R — while the orchestrator and Redis form the serial fraction that sets the true ceiling. So the honest scalability story isn’t “unbounded”; it’s *linear in workers, bounded by one coordinator and one Redis*, with LLM latency and provider rate limits — not Big-O — as the usual binding constraints in practice.\n\n**Algorithmic & data-structure foundations.** The runtime is an assembly of well-understood building blocks rather than novel algorithms. Concurrency rests on the **Actor model** (Hewitt, 1973) — share-nothing, message-passing, one-message-at-a-time — with **let-it-crash supervision** borrowed from Erlang/OTP (here realized as container restart). Transport uses classic queue structures: Redis **FIFO lists** (BRPOP/LPUSH, O(1)) and BullMQ's **priority/delayed jobs** over Redis **sorted sets** (skip-lists, O(log n)); or Kafka's **partitioned append-only commit log** with **consumer-group** competing-consumers, offset tracking, and key→partition **hash partitioning** (murmur2 mod P) — whose rebalance/assignment (range, round-robin, cooperative-sticky) and KRaft/**Raft consensus** for leader election are inherited from the broker, not the app. Reliability leans on standard **stability patterns** (Nygard, *Release It!*): bounded **retry with linear backoff**, **timeouts**, **dead-letter queues**, **bulkhead isolation** (per-actor process boundaries), and a **sliding-window circuit breaker**. Flow and cost control use a **token-bucket rate limiter** and cost-reservation; correctness under redelivery uses **content-keyed idempotency** (taskId as a dedup key → O(1) hash-set membership) and **checkpoint/resume memoization** (a saga-style orchestration without compensation). Integrity uses a **hash-chained audit log** (append-only, tamper-evident — the same primitive that underlies Merkle logs and blockchains), and security uses **HMAC/JWT** signing and **TLS/mTLS** handshakes. Notably absent — and worth stating plainly — are application-level **topological sort / DAG scheduling** (workflows are coded as ordered steps) and any app-tier **consensus or leader election** (the orchestrator is single-active by design).\n\n**The theory it rests on.** Performance is governed by **queueing theory**: **Little’s Law** (L = λW) sizes backlog, and **Kingman’s formula** explains why latency rises sharply as any single stage’s utilization → 1 — which, combined with **Amdahl’s Law**, is exactly why the embarrassingly-parallel worker tier scales near-linearly while the single-active orchestrator and always-on Redis (the serial fraction) cap end-to-end speedup; the message bus itself imposes a Shannon-style **channel-capacity** ceiling on throughput. Consistency is a deliberate point on the **CAP/PACELC** design space — **AP + BASE** (eventually-consistent, last-writer-wins) rather than ACID — paired with **at-least-once delivery semantics** and developer-supplied idempotency to approximate exactly-once effects. Architecturally it is textbook **Clean / Hexagonal (Ports & Adapters)** with **Domain-Driven Design** layering and the **Dependency-Inversion Principle**, so transports and LLM providers are swappable **Strategy/Factory** implementations behind interfaces; errors flow through a **Result monad** (railway-oriented programming) in a **functional-core / imperative-shell** split. The system is **event-driven** with a **CQRS-style read model** — task routing (the write path) is separated from the state/telemetry stream that feeds the Kanban board (the read path) — and observability follows **distributed-tracing theory** (Google’s Dapper lineage, via OpenTelemetry). Operationally it adheres to **twelve-factor** config (all state external, config via env), and its resilience story is the classic **microservices** trio of isolation, redundancy, and graceful degradation. In short: no new theory is invented here — the contribution is the disciplined *composition* of these established results into one coherent runtime.\n\nNo system is infallible, but every limit above is *architectural*, not fatal — and each has a well-worn mitigation. Apply these and the runtime behaves like production infrastructure.\n\nRun this playbook and you have a genuinely product-grade deployment: contained failures, bounded recovery, capped spend, enforced policy, full observability.\n\n**Shipped and verifiable:** actor isolation; BullMQ + Kafka drivers; the five-channel topology; Redis Pub/Sub state + Socket.io board; A2A v0.3; MCP client + server; single-active checkpoint/resume orchestrator; 3× linear-retry → DLQ with replay; economics + governance (default-off); OTel; the test/chaos/mutation gates; dual-license supply chain; blog-team and global-research examples.\n\n**Roadmap / in-progress (labeled as such):** the package is version 2.0.0 \"in progress\"; main is described as beta. Not yet built: the **AMQP driver** (stub), **leader election / true HA** for the orchestrator, and various hardening items. Treat 2.0 as a maturing beta, not a battle-tested GA.\n\n**Vision (clearly future-facing):** the large-scale scenarios — a financial-auditing swarm, mega-scale support triage, a planetary multi-hazard early-warning system — are illustrative direction, not deployments or benchmarks. The **actual shipped examples are two**: a blog team and a global-research team, with additional examples presented in [https://github.com/andreibesleaga/kaiban-distributed-examples](https://github.com/andreibesleaga/kaiban-distributed-examples).\n\nKaiban Distributed is a **legitimately interesting, unusually well-engineered piece of work**: a faithful actor-model runtime that brings enterprise-bus messaging, live Kanban visualization, standards-based federation, and real FinOps/governance to the JavaScript agent ecosystem — a combination I couldn’t find elsewhere in JS/TS, and a fair “first-of-kind in its niche.” The engineering hygiene (100% coverage, mutation and chaos testing, signed supply chain, 21 ADRs) is well beyond what its solo-maintainer status would predict.\n\nThe precise picture: the consistency model is AP/BASE last-writer-wins (not a stronger guarantee), retries are linear, the orchestrator is single-active, AMQP is a stub, Redis is always required, and the large-scale scenarios are illustrative. What *is* there — clean actor isolation, swappable Redis/Kafka transport, checkpoint-safe orchestration, A2A+MCP federation, opt-in cost/governance guardrails, and a serious test/supply-chain posture — is more than enough to be worth your attention.\n\nAdopt it for what it verifiably is: an ambitious, disciplined, first-of-its-niche **beta** with a clear-eyed maintainer, a single-orchestrator SPOF to architect around, an unavoidable Redis dependency, and idempotency homework you must do yourself. Judged on its actual code, it earns real respect.\n\n**Everyday user (creator, analyst, builder):** think of it as a small, dependable AI *assembly line* you can run and watch. Today, on your own machine, you can stand up a research-write-edit team and see the work move across a board. More setup than a single chatbot, but you get repeatability, visibility, and the ability to jump in. Overkill for one-off answers; a good fit for a **repeatable workflow you can trust and observe**.\n\n**Business / decision-maker:** the value is **reliability, control, and cost-safety**, not novelty — AI work that fails safely, scales by the piece, keeps a human in the loop, and caps the spend, all with an audit trail, on infrastructure you already run. The honest asterisks: an **in-progress beta from essentially one maintainer**, no HA coordinator yet, and enterprise scenarios that are patterns you’d build. Pilot it; don’t bet the quarter on it unproven.\n\n**Technical user (architect / platform engineer):** a cleanly-layered, well-tested actor runtime with a real messaging seam, A2A/MCP federation, opt-in security/FinOps/governance, and a serious supply chain. Architect around two facts the code is honest about — a **single-active orchestrator SPOF** and a **permanent Redis dependency** — and remember **at-least-once delivery makes handler idempotency your job**. If those fit, the quality here is a pleasant surprise.\n\n**Investor / ecosystem watcher:** a bet on a real gap — distributed, observable, governable multi-agent infrastructure *in the JS/TS ecosystem*, where most tooling is in-process. Credible integration, unusually high discipline for its stage; risks are single-maintainer bus factor, beta maturity, no independent benchmarks, and a crowded market. Promising niche, unproven at scale.\n\n*Assessment method: every capability and limitation above was verified directly against the project’s source (**src/), ADRs (** docs/decisions/), API spec, CHANGELOG, and test configuration with the help of AI and AI coding assistants. Where a description and the code differed, the code was treated as ground truth.*\n\nSources: [kaiban-distributed](https://github.com/andreibesleaga/kaiban-distributed) · [kaiban-distributed-examples](https://github.com/andreibesleaga/kaiban-distributed-examples) · [KaibanJS](https://github.com/kaiban-ai/KaibanJS) · A2A / @a2a-js/sdk · Model Context Protocol / @modelcontextprotocol/sdk. Author’s own code and docs and other AI online research.\n\n[Kaiban Distributed, Evaluation](https://pub.towardsai.net/kaiban-distributed-evaluation-06a1420753f5) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/kaiban-distributed-evaluation", "canonical_source": "https://pub.towardsai.net/kaiban-distributed-evaluation-06a1420753f5?source=rss----98111c9905da---4", "published_at": "2026-07-12 13:36:44+00:00", "updated_at": "2026-07-12 13:39:34.449680+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-infrastructure", "ai-startups", "developer-tools"], "entities": ["Kaiban Distributed", "KaibanJS", "Docker", "Kafka", "Redis", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/kaiban-distributed-evaluation", "markdown": "https://wpnews.pro/news/kaiban-distributed-evaluation.md", "text": "https://wpnews.pro/news/kaiban-distributed-evaluation.txt", "jsonld": "https://wpnews.pro/news/kaiban-distributed-evaluation.jsonld"}}