Bus-Reactive Ambient Intelligent Nodes
I'm a Flutter mobile and AI engineer, and honestly I've never been satisfied with the agent frameworks out there. Most of them boil down to a loop or a cron poking a model on a timer, with everything funnelled through a single chat. So I tried to build the thing I actually wanted to use. Full transparency: I leaned on AI heavily to write this code (that's part of the experiment too), and I think what came out is worth sharing.
Under the hood it's really a NATS pub/sub bus with long-lived daemon nodes, closer in spirit to ROS than to a chat framework. The LLM is just one thing a node can reach for, not the center of gravity.
A few things I think it gets right:
It doesn't burn tokens waiting. Nodes arereactive, not scheduled. They stay parked and only call the LLM when a message they actually care about arrives. No cron, no idle polling, so you pay for thinking instead of ticking.Every node can have its own interface, local or remote. A node isn't just a handler; it can ship a UI, and that UI stays reusable and reachable even when the node runs on another machine.You watch your agents, not a chat box. Instead of one conversation doing everything, you get a live graph of small, dedicated nodes, each with one job and its own view.You can spread the load. The same node runs in-process or on a remote machine joined to the shared bus. Distribute work across hardware without touching the code.
It's not a polished product and it's not trying to replace anything. It's my honest attempt at a problem I don't think is solved yet. If any of this resonates, take a look.
brain-explainer-web.mp4 #
A 2-minute tour of what makes brAIn different β reactive many-to-many nodes, criticality with preemption, the live dashboard, any-model-per-node, distributed + MCP, and what you can build with it. βΆοΈ Also on
[YouTube].
voice.chat.with.intent.2026-07-08.at.16.02.16.mp4 #
Sound on. A video is replayed as a virtual camera and microphone, and
the whole perception stack treats it as live input. voice
transcribes and diarizes the two speakers, gaze
tracks who is
looking where, and intent
pairs the two: people talking to each
other stays context, but the moment a speaker addresses the camera
the brain
wakes up, answers, and the Kokoro tts
node speaks the reply out loud. No wake word, and overheard chatter never costs an LLM call.
The dashboard is the network: every node, the messages flowing between them, and the wiring you edit by dragging. No single chat, just each agent doing its own job.
Two machines share one bus and one canvas. Nodes spawned on a peer show up next to the local ones, so you can spread the load across hardware. The wiring doesn't care where a node actually lives.
A node is more than a function: it can serve its own interface. Here a small "brainpet" node with a dedicated UI, reusable and reached the same way whether it runs locally or on a remote peer.
A tic-tac-toe node played through the Telegram bridge. The same node logic is exposed over an external channel, so interfaces stay reusable across the bus, even off your machine.
phone.loneliness.mp4 #
brAIn-mobile turns a phone into a node: its sensors (light, motion, battery) stream onto the bus like any other publisher, and the brain reacts to what physically happens to the device. Here it has strong opinions about where it just got put down. The reply loops back to the phone's own screen and speaker, so perception and reaction both live at the edge.
brAIn isn't a competitor to most of the agentic tools you may know; they solve adjacent problems and the right one depends on the shape of the agent you want.
LangGraph / Vercel AI SDK / Mastra: for chat-shaped agents (user types β LLM thinks β tools β LLM responds), these are excellent and deeper than brAIn's chat support. Reach for them when the agent is fundamentally a conversational interface.
AutoGen / CrewAI: multi-agent conversations with roles. Use these when you want several LLM personas debating or collaborating within a single dialogue.
ROS 2: the closest architectural cousin. Pub/sub bus, daemon nodes, multi-language, cross-machine over DDS. brAIn shares a lot of its mental model with ROS. The differences are domain-specific: brAIn is built around LLM constraints (token budgets, abortable inference, tool-call loops, MCP), runs over NATS rather than DDS (easier to deploy when you don't need real-time guarantees), and preempts at the work-in-progress level rather than between callbacks.
Inngest / Trigger.dev / Temporal: durable workflows. If your agent is a finite-shape DAG with retries and backoff, those are production-grade options. brAIn's nodes are open-ended daemons; the two run on different mental models.
n8n / Flowise / Langflow / Dify / Node-RED: visual
node-based authoring. brAIn nodes are written in code (a
TypeScript handler plus a config.json
, or any HTTP/WS service
via transport: "web"
). The dashboard is observation-first.
Claude Cowork / OpenAI scheduled tasks / cron-driven agents: time-triggered prompts. brAIn is event-triggered; if your agent's cadence is "every Monday morning summarise X", a scheduler is the right primitive.
A condensed comparison of the architectural traits brAIn happens to have, for context:
| brAIn | LangGraph | AutoGen | ROS 2 | |
|---|---|---|---|---|
| Long-lived daemon nodes | yes | graph runs per call | per conversation | yes |
| Many-to-many bus | yes | inputs flow through the graph | conversation channel | yes |
| Mid-handler abort on priority | yes (LLM/CLI/MCP signal-aware) | no | no | priorities at the queue level |
| LLM-native primitives | yes | yes | yes | no |
| Cross-machine | NATS | no | no | DDS |
| MCP client | yes (4 transports) | per-tool wrappers | per-tool wrappers | no |
| Causal trace + replay | yes (/network/traces/:id/replay ) |
|||
| LangSmith captures traces | no | no |
A runtime for autonomous agents that live in a many-to-many event world.
Nodes are long-lived daemons. Each one subscribes to several input streams (chat messages, sensor events, webhooks, internal bus traffic, anything you can publish), reacts when something relevant shows up, and can publish to as many outputs in parallel. There's no single triggering channel: the agent watches and decides when to act.
A node may also be preempted mid-flight: when a higher-criticality
message lands during a slow operation (an LLM call, a tool
invocation, a CLI agent), the runner aborts what's in progress and
re-runs the handler with the new context surfaced in
ctx.preemptionContext
. Same node config can run in-process, behind
a WebSocket, or on a remote brain-agent
joined to a shared NATS bus.
A few shapes you can build with this:
- An ambient room agent that watches camera + mic and only speaks when someone is looking at it while talking (the flagship demo: voice + gaze + intent + brain). - A Slack-channel listener that lives in a thread, picks up context across messages, and summarises or replies when the conversation s. - A monitoring agent subscribed to Grafana alerts + oncall-rotation events + recent deploys, that surfaces a hypothesis when a correlation crosses a threshold. - An IoT controller that fuses temperature, motion, calendar, and time-of-day to decide when to change the environment.
The framework's primitives:
Daemon model: nodes live across iterations; the framework auto-parks them when idle and the bus wakes them on the next subscribed message. State persists across runs.Many-to-many I/O: each node subscribes to N topic patterns (with wildcards) and publishes to as many.** Criticality with preemption**: every message carries a criticality. A higher-criticality message arriving mid-handler aborts the running iteration and triggers a re-run withctx.preemptionContext.{interrupting_message, previous_messages}
available.Distributed runtime: same node config runs in-process or on a remotebrain-agent
. Lifecycle (stop / start) and read-back (logs / mailbox / DLQ) work transparently across machines over NATS.MCP-native:mcp-config
(manager) +mcp-server
(one per upstream) bridge any MCP server's tools onto the bus, so the agent reaches into filesystem, git, Slack, Linear, Notion, Sentry β¦ as it would call any other node.
packages/core/src/bus
: NatsBusService
implements IBusService
on top of a NATS broker. The framework boots an embedded
nats-server
(the bundled Go binary, downloaded by the postinstall
hook) on a free localhost port; remote brain-agent
processes
connect to that same broker and share the bus. Set
BRAIN_NATS_URL
to skip the embedded broker and join an external one instead, typical when running across multiple hosts.
BusService
(in-memory) is still exported but only as a test fixture; the production code path always goes through NATS.
Known limitation (scaling):each instance subscribes to the whole hierarchy (<prefix>.>
) and filters locally withmatchTopic
to keep brAIn's wildcard semantics. Every instance therefore sees all bus traffic and filters it in memory, fine at small/LAN scale, but the first bottleneck as instance/message counts grow. The targeted fix is narrower NATS subscriptions per real subject.
Features:
- Wildcard topic matching (
alerts.*
). - Per-subscription
mailbox with configurablemax_size
and alatest
/lowest_priority
retention policy.dropped
andcapacity
are exposed so the dashboard can show per-mailbox backpressure. Causal traces: every message carriestrace_id
+parent_id
. Survives NATS encode/decode, queryable viaGET /network/traces/:id
. Each trace can bereplayed as fresh emissions throughPOST /network/traces/:id/replay
: fresh ids, rewritten parent chain, originaltrace_id
carried asmetadata.replayed_from
for debugging.History sliding window (10k messages by default).
packages/core/src/runner
, picked from a node's tags (not its
name): a node carrying the llm
tag gets the LLMRunner
, anything
else gets the ServiceRunner
(a web
transport overrides both). The node names in parentheses below are just examples of nodes that carry each tag.
for reactive non-LLM nodes (memory, http-bridge, terminal, β¦): message arrives β handler called once β node parks until the next subscribed message arrives.ServiceRunner
for LLM nodes (brain, memory-proxy, memory-consolidator): handler called in a budget loop (default 5 iterations). New messagesLLMRunner
reset the budget(fresh attention). When exhausted β node parks until something rewakes it.
When a higher-criticality message lands while a handler is running, the runner aborts the iteration instead of waiting it out.
ctx.signal
is an AbortSignal
exposed to every handler. LLM
handlers pass it to generateText({ abortSignal: ctx.signal })
, CLI
nodes to spawn(..., { signal })
, MCP nodes to
client.callTool(..., { signal })
. The abort propagates through to the underlying HTTP request, so a long inference at the LLM provider or a long subprocess invocation gets cut at the source, not just queue-reordered between iterations.
The threshold (how much higher the incoming criticality must be) is
configurable; default is 3. The next handler invocation runs with
ctx.wasPreempted = true
and the preemption details in
ctx.preemptionContext
.
packages/agent
ships a brain-agent
CLI binary that joins the
shared NATS bus and hosts nodes on a remote machine. It announces
itself every 10 s, subscribes to its control topics
(brain.agents.<self>.{spawn,kill,stop,start}
), and answers
read-back requests for logs / mailboxes / dead_letters
via NATS request-reply. The dashboard's Agents tab lists every live agent and the Node Creator's Target dropdown lets you pick "Local" or any agent for a new spawn. When an agent stops announcing past ~30 s, its remote-node stubs are automatically pruned from the API.
Lives in brAIn-essentials.
mcp-config
owns a single Claude-Desktop-shaped JSON
({mcpServers: {...}}
) and reconciles it by spawning / killing
mcp-server
children, one per upstream. Each mcp-server
connects
via the official @modelcontextprotocol/sdk
and exposes each tool
as its own bus topic mcp.<alias>.<tool>
, so callers wire to
capabilities directly. Status, OAuth state and tool catalog are on
mcp.<alias>.{status,tools,oauth.required}
. Four transports
supported per upstream: stdio
, http
(Streamable HTTP), sse
,
ws
. ctx.signal
propagates, so preemption kills MCP calls in flight.
packages/core/src/skills
. A shared library of SKILL.md
files that
teach the LLM nodes how to do things, separate from the message-passing
wiring. The idea is borrowed from the Hermes agent's skill library,
and it follows the open Agent Skills format
(name
description
frontmatter, progressive disclosure: the catalog first, the full body on demand).
Serving is a framework concern, not a node one. The framework owns the
store and answers over NATS request/reply
(skills.rpc.{search,load,save,delete,list}
); a node only consumes,
through the thin ctx.skills
facade that rides the bus. So any LLM node,
including a remote brain-agent
on another machine, asks "how do we do
X?" and gets the file back without keeping a local copy. One library,
every LLM node, cross-machine. Libs don't own skills; they just ship
SKILL.md
files that the framework merges into the single store.
Three tiers:user(personal, indata/skills
, always available),lib-capability(aSKILL.md
shipped by an installed lib, e.g.web-fetch
), andnode-scoped(requires_node:
frontmatter, only surfaced once an instance of that type is live, so no dead skills clutter the catalog).Retrieval: semantic ranking via Ollama embeddings (qwen3-embedding:0.6b
, cosine, content-versioned cache) with a keyword fallback. The brain auto-injects the single most relevant skill's body and lists the rest, then canload_skill({name})
for others. Auto-inject, rather than relying on the model to call a tool, is what makes skills land reliably even on a small model likegemma4:e4b
.Writable: LLM nodes (and the dashboard's Skills panel) cansave
/delete
personal skills; bundled ones are read-only.
DLQ: every message in flight when a handler crashes / times out is captured in a per-runner ring (50 entries). Surfaced under the NodePanel's DLQ tab; the tab badge flips red on first entry.Backpressure: each mailbox tracksdropped
(cumulative evictions) andcapacity
. The Mailbox tab shows a fill bar coloured by load.Tracing: any message in the history can be opened as a tree in the dashboard, with one-click replay.
SQLite (data/brain.db
) via better-sqlite3
. Spawned nodes, their subscriptions, and mailbox config survive restarts; on boot each node re-subscribes and waits, idle, for its next message (the runtime is purely reactive, with no separate persisted dormancy state).
Three minimum files under nodes/<your-node>/
. Wiring is declared
with ports: each node states its input/output ports (the immutable
contract) plus default_port_bindings
mapping every port to the bus topics it listens on or emits to. Every input port carries a JSON Schema, so it shows up as a typed, callable tool. There is no auto-derivation: a node without explicit ports is rejected at registration.
// config.json
{
"name": "my-node",
"description": "What this node does",
"tags": ["utility"],
"default_authority": 0,
"default_priority": 1,
"ports": {
"inputs": {
"command": {
"description": "What this node accepts (becomes an MCP tool).",
"inputSchema": { "type": "object", "properties": { "x": { "type": "string" } } }
}
},
"outputs": {
"result": { "description": "What this node emits." }
}
},
"default_port_bindings": {
"inputs": { "command": ["some.topic"] },
"outputs": { "result": ["some.output"] }
},
"supports_transport": ["process"],
"has_ui": false
}
python
// src/handler.ts
import type { NodeHandler, NodeOnSpawn, NodeTeardown } from "@brain/sdk";
export const onSpawn: NodeOnSpawn = async (info) => {
// boot external resources (child processes, sockets, β¦)
};
export const handler: NodeHandler = async (ctx) => {
for (const msg of ctx.messages) {
ctx.publish("some.output", {
type: "text",
criticality: 1,
payload: { content: `Got: ${JSON.stringify(msg.payload)}` },
});
}
// Pass ctx.signal to anything long-running so the runner can
// preempt this iteration when an urgent message lands.
};
export const teardown: NodeTeardown = async () => {
// release whatever onSpawn acquired
};
supports_transport
may include "process"
(in-tree TS handler),
"web"
(external HTTP/WS service, also requires a web: { url }
block), and/or "remote"
(any node hosted on a brain-agent).
The type is auto-discovered on engine boot, or registered live via
the dynamic scanner if dropped under nodes/_dynamic/
.
This repo ships zero nodes: nodes/
only contains a _dynamic/
slot for the runtime scanner. Every capability comes from a sister repo, installed via the in-app Marketplace tab (backed by brAIn-store). A library can ship its own ready-made workflows (seed YAMLs) that show up once it's installed.
:brAIn-essentialsbrain
(LLM orchestrator with a tolerant tool-call parser),developer
(writes new node types at runtime via Claude / Codex / Gemini CLIs),attention
,clock
,cron
,echo
,mcp-config
mcp-server
.
:brAIn-memorymemory
(KV + tags),memory-vector
(LanceDB + Ollama embeddings),memory-proxy
(LLM-mediated gateway: the brain talks here, never to the underlying stores),memory-consolidator
,reminder
.:brAIn-toolsterminal
,http-bridge
,calc-py
(Python node behind a WebSocket, demonstratestransport: "web"
).:brAIn-llmllm-basic
(Vercel AI SDK wrapper),llm-cli
(Claude Code / Codex / Gemini wrapper).:brAIn-uichat
(browser interface for human β network).:brAIn-perceptionvoice
(faster-whisper + WeSpeaker),gaze
(InsightFace + Gazelle + Moondream),intent
(voice Γ gaze correlator),tts
(OS voices or the Kokoro-82M neural voice β the network answers out loud).:brAIn-bridgestelegram
,discord
,whatsapp
(reach the network from a chat app, off your machine).:brAIn-gamesbrainpet
,hangman
,tictactoe
(playable nodes, each with its own UI).:brAIn-demo-lonelinessphone-loneliness
(a small demo scenario).
voice
publishes voice.transcript
, gaze
publishes
gaze.target.resolved
, intent
matches them on a sliding window
and emits intent.detected
for every correlated utterance (the
chat displays them as reported speech). When the speaker is looking
at the camera β addressing the AI β intent also emits
intent.addressed
, carrying the question plus everything overheard
since the last exchange, and THAT is what wakes brain
: humans
talking among themselves never cost an LLM call. Combined with
chat
and the Kokoro tts
node, the room agent hears the
conversation, answers without a wake word, and speaks its reply out
loud. The voice and gaze servers auto-install their virtualenv and
download the ML weights on first spawn β and both accept a video
file in place of the live camera/mic ({"file": β¦}
on
/api/capture/start
) to replay a scene deterministically. This whole pipeline is the first video in See it in action above.
npm create brain
npm create brain my-instance
This bootstraps the dev workspace via the create-brain package: clones
brAIn/
and brAIn-store/
, creates an empty
storeprojects/
directory, runs pnpm install
(downloads the bundled
nats-server
binary, builds the framework), and launches the stack. End-to-end, one command. Layout produced:
brain/ (default folder)
βββ brAIn/ framework
βββ brAIn-store/ marketplace registry
βββ storeprojects/ empty, filled at runtime by `pnpm brain pull`
Once it boots (first boot takes ~1 min while the auto-seed clones a few sister repos), open:
API β http://localhost:3000
Dashboard β http://localhost:5173
To stop without auto-launch (just clone + install) pass --no-start
.
To re-launch later: cd brain/brAIn && ./run
(run.cmd
on Windows).
Node.jsβ₯ 20 (pnpm is auto-bootstrapped viacorepack
if missing)git inPATH
Ollama only if you install LLM nodes (ollama pull gemma4:e4b
,ollama pull qwen3-embedding:0.6b
)Python 3.11 only if you install the perception nodes (voice / gaze) frombrAIn-perception
nats-server
ships embedded; pnpm install
fetches the right binary
for your platform. Set BRAIN_NATS_URL
to skip the embedded broker and join an external one instead.
If you're going to hack on the framework itself:
git clone https://github.com/tibzejoker/brAIn && cd brAIn
pnpm install # postinstall: builds sdk/core/agent, clones
pnpm start
pnpm brain list # marketplace registry: installed + available
pnpm brain pull memory # install a node from the marketplace
pnpm brain remove memory --yes # uninstall the node's parent sister repo
Or open the dashboard's Marketplace tab. Workflows (seed YAMLs) ride along with their library: install a lib and its seeds appear in the Seeds view, ready to apply as a pre-wired starter network. You can also snapshot your running network as a personal seed.
Custom node: dropnodes/_dynamic/<your-node>/{config.json, dist/handler.js}
; the dynamic scanner registers it on the fly, no restart.Custom seed: snapshot the running network from the dashboard's** Seeds**view (or drop a YAML the framework can read). Apply via the dashboard orPOST /network/seeds/<name>/apply
.
On the API host, open the Distributed tab and click Open to LAN
once. That binds the embedded broker on 0.0.0.0
and pins an auth token. The panel shows a one-liner snippet, copy it.
On the target machine:
npm create brain
cd brain/brAIn
pnpm brain pull memory # (or whichever nodes the agent should host)
BRAIN_NATS_URL=nats://<api-lan-ip>:<port> BRAIN_NATS_TOKEN=<token> npx brain-agent
The agent shows up in the Distributed tab. From the Node Creator, pick it as Target to spawn nodes on it.
Pin the broker port across restarts with BRAIN_BROKER_PORT=4222
.
Run an external broker with BRAIN_NATS_URL=nats://<host>:<port>
: the API skips the embedded one and joins yours.
pnpm kill-orphans # smart cleanup by cmdline + ports
pnpm kill-ports # blunter
GET /nodes List
GET /nodes/:id Detail
POST /nodes Spawn { type, name, transport?,
target_agent_id?, β¦ }
DELETE /nodes/:id Kill
POST /nodes/:id/{stop,start,tick}
PATCH /nodes/:id/config Update config_overrides
PATCH /nodes/:id/position Persist dashboard layout
GET /nodes/:id/{logs,mailboxes,dead-letters}
POST /nodes/:id/ports/:side/:port/topics Bind a port to a topic
DELETE /nodes/:id/ports/:side/:port/topics/:topic Unbind
GET /types
POST /types/register { path }
DELETE /types/:name
GET /network Full snapshot
GET /network/messages History
GET /network/history Lifecycle audit log
GET /network/transport { nats, url? }
GET /network/{providers,devmode}
POST /network/{devmode,tick,reset}
GET /network/traces/:trace_id Walk a causal chain
POST /network/traces/:trace_id/replay Re-publish as fresh emissions
GET /network/seeds List on-disk YAML seeds
POST /network/seeds/:name/apply Apply a seed (?merge=true to add)
GET /store/{index,nodes,candidates,upstream-status,installed-updates}
POST /store/install { package_name }
POST /store/uninstall { package_name }
POST /store/refresh Pull brAIn-store
POST /store/rescan Register types installed by the CLI
GET /agents
GET /nodes/:id/ui/ Static node UI
POST /nodes/:id/ui/send Publish into the node
GET /nodes/:id/ui/messages Conversation log
GET /mcp/oauth/callback
WebSocket events on /socket.io
: node:spawned
, node:killed
,
node:state_changed
, message:published
.
SDK: TypeScript, types-only package consumed by every node.** Core**: TypeScript engine: pino, eventemitter3, better-sqlite3, ws, nats.js, ai (Vercel SDK),@modelcontextprotocol/sdk
.API: NestJS 10 + Socket.IO + express.** Agent**:brain-agent
CLI binary (packages/agent
) for remote-host node execution over the shared bus.Dashboard: React 19, React Flow, d3-force, Tailwind v4, Vite.** Bus**: NATS by default (an embeddednats-server
boots on a free port, or join an external broker withBRAIN_NATS_URL
); the in-memoryBusService
exists only as a test fixture.Persistence: SQLite via better-sqlite3.** Monorepo**: pnpm workspaces, with cross-repo sibling resolution to the checked-out companion repos under../storeprojects/brAIn-*/nodes/*
. Missing paths are silently ignored.Marketplace:../brAIn-store
is auto-cloned by the postinstall hook; the dashboard's Marketplace tab installs nodes from it (SHA-pinned, per-file checksums), and each library brings its own seeds.Cross-language nodes:packages/python-sdk
(brain-web
) helper for nodes that speak the bus from Python over WebSocket (transport: "web"
).Tests: vitest, with optional integration suites that gate on the relevant dependency being available (NATS, Ollama, MCP servers).
pnpm test # all
RUN_LLM_E2E=1 npx vitest run tests/preemption-llm-e2e.test.ts
RUN_MCP_E2E=1 npx vitest run tests/mcp-host-public-server-e2e.test.ts