When I started building omadia — an open-source (MIT), self-hostable runtime for composing AI agents out of plugins — I assumed the hard part would be the model: prompting, tool-calling, getting reliable output.
It wasn't. The LLM bits were mostly solved problems. The parts that ate my time were isolation, data flow, and UX. Here are four lessons I wish I'd internalised earlier.
The project is open source (MIT) and self-hostable — code and docs here:
[https://github.com/byte5ai/omadia]
With a single agent you never notice. With several agents sharing a memory store, context bleeds: agent A starts "remembering" facts only agent B was ever told. It's subtle, it's intermittent, and it erodes trust fast — especially once different agents serve different users or departments.
What worked: give each agent (and each orchestrator) its own memory namespace and its own slice of the knowledge graph, and route every inbound message to the correct agent's namespace at the point where the channel binding resolves — not later, deep in the agent logic.
// simplified: scope every read/write to the agent that owns the turn
const mem = memoryFor(agent.id); // isolated namespace + KG slice
await mem.write(turn);
const context = await mem.recall(query);
The tradeoff: deliberate cross-agent sharing becomes the awkward case you have to design explicitly. That's the right default — isolation by construction, sharing by intent.
Everyone talks about prompt injection. The leak that actually bit me was on the way out: tool results.
A tool that returns a "summary + details" blob can quietly carry fields — including PII — into a context that has no business seeing them. The model then happily surfaces them. No injection required; the data just rode along in a tool response.
The fix was a privacy guard that expands and scopes tool output per record before it ever reaches the model, so each field is gated rather than passed through wholesale. The mental shift: treat tool outputs as an untrusted boundary, the same way you treat user input.
I built a full-featured agent builder — every knob exposed. The people it was meant for, the non-engineers, found it intimidating and left.
Shipping a stripped-down default — describe the agent in plain language, wire up a couple of tools, preview it side-by-side — moved adoption more than any single feature did. The power-user builder is still there for those who want it, but it's no longer the front door.
Lesson: for a builder product, the default surface is a product decision, not a settings decision. Optimise it for the least technical user you actually want.
omadia splits the world into:
Getting those boundaries right is what makes the system composable instead of a monolith. Getting them wrong early was my single biggest source of rework. Spend the time on the seams before you have ten plugins fighting them.
If you want to see how the seams are drawn in practice, the code is here: https://github.com/byte5ai/omadia
omadia is pre-1.0 / public preview. It works and we run it in production, but DB schemas can still break between minor versions and the upgrade path is hand-rolled today. It's single-tenant and self-hostable: run it on your own infrastructure — in containers (Docker) or directly on the Node stack, bring your own LLM API key, and keep all the data on your side. Stack is TypeScript/Node + a Next.js builder UI.
I'd genuinely like your take on the two tensions at the heart of this:
If you're building self-hosted agents, I'd love your feedback in GitHub Discussions: https://github.com/byte5ai/omadia/discussions/216 — and star the repo if you want to follow the public preview.
Repo and docs: https://github.com/byte5ai/omadia