Building a Lightweight AI Agent in Go: Baize's Architecture and Trade-offs

A developer built Baize, a lightweight AI agent runtime written in Go that runs as a sidecar process alongside existing services, converting OpenAPI documentation into callable tools and gating important writes behind human-in-the-loop approval. The runtime uses a three-layer architecture of core loop, tool router, and executors, and instead of in-process plugins it POSTs tool invocations to the user's own endpoint with idempotency keys and signed callbacks, trading an extra network round-trip for language-agnostic, process-isolated execution.

Baize is a "sidecar" AI assistant runtime: a single process that sits beside services you already run, turns API documentation into tools the assistant can call, and pauses important writes until a person approves them. Stop it and it leaves almost nothing behind. One sentence positioning: it's a runtime, not a framework. That positioning drives every architecture decision below. The core requirement of sidecar deployment is "one process, copy it over, run it." A Go binary needs no interpreter, no dependency installation, no virtualenv on the target machine. For an assistant meant to live in enterprise environments, that's the lowest-cost delivery form. One assistant process serves multiple entry points at once: the web console, signed alert/ticket ingress, and IM channels. Goroutines make "one process handling many sessions concurrently" straightforward — and parallel tool calls later become almost free. Enterprise environments run everything: Windows, Linux, macOS, ARM. A single GOOS=linux GOARCH=arm64 go build produces a binary for the target platform, with no toolchain setup on the destination machine. Tool input schemas come from OpenAPI documents; mapping them onto Go's strong types catches many errors at compile time. For a long-running daemon, that saves a lot of operational pain. A clean three-layer structure: core loop → tool router → executors . Users / Channels ──► Agent Core Loop think → pick tool → run → report │ ▼ Tool Router Registry │ ┌────────────────┼────────────────┐ OpenAPI connector HTTP plugin MCP connector └────────────────┼────────────────┘ ▼ Invoker closures registered in Registry │ HITL approval gate │ ┌─────────────┴─────────────┐ Direct execution HTTP callback executor plugin / proxy callback to your side Core loop internal/run : LLM thinks → picks a tool → executes → reports. The event stream llm.thinking , llm.tool call , tool.result is persisted, so the console can follow along in real time. Tool router internal/tool : a mutex-protected map that registers not functions but "tool contracts + invoker closures". Executors internal/connector : tools enter through three sources — OpenAPI docs, HTTP plugins, MCP tool servers — plus a "callback execution" mode that hands execution back to your side. This is Baize's most important trade-off. The problem with plugin protocols : in-process plugins Go plugins, shared libraries, language-binding SDKs require the plugin to compile with the host process — language, version, and ABI must all align. Most enterprise systems aren't written in Go: legacy systems, Java/.NET/Python services can't load an in-process plugin at all. And even when they could, upgrading a plugin means restarting the process — "sidecar in, clean out" is gone. What Baize does instead : it doesn't execute the tool itself; it POSTs the invocation to your own endpoint: { "tool": "create ticket", "arguments": { ... }, "run id": "run xxx", "agent id": "agent xxx", "idempotency key": "uuid-xxx", "callback urls": { "event": "https://your-service/baize-events" } } Your side executes it and returns the result. Benefits: language-agnostic, process-isolated, auditable; idempotency key makes retries safe no duplicate execution ; callback urls lets your side keep driving follow-up actions. The cost : one extra network round-trip, the callback endpoint must be reachable, and to prevent forged callbacks you need signed requests Baize uses callback signing with a TTL against replay . The registry tool.Registry is the core data structure: a sync.RWMutex guarding a map, with runtime register/unregister and per-connector bulk unregister — adding a tool or disabling a connector never requires a restart. Three tool sources share one registration path: OpenAPI docs : import Swagger/OpenAPI/Postman-style docs, each operation becomes a tool; HTTP plugins : a small companion service declares "what tools exist and how to invoke them"; MCP tool servers : connect to external tool ecosystems as an MCP client. Security policy is baked into each entry at registration time: require approval needs a human , require login needs a session , security schemes which auth scheme to use . Security policy is decided at registration, not asked at execution time — this is the precondition for letting the assistant actually act. Discovery is trivial: Registry.List / Registry.Specs feed the model's tool list, visible live in the console. Failures are the norm in AI agents, so degradation design matters more than the happy path: Timeout guardrail : every tool invocation is bound to context.WithTimeout default 60s, configurable ; Failure is content : Invoker returns content, isError, err — err is an infrastructure failure timeout, network , isError is a business-side failure. Both flow back to the model as structured content, so the model can retry, switch tools, or explain to the user — instead of crashing the whole session; Approval rejection is not a crash : when a human rejects a write, the run settles into an explicit "rejected" terminal state with a trail, no panic; Everything is observable : the llm.tool call → tool.result event stream is persisted, so any problem can be traced step by step; Context compaction : long sessions get rolling summaries so quality doesn't degrade as the thread grows. Let's be honest first: Python is the best choice in the AI/Agent ecosystem . LangChain, LlamaIndex and most reference implementations live there. If your goal is fast experimentation and deep reuse of the LLM ecosystem, Python has no rival. Baize chose Go because its positioning is different: | Dimension | Go | Python | Node.js | |---|---|---|---| | Deployment | Single binary, zero deps | Interpreter + deps/venv | Node runtime + node modules | | Resource footprint | Low; one resident process is cheap | Higher; resident processes need care | Medium | | Concurrency | Native goroutines | GIL-limited; multi-process/async | Event loop | | Type safety | Static, compile-time checks | Dynamic, found at runtime | Dynamic / TypeScript | | LLM ecosystem | Newer, catching up fast | Richest | Rich | | Cross-platform | Cross-compile to all platforms | Needs interpreter on target | Needs Node on target | The conclusion isn't "Go is better than Python" — it's positioning decides the language : Goal: framework / fast experimentation → Python; Goal: sidecar, resident, one-command deployment to enterprise environments, running on modest hardware for a long time → Go's advantages in deployment and resource usage are hard to replace. All snippets are from the project source, lightly trimmed. Each comes with one line on what problem it solves. Modeling a "tool" as "a contract the model sees Spec + an invoker closure injected by the connector" fully decouples routing from execution: type Invoker func ctx context.Context, args map string any content map string any, isError bool, err error type Meta struct { Spec llm.ToolSpec ConnectorID string Method string Path string RequireLogin bool SecuritySchemes string } require approval / require login are written into the entry at registration; the tool list is "hot" — adding/removing connectors never requires a restart: func r Registry RegisterMeta meta Meta, inv Invoker, requireApproval bool { r.mu.Lock defer r.mu.Unlock r.tools meta.Spec.Name = entry{ spec: meta.Spec, invoker: inv, requireApproval: requireApproval, requireLogin: meta.RequireLogin, connectorID: meta.ConnectorID, method: meta.Method, path: meta.Path, } } Hands execution back to your side; the idempotency key makes network retries safe: payload := map string any{ "tool": tool, "arguments": args, "run id": meta.RunID, "agent id": meta.AgentID, "idempotency key": meta.IdempotencyKey, } if strings.TrimSpace meta.CallbackEventURL = "" { payload "callback urls" = map string any{ "event": meta.CallbackEventURL, } } rawPayload, := json.Marshal payload req, := http.NewRequestWithContext ctx, http.MethodPost, c.URL, bytes.NewReader rawPayload Non-GET/HEAD/OPTIONS operations are automatically flagged "needs approval" at registration, and only run after a human clicks approve/reject in the console: needApproval := t.RequireApproval if ctx.requireApprovalMutating && isMutatingMethod t.Method && t.Source == store.ToolSourceSpec { needApproval = true } A timeout guardrail plus "failure is content" semantics keeps one bad tool call from blowing up the whole session: toolCtx, toolCancel := context.WithTimeout ctx, e.toolTimeout defer toolCancel content, isError, invErr := e.Tools.Invoke toolCtx, payload.ToolName, payload.Arguments if invErr = nil { // Infrastructure failure timeout/network : persist and close the round return e.finalizeFailedRun runID, invErr } // When isError is true, the failure flows back to the model as content; // the model decides whether to retry or explain. Baize is still early. The trade-offs above are far from "optimal" — especially the approval UX, channel adapters, and executor extensibility. If you have real-world scenarios, I'd love to hear them. Repo: https://github.com/rebornace/baize https://github.com/rebornace/baize MIT Issues: open a discussion with your scenario — I'll follow up.