{"slug": "ergo-framework-erlang-otp-style-distributed-systems-in-go", "title": "Ergo Framework – Erlang/OTP-style distributed systems in Go", "summary": "Ergo Framework 3.3.0, an Erlang/OTP-style distributed systems toolkit for Go, adds distributed tracing, a testing framework, and an Observer that serves as a full cluster inspector with an MCP endpoint for AI agents. The Saturn module is now MIT licensed, making the entire ecosystem free for production use. The release includes breaking changes requiring applications to embed app.Application.", "body_md": "Ergo Framework 3.3.0 is about seeing what a running cluster is doing, and proving what it will do before it runs. A trace now follows a message across nodes, the Observer became a full cluster inspector that an AI agent can drive, and a testing framework ships with the framework itself. Saturn is MIT from this release on, so every module of the ecosystem is free for production use.\n\n## Highlights\n\n**Distributed tracing**. A trace context propagates across `Send`, `Call`, spawn and every hop a message takes between nodes, with configurable sampling and cross-node clock-skew correction. The new `Pulse` application exports the collected spans over OTLP to Tempo, Jaeger or anything else that speaks it, and ships a Grafana dashboard.\n\n**Testing framework**. ergo.services/ergo/testing gives four layers on one fluent assertion grammar: unit runs a single actor against a mock node, stage runs real nodes over the real protocol, mock provides standalone gen.* mocks for code that consumes the interfaces, and check is the assertion core they share. You assert over messages, calls, spawns, links, monitors and events, positively and negatively.\n\n**Observer, and MCP for AI agents**. The Observer application is now a full cluster inspector: node info with live memory graphs, the process list with mailbox depth, latency, running time and wakeups, per-process detail with supervision tree and live HandleInspect state, meta processes, applications, the network stack and its wire-type registry, events, a live log stream, and on-demand goroutine and heap profiles with no special build. The same application serves an MCP endpoint: 38 tools and 13 resource lenses over the live cluster, 26 of them read-only, with capability ceilings that can only narrow. **One Observer inspects every node in the cluster**, because every node already runs the built-in system application.\n\n**`act.Router`**. A behavior that picks the destination from the message instead of taking the next worker in order. It can own its routes or resolve them through the node registry, the routing table is editable while it runs, and high-priority traffic bypasses routing so an admin path stays open while the routed path is busy.\n\n**EDF schema evolution**. With `NetworkFlags.EnableSchemaEvolution` on both nodes, fields appended to the end of a registered struct stay wire-compatible: a peer that does not know them skips them, a peer missing them zero-fills. Off by default, so EDF stays strict unless you ask for it.\n\n**Grid**. A new application giving a distributed registry, key lifecycle monitors and per-key process groups. Every node keeps a full local copy, so a lookup is a local read; writes are serialized per key by a shard actor and replicate in the background. AP and eventually consistent.\n\n**argus**. A vet tool for the actor-model invariants the compiler cannot express: shared memory in a message, an unbounded wait in a callback, a goroutine reaching back into actor state, a HandleCall that can never reply, and thirty-one more. Runs as go vet -vettool=$(which argus) ./..., with a baseline mode for codebases that have never seen it.\n\n## Breaking changes\n\nApplications must embed `app.Application`. `ApplicationBehavior` gained lifecycle callbacks and `Load` lost its node `gen.Node` parameter:\n\n```\n// before\ntype MyApp struct{}\nfunc (a *MyApp) Load(node gen.Node, args ...any) (gen.ApplicationSpec, error) { ... }\nfunc (a *MyApp) Start(mode gen.ApplicationMode) {}\n\n// after\ntype MyApp struct{ app.Application }\nfunc (a *MyApp) Load(args ...any) (gen.ApplicationSpec, error) { ... }\n```\n\nThe embed supplies the framework plumbing and no-op defaults for the new callbacks. If you used `node` inside `Load`, switch to `a.Node()`. If your `Start` was an empty stub, delete it; if it had logic, its signature is now `Start(ref gen.Ref, mode gen.ApplicationMode)`. Step-by-step in the Application documentation.\n\n## Also\n\n- `gen.ProcessBehavior` requires`ProcessKind() gen.ProcessKind` . The base behaviors (`act.Actor` ,`act.Supervisor` ,`act.Pool` ,`act.Router` ,`act.WebWorker` ) report their own, so only hand-written behaviors are affected.\n- The `trace` build tag is now`verbose` : go run --tags verbose ./cmd.\n\n## Also in this release\n\n- **Network** . Automatic**fragmentation** of large messages, application-level**keepalive** that catches the silent failures TCP keepalive cannot, a handshake deadline, and a fix for the dead loop two nodes could enter when dialing each other at the same time.\n- **Observability** without pauses.`Node.Info()` and the heap inspector no longer stop the world; memory and GC figures come from runtime/metrics. New`NodeShortInfo` /`RemoteNodeShortInfo` /`ProcessShortInfo` are the cheap counterparts for polling a whole cluster, alongside per-event metrics, process lifecycle counters, per-process running time, wakeups, init time, and mailbox latency under`-tags=latency` .\n- **Application** teardown. An application's`Terminate` now runs after every one of its processes has terminated, its own`Terminate` included, so closing a shared resource there is safe. Detached processes are stopped with the application instead of outliving it.\n- **Errors** that keep their causes.**`gen.Errorf`** collects %w arguments so` errors.Is` and`errors.As` still find them, and the chain**survives the wire** when both nodes enable NetworkFlags.EnableWrappedErrors.\n- **Radar** ,**Health** ,**Sentry** . One application serving Kubernetes probes and a Prometheus endpoint on one port; the Health actor behind it; and a logger that forwards framework panics and errors to Sentry with stack traces from the panic site.\n- **Metrics** actor expanded well beyond node/network telemetry, with a complete Grafana cluster dashboard.\n- Target Manager rewritten lock-free. One lock used to serialize every link, monitor and event subscription on the node; a node **now carries hundreds of thousands** of them without contention\n- **Saturn** moved from Business Source License 1.1 to MIT. No node cap, no licence to buy.\n\n## Documentation\n\nNew articles on [docs.ergo.services](https://docs.ergo.services):\n\n- [FAQ](https://docs.ergo.services/faq) - the questions that come up before the first node starts\n- [AI Agents](https://docs.ergo.services/ai-agents) - the skills and agents shipped for working on an Ergo codebase\n- [Router](https://docs.ergo.services/actors/router) -`act.Router` , its routing modes and runtime management\n- [Distributed Tracing](https://docs.ergo.services/advanced/distributed-tracing) - samplers, business spans and exporters\n- [Inspecting Actor State](https://docs.ergo.services/advanced/inspecting-state) -`HandleInspect` and reading a live process\n- [Inspecting With Observer](https://docs.ergo.services/advanced/observer) - the cluster inspector, panel by panel\n- [Inspecting With an AI Agent](https://docs.ergo.services/advanced/mcp) - the MCP surface, its lenses and capability ceilings\n- [Actor Model Vet Tool](https://docs.ergo.services/tools/argus) -`argus` , its rules and how to adopt it on existing code\n\nTesting:\n\n- [Overview](https://docs.ergo.services/testing/overview) - the four layers and which one answers which question\n- [Check](https://docs.ergo.services/testing/check) - the assertion grammar shared by every layer\n- [Mock](https://docs.ergo.services/testing/mock) - standalone`gen.*` mocks for code that consumes the interfaces\n\nExtra library:\n\n- [Grid](https://docs.ergo.services/extra-library/applications/grid) - distributed registry, key monitors and process groups\n- [Pulse](https://docs.ergo.services/extra-library/applications/pulse) - OTLP exporter for the collected spans\n- [Radar](https://docs.ergo.services/extra-library/applications/radar) - health probes and Prometheus metrics on one port\n- [Health actor](https://docs.ergo.services/extra-library/actors/health) - Kubernetes liveness, readiness and startup probes\n- [Sentry logger](https://docs.ergo.services/extra-library/loggers/sentry) - forwards panics and errors to a Sentry project\n\nFull list of changes: CHANGELOG.md", "url": "https://wpnews.pro/news/ergo-framework-erlang-otp-style-distributed-systems-in-go", "canonical_source": "https://github.com/ergo-services/ergo/releases/tag/v1.999.330", "published_at": "2026-09-08 13:32:15+00:00", "updated_at": "2026-09-08 13:56:47.365063+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents"], "entities": ["Ergo Framework", "Saturn", "Pulse", "Observer", "MCP", "act.Router", "Grid", "argus"], "alternates": {"html": "https://wpnews.pro/news/ergo-framework-erlang-otp-style-distributed-systems-in-go", "markdown": "https://wpnews.pro/news/ergo-framework-erlang-otp-style-distributed-systems-in-go.md", "text": "https://wpnews.pro/news/ergo-framework-erlang-otp-style-distributed-systems-in-go.txt", "jsonld": "https://wpnews.pro/news/ergo-framework-erlang-otp-style-distributed-systems-in-go.jsonld"}}