The runtime boundary behind RocketRide's crash isolation, task lifecycle, and Cloud operations.
By Krish Garg and Mithilesh Gaurihar
At 9 a.m., with ten thousand users mid-session, a node in an AI pipeline dereferences a bad pointer. The process running it is gone before Python can raise a useful exception. That is an unpleasant failure, but it is not the question we care about most.
The question is what happens next. Does that crash take unrelated pipelines with it? Does the server need a restart? Does the on-call engineer walk into a system-wide incident, or into one failed task and a useful record of why it failed?
In RocketRide, a failed task is meant to be contained and recorded. The server sees the child process exit, updates the task's state and exit code, releases the task's ports and connections, and sends status updates to subscribed monitors. The run stops. Its history does not vanish. Other task processes are not sharing its memory, interpreter, or worker threads.
That behavior comes from a decision we made early: every pipeline run gets its own isolated process.
It is not the cheapest or fastest possible architecture. Starting a process has a cost, and keeping one around has a cost too. We accepted those costs because the alternative makes failures much harder to reason about once Python code, native libraries, model runtimes, and user-defined nodes are all running in the same service.
An AI pipeline does not fail like a typical request handler. A normal exception is one thing. A segfault in a C extension, a crash in a media decoder, or a broken native inference library is another. Once a process has corrupted memory, application-level error handling is no longer a reliable line of defense.
So each RocketRide task starts as a fresh child process with its own embedded Python interpreter. It loads one pipeline, initializes that pipeline's nodes, and owns the work for that run. The parent runtime keeps the task registry, allocates the task's local ports, monitors the child, and handles its lifecycle.
The transport details matter here. The parent supervises the child through standard input and output streams, while pipeline data is handled through a local WebSocket endpoint assigned to that task. It is not a simple request pipe with a response pipe on the other end. The important property is the boundary: a task has a supervisor, a known process, and an exit status.
When the child exits unexpectedly, the runtime marks it as stopped, records the exit code, cleans up the data and debug connections, and makes that state visible to clients subscribed through rrext_monitor
. The system has a record for a live incident and a post-mortem later. We are not trying to hide a failure inside "continuous operation." We are trying to keep one failure from becoming every customer's failure.
The same boundary also works in reverse. Child processes are launched with an auto-terminate setting, so they exit when their parent goes away rather than becoming unsupervised work that lingers after a server shutdown.
This is fault isolation, not a security sandbox. Separate processes limit the blast radius of a broken workload. They do not, by themselves, turn untrusted code into a safe multi-tenant security boundary.
The obvious downside of process isolation is startup. A new task may have to start Python, import packages, read the .pipe
file, construct the graph, connect to services, and prepare any state the nodes need. Paying that cost for every message in a chat or every page in a document job would be wasteful.
Today, a pipeline is either running or it is not. There is no pre-warmed middle state yet: tasks that are known to the runtime and spin up the moment a request arrives are planned, but we would rather describe what ships. What ships is a set of controls that decide when a running pipeline stays up and when it goes away.
In production, the application owns the pipeline's lifetime: it starts the pipeline it needs and stops it when it is done. Development is where the runtime steps in, with three mechanisms:
useExisting
.useExisting
and leave the TTL alone. If the pipeline is already running, the app uses it; if not, the runtime starts it. On the next run, a stopped pipeline starts again automatically, and one still inside its idle window is simply reused.While a task stays up, it keeps its state: the same process, the same initialized pipeline. The first request pays the initialization cost, and the requests that follow use the process that is already ready. It is still not a shared worker pool. A running task remains one pipeline's process, with one pipeline's state.
We do not think process-per-run is the right answer for every system. An in-process runtime starts faster, uses less memory, and avoids crossing process boundaries. For a local experiment, a trusted script, or a short-lived internal tool, that can be the right choice.
Our concern was the point at which a runtime becomes shared infrastructure. At that point, the platform needs to know which process owns a pipeline, which process owns a port, how a task is terminated, what happened when it died, and how to avoid orphaned work. A team can assemble those pieces with worker frameworks and process managers. We wanted them to be the default behavior of the runtime rather than an operations project every pipeline author has to recreate.
The C++ layer is not there to make CPU-bound Python magically faster. Python remains where developers use the model libraries, data ecosystem, and notebook-friendly tools that make modern AI practical. The runtime's job is to give that Python code a dependable floor: native orchestration, task lifecycles, explicit cleanup, and a process boundary when something goes wrong.
The open-source runtime gives a team this execution model on its own infrastructure. Running it as a service adds a different job. Running tasks are stateful; access needs an identity; and task usage needs to belong to an organization, team, and user. Once those concerns stretch beyond one developer's laptop, they are not incidental deployment details. They are the operating system around the runtime.
RocketRide Cloud takes on that work while keeping the pipeline format portable. Cloud uses API-key access, records task-level usage against the relevant organization, team, and user, and uses health checks around the services that supervise and route tasks. Session-level routing is part of the Cloud architecture because a running task belongs to a particular runtime instance. That is operational work, not a reason to turn a deployment diagram into a promise of unlimited scale.
Cost tracking is where that attribution earns its keep, and it covers both halves of a pipeline's life. Development and deployment are tracked as separate costs, so a team can see what building a pipeline cost before it ever served production traffic, and what running it costs after.
During development, a project team with five members can see exactly what each person's runs cost, then roll those numbers up into the total for the project. At deployment, a pipeline is deployed into a production team, and that team's costs are visible the same way, down to the individual user. Every task's usage lands against the organization, the team, and the user that spent it, so the question of what a pipeline costs has an answer at every level someone actually asks it: per person, per team, per organization.
That is also why self-hosting remains a real option, not a grudging footnote. Teams with a VPC requirement, air-gapped deployment, strict data residency, or an existing platform group may want to own every layer themselves. The same .pipe
files and runtime are available for that path. Cloud is for teams that want the execution model without taking on the day-to-day responsibility of operating it.
The decision rule is straightforward. Self-host when owning the infrastructure is part of the job. Use Cloud when the pipeline is the work and operating the pipeline platform is not. Either way, the artifact you build remains yours.
Build and run a pipeline on RocketRide Cloud, self-host with RocketRide Server, or start in the RocketRide VS Code extension. And if you have questions about any of this, come find us on the RocketRide Discord.