{"slug": "how-grok-bot-works-vm-sand-host-gateway-agents-subagents-tools-persistence-and", "title": "How Grok Bot works: VM, Sand host, gateway, agents, subagents, tools, persistence, and failover", "summary": "A developer has reconstructed the deployed architecture of Grok Bot, a stateful agent service that runs inside a Linux container, from recovered JavaScript bundles and runtime scripts. The reconstruction, covering 1,804 modules at commit 8740c03c8c6f14f28d1af653c2f0869da021b359, documents a host process (host-main.cjs) that owns conversations and checkpoints each turn, plus a separate exec-daemon providing controlled access to shells, files, terminals, MCP servers, browsers, and virtual desktops. The system is described as a durable coordinator connected to replaceable services, with inference handled by a Cursor-backed service.", "body_md": "Grok Bot is a stateful agent service that runs inside a Linux \"box.\" The model is only one part of it. A host process accepts commands, owns conversations, assembles prompts, calls an inference backend, delegates tool work, checkpoints each turn, and publishes events. A separate execution daemon gives the host controlled access to shells, files, terminals, MCP servers, browsers, and virtual desktops.\n\nThis document explains the deployed system recovered from the Grok Bot image and the local VM built to preserve and run that image. It keeps those two systems separate. The recovered source describes the original runtime. The clone-lab code describes the recovery VM, its routing, and its known parity gaps.\n\nThe source reconstruction comes from the deployed `host-main.cjs`, five auxiliary\nhost bundles, the separate `exec-daemon` bundle, exact unbundled runtime scripts,\nprotocol reflection, and native binaries. The original TypeScript source maps were\nnot present. Recovered JavaScript bodies are strong evidence of behavior, but they\ndo not recover erased TypeScript annotations or every original import spelling.\n\nThis document uses four labels:\n\n- **Exact runtime source** is a byte-identical script captured from the image.\n- **Recovered source** is a body extracted from a deployed JavaScript bundle.\n- **Derived** means a relationship can be reproduced mechanically from recovered\nfiles or protocols.\n- **Inference** means several sources support an architectural conclusion that no\nsingle source states as a contract.\n\nThe canonical recovered tree contains 1,804 modules. The clean developer-facing\nprojection divides them into `sand-host`, `exec-daemon`, `dune`, and 36\n`@anysphere/*` workspace packages. These counts come from the reconstruction at\ncommit `8740c03c8c6f14f28d1af653c2f0869da021b359`.\n\n```\nGrok client or control plane\n        |\n        | authenticated HTTP command\n        | long-lived SSE event stream\n        v\n+----------------------- Linux box ------------------------+\n|                                                           |\n|  host-main.cjs                                            |\n|  +---------------- SandHost ---------------------------+  |\n|  | gateway -> transcript manager -> per-agent runner   |  |\n|  |                  |                 |                 |  |\n|  |                  |                 +-> inference RPC |  |\n|  |                  |                 +-> Task children |  |\n|  |                  |                 +-> approvals     |  |\n|  |                  |                 +-> tools         |  |\n|  |                  v                                   |  |\n|  |       store.db + blob DB + transcript projections   |  |\n|  +------------------------------------------------------+  |\n|                         | authenticated ConnectRPC/WS      |\n|                         v                                  |\n|  exec-daemon                                               |\n|  +-> shell and file operations                             |\n|  +-> PTY and tmux sessions                                 |\n|  +-> MCP subprocesses                                     |\n|  +-> X11 computer use and screenshots                     |\n|                                                           |\n|  sand-window-router -> per-display exec daemons            |\n|  sand-supervisor -> host, Xvfb, Chrome, VNC, noVNC         |\n+-----------------------------------------------------------+\n                         |\n                         | short-lived access token\n                         v\n              Cursor-backed inference service\n```\n\nThe useful mental model is a durable coordinator connected to replaceable\nservices. `host-main.cjs` decides what a turn means. The inference backend\ngenerates model steps. `exec-daemon` performs operating-system work. The gateway\ntranslates client commands and streams state changes. No one component is the\nwhole bot.\n\nThe captured deployment ran as a Docker-style container. `/tini` was PID 1 and\nstarted `pod-daemon`. `pod-daemon` created processes on demand and parented the\nlong-lived Sand runtime. The observed process tree was:\n\n```\n/tini\n└── /pod-daemon                         gRPC process creation, TCP 26500\n    └── sand-exit-watch                 top-level child ownership\n        ├── supervise-exec-daemon\n        │   └── exec-daemon             ConnectRPC 1337, PTY WS 1338\n        ├── supervise-sand-supervisor\n        │   └── sand-supervisor.mjs\n        │       ├── host-main.cjs       HTTP/SSE gateway 1340\n        │       ├── x11vnc children\n        │       └── fork noVNC router   TCP 6081\n        ├── sand-window-router.mjs      TCP 1339\n        ├── sand-session-sync.mjs\n        ├── browser authentication and cookie helpers\n        ├── desktop :1                  primary agent desktop\n        └── desktops :2, :3, :4         forked task desktops\n```\n\n`pod-daemon` is below the hosting control plane and above the application. It is\nuseful for creating a process inside the box, but it is not the conversational\nAPI. The client experience enters through `host-main.cjs` on port 1340.\n\nThe VM boundary matters. Same-user processes inside the box can often reach loopback listeners, inspect process metadata, and share X11 state. Display numbers separate desktops and browser profiles. They are not separate operating-system users or security principals.\n\nThe recovered entry point `recovered-monorepo-final/src/host/main.ts`\nacquires a host lock, starts `SandHost`, resolves gateway settings, starts the\nserver, writes discovery data, and installs shutdown handlers. The composition\nroot in `recovered-monorepo-final/src/host/sand-host.ts` loads\nextensions, restores transcript state, creates runner services, wires events, and\nexposes the gateway API.\n\nThe exact `sand-supervisor.mjs` script owns a wider dependency graph. It starts or\nre-adopts the host and desktop processes, writes health snapshots, detects crash\nloops, and probes the VNC path. `supervise-exec-daemon` separately restarts the\nprimary execution daemon and checks its listener. The host adds an authenticated\nping because an open TCP port does not prove that the daemon event loop can answer.\n\nShutdown and upgrade are coordinated states. The host stops new work, expires pending approvals, checkpoints active turns at safe boundaries, records agents that need continuation, and lets the replacement process consume those resume markers.\n\n`recovered-monorepo-final/src/host/gateway-server.ts` provides\nHTTP authentication, body-size limits, JSON command routing, health, SSE, avatars,\nlocal-exec queues, WebAuthn queues, and upgrade preparation. The dispatch table in\n`recovered-monorepo-final/src/host/gateway-protocol.ts` maps\ncommand names to methods in\n`recovered-monorepo-final/src/host/host-gateway-api.ts`.\n\nThe ordinary command path is:\n\n``` php\nPOST /api/<command>\n  -> check the browser Origin and Host policy\n  -> verify the bearer token in constant time when auth is configured\n  -> enforce the request-body limit and parse JSON\n  -> resolve <command> in the dispatch table\n  -> call the host API adapter\n  -> mutate or read the owning subsystem\n  -> return JSON\n```\n\nThe gateway covers messages, transcripts, named agents, rooms, channels, automations, workflows, memories, settings, permissions, approvals, secrets, plugins, skills, box state, and updates. It does not execute model tokens or shell commands. It routes requests to the services that own those operations.\n\n`sendPrompt` returns after the transcript manager durably accepts the prompt. It\nnormally returns `{ \"accepted\": true }` before inference finishes. Replies arrive\nas later transcript events over `GET /events`.\n\nThis split is why the gateway has `clientNonce` and\n`promptAcceptanceStatus`. A client that times out after sending a prompt cannot\nknow whether the server committed it. The correct retry sequence is:\n\n```\n1. Open and authenticate GET /events.\n2. Generate one clientNonce for the logical prompt.\n3. POST /api/sendPrompt with the agent ID, prompt, and clientNonce.\n4. If the response is ambiguous, call promptAcceptanceStatus.\n5. Retry only the identical payload with the identical nonce when needed.\n6. Follow transcript events until the turn settles or waits for input.\n```\n\nChanging the payload while reusing a nonce is a conflict. Generating a new nonce for an ambiguous retry risks running the mutation twice.\n\n`createAgent` has its own bounded in-memory nonce ledger. It prevents concurrent\nduplicate creation requests from minting more than one agent while the operation\nis active.\n\n`GET /events` is a long-lived Server-Sent Events stream with heartbeat, optional\ngzip, and optional channel filtering. `SandHost.wireEvents()` fans out changes to\ntranscripts, agents, outlines, subagents, asynchronous tasks, memory, automations,\nworkflows, trays, and box state.\n\nThe client therefore does not poll one \"run\" object for everything. It sends commands and observes a stream of domain events. This is how prompt acceptance can return fast while thinking, tool calls, child agents, approvals, and final messages continue in the background.\n\nThe main turn path spans the gateway, transcript manager, runner, inference transport, tools, and persistence:\n\n``` php\nprompt accepted and enqueued\n  -> allocate request ID, trace, and cancellation generation\n  -> snapshot privacy mode and agent profile\n  -> build the ConversationAction\n  -> create main and summarization inference sessions\n  -> discover MCP tools\n  -> restore or skip a transcript checkpoint\n  -> ensure that the agent has a box\n  -> compose the tool-bearing agent\n  -> stream model steps\n       -> text and reasoning parts\n       -> tool calls\n       -> approval pauses\n       -> Task child dispatches\n  -> persist step checkpoints\n  -> settle as success, awaiting-user, aborted, quiesced, or error\n  -> persist final state and emit events\n```\n\n`recovered-monorepo-final/src/host/runner/prompt-collector-glue.ts`\ncombines the visible message with reply context, attachments, recent messages,\nprofile changes, automation markers, trusted hidden context, and unanswered\nquestions. `recovered-monorepo-final/src/host/runner/turn-run-shell.ts`\nthen creates the model sessions and drives the turn.\n\nMCP discovery degrades to no MCP tools if discovery fails. That failure does not erase the conversation. First-token deadlines and retry policy belong to the runner, while transport-specific request and stream conversion belong to the inference layer.\n\nEach run has an ownership generation. A cancelled or superseded run cannot write over a newer generation. The transcript manager also checks whether a replacement run can deliver a superseding user message before it cancels the old run. This guard prevents a fast cancellation from stranding the new message.\n\nThe runner records step checkpoints separately from the final turn state. A widget request, secret request, approval request, or forced upgrade can pause at a committed boundary. After restart or user input, the host resumes from that boundary instead of blindly replaying the whole turn.\n\nThis is an exactly-once approximation, not a universal transaction across the model, filesystem, network, and third-party APIs. Side-effecting tools still need idempotency. The checkpoint prevents many accidental repeats by recording which step completed before the host continues.\n\n`recovered-monorepo-final/src/host/extensions/auth/credential-renewer.ts`\nexchanges a renewal credential for a short-lived access token. It refreshes before\nexpiry, coalesces concurrent refresh requests, backs off after errors, and redacts\ndiagnostics. `recovered-monorepo-final/src/host/extensions/auth/auth-service.ts`\nkeeps the access token in memory and rejects a token that is too close to expiry.\n\n`recovered-monorepo-final/src/host/extensions/inference/inference-service.ts`\ncombines authentication, settings, experiments, default-model selection, and\nspecial models for browser or computer work. `cursor-session.ts` creates the\nCursor-backed prompt session. The protobuf client serializes messages, tools,\nparameters, lineage, and provenance, then converts streamed response parts into\ntyped text, reasoning, tool, metadata, usage, and error events.\n\nThe runner creates a separate summarization session. General Task children inherit the normal host model selection. Media-review children may force a specialized model.\n\nInference credentials and execution credentials have different jobs. A shell in the box can perform an approved command without receiving the host's renewal credential or short-lived model token. This separation limits how much authority a tool subprocess needs.\n\nThe user-visible agent list contains durable named agents. The gateway can create, open, update, duplicate, interrupt, and delete them. A named agent owns persistent conversation state, profile, schedules, memory, source identity, box assignment, and permission state. Host startup reloads these agents.\n\nA Task subagent is a logical child session created for one delegated job. It is not a new VM and usually is not a separate operating-system process.\n\n``` php\nparent model calls Task\n  -> review the proposed delegation against the parent conversation\n  -> create a child SandAgentRunner\n  -> assign child transcript and request identities\n  -> record parent, type, tool call, title, and lineage\n  -> arm the parent's pending wake and a stall watchdog\n  -> run the child in the background\n  -> publish child status and outline events\n  -> on completion, capture the result and free any desktop window\n  -> revive the parent with a completion event\n  -> parent model reads the result and continues\n```\n\n`recovered-monorepo-final/src/host/runner/turn-agent-composition.ts`\ncreates the child runner with shared host services for inference, box access, MCP,\npolicy, and events. The child gets distinct transcript and request identity plus\nthe parent's provenance. The recovered composition disables nested Task\nconfiguration inside a Task child, so the ordinary child cannot recursively fan\nout another level.\n\n`recovered-monorepo-final/src/host/runner/subagent-runtime.ts`\nkeeps separate maps for live sessions, run promises, metadata, registry status,\noutlines, watchdogs, pending steering messages, and abort state. Steering records a\nmessage, interrupts the current child turn, and starts a new turn on the same child\ncontext after settlement. Stopping a child clears the pending wake, interrupts the\nturn, and suppresses later parent revival.\n\nThe host can offer several child types:\n\n- A general-purpose child handles research or coding in the background.\n- A `browserUse` child drives Chrome at the page and element level.\n- A `computerUse` child operates the whole X11 desktop through screenshots and\ninput events.\n- Media-review children use specialized prompts and may select a different model.\n\nComputer-use windows are scarce resources. The host allocates a display to the child and releases it when the child settles. Browser use and computer use are separate tools because DOM-level automation is faster and less disruptive, while pixel-level control works with native applications and hostile web interfaces.\n\nThe recovered host also exposes cloud-agent launch and watch tools. A cloud coding agent is not a durable named Sand agent and is not an in-process Task child. It can run on a separate remote worker or VM with its own checkout, generated branch, transcript, status, and artifacts. The parent watches or imports its result through the cloud-agent service.\n\nThe three categories have different ownership:\n\n| Category | Runner location | Machine | Durable state | How completion appears | \n|---|---|---|---|---|\n| Named Sand agent | `SandHost` | Its assigned box | Per-agent directory, SQLite, and blobs | Direct transcript and SSE updates | \n| Task child | Another `SandAgentRunner` in the same host | Reuses the parent's box, with an optional fork display | Child conversation state plus live runtime maps | Completion wake revives the parent | \n| Cloud coding agent | Remote cloud-agent service | Separate worker or VM | Remote transcript, branch, and artifacts | Parent watches and imports the result | \n\nThe generic Task package contains richer resume and cloud-placement concepts than the recovered Sand background adapter visibly persists. The code proves steering a live Sand child and delivering its completion. It does not, by itself, prove that a completed child's private runner can resume after a host restart. That behavior needs a live-runtime test before publication as a durable guarantee.\n\nThe host composes separate review providers for host shell, box shell, MCP, computer use, automation writes, cloud-agent operations, and Task launch. A review can allow an action, block it, or pause the turn for the user.\n\n``` php\nmodel proposes an action\n  -> surface-specific policy and auto-review\n  -> allow: execute the exact action\n  -> deny: return the refusal\n  -> needs user: persist approval request and pause\n  -> gateway receives resolution\n  -> resume from the checkpoint or expire the request\n```\n\nThe runtime permits only one pending approval at a time for an audited turn. Task launch is reviewed in the real parent context before the child sees the delegated prompt. The child must still pass review for each concrete shell, MCP, browser, or desktop action it proposes.\n\nExecution-side permissions merge administrator, user, repository, and hardcoded policy. The daemon checks parsed commands, redirects, delete protection, sandbox availability, workspace access, extra paths, and network policy. Approval is best understood as permission for one exact proposed action, not a permanent grant for a tool category.\n\n`exec-daemon` starts two authenticated servers. The HTTP ConnectRPC server exposes\nexecution and control services. The WebSocket server exposes PTY operations such as\nspawn, attach, input, resize, list, and terminate. Optional tmux services keep\nterminal sessions alive across individual requests.\n\nThe control service covers health, capabilities, direct file operations, diffs, artifacts, environment refresh, plugin operations, and MCP loading. The execution service carries typed streaming envelopes to registered resource executors. Shell, files, MCP, and computer use share transport, but each has its own executor and policy.\n\nShell sessions can persist executable Bash, zsh, or PowerShell state. Before each command, the daemon unsets old conversation and agent-store variables and injects the current request's server-derived values. This ordering prevents an older shell snapshot from impersonating a later conversation.\n\nThe sandbox policy describes workspace read or write access, read boundaries, extra read-only and read-write paths, protected mappings, temporary-directory rules, and network rules. Hardcoded policy protects Git hooks, agent settings, SSH material, and other executable configuration. If sandbox enforcement is required but unavailable, a safe replica must fail closed rather than run unrestricted.\n\nMCP servers run as workload processes behind the execution daemon. The host\ndiscovers their tools for each turn and presents those tools to the model. MCP has\nits own sandbox policy and approval path. A long-lived `railway mcp` or\nXcodeBuildMCP process in the observed box was workload state, not a boot daemon.\n\nEach desktop is an Xvfb display with a window manager, compositor, dock, VNC\nserver, and a Chrome profile. For display `N >= 2`, the exact runtime maps ports as\nfollows:\n\n```\nexec ConnectRPC: 14000 + N\nPTY WebSocket:    13600 + N\nVNC:               5900 + N\nChrome CDP:         9222 + N\n```\n\n`recovered-monorepo-final/src/host/box/box-windows.ts` allocates\nthe display and mints an owner token. Requests carry `x-sand-display`. Fork routes\nalso carry `x-sand-window-owner`, which must match the token bound to that display.\n`sand-window-router.mjs` selects either the primary daemon or the daemon for that\ndisplay.\n\nLocal computer use sends X11 input and captures screenshots. Browser use talks to\nthe browser at a page level. `sand-session-sync.mjs` connects to live Chrome\ninstances over loopback CDP and mirrors cookies, including HttpOnly cookies, plus\nlocalStorage. It does not mirror IndexedDB. A per-display busy lease delays page\nreloads while computer use is acting.\n\nSession sync makes login state convenient across an agent's windows, but it expands the authentication trust domain. The display token protects routing. It does not turn each display into a separate user account or filesystem boundary.\n\nEach durable agent has a SQLite `store.db` with key-value state, legacy blobs, and\ntranscript entries. A separate strict SQLite blob database stores\ncontent-addressed conversation payloads. A root in the agent state points to the\nreachable payload graph.\n\n``` php\nstore.db\n  +-> current root and transcript metadata\n          |\n          +-> conversation-blobs.db\n                    |\n                    +-> transcript JSONL journal and mirror\n                    +-> search-index.db\n\nprofile, memory, routines, workflows, and store snapshots\n  +-> box-store manifest and immutable uploaded objects\n```\n\nThe JSONL transcript and search index are projections. The host can rebuild them from canonical agent and blob state. Search corruption must not decide whether a conversation exists. Database recovery quarantines damaged files, salvages rows that can be read independently, verifies replacements, and only then installs them.\n\nThe blob store uses Node worker threads. `AgentWorkerPool` creates one worker per\nblob-database path, up to its capacity, and evicts idle workers. The worker owns\nblob reads, writes, checkpoint-root lookup, stale-root cleanup, garbage collection,\nlegacy retirement, flush, and close. These are storage workers. They are unrelated\nto model Task children, which are `SandAgentRunner` objects in the host process.\n\nLive SQLite files are captured through database-safe immutable snapshots. Box-store hydration uses manifests and an explicit completeness state. Deletion propagation starts only after a trusted baseline exists, which prevents a partial download from being mistaken for an authoritative empty state.\n\nProfiles use atomic `profile.json` updates. Memory is split into agent, shared-user,\nand project scopes with single-writer shards. Automations store definitions and a\nbounded run history. Workflows merge user, managed, and plugin-owned definitions\nwithout discarding ownership.\n\nAgent identity, source identity, conversation identity, box identity, Task child identity, and automation-run identity are distinct. Treating any two as the same ID would break routing or persistence isolation.\n\nSchedules, listeners, and backend events feed a trigger hub. The hub creates a durable wake for a named agent and sends it through the ordinary turn queue. The turn gets automation provenance and a trusted or untrusted origin marker, then uses the same runner, tools, approvals, checkpoints, transcript, and inference path as an interactive prompt.\n\nAt startup, the host reconciles definitions, rearms pending wakes, and retries acknowledgements that did not complete. During upgrade, it suspends new scheduler wakes while backend events remain queued for the replacement process. Deleting a named agent also deletes its schedules.\n\nThere is no separate \"automation agent runtime.\" Automation is another producer of durable turns.\n\nThe captured `/dev/vda` was a 128 GiB ext4 container filesystem. It had no partition\ntable, bootloader, or kernel, so it could not boot as a VM. The clone uses a Debian\n12 x86-64 wrapper disk to supply the missing operating-system pieces.\n\n```\nApple Silicon Mac\n└── QEMU x86-64 with TCG translation\n    ├── writable Debian 12 wrapper disk\n    ├── captured ext4 disk attached read-only\n    └── reconstructed Docker overlay\n        ├── 66 captured lower layers, read-only\n        ├── /var/lib/grok-clone/upper, writable\n        └── /mnt/grok-container, merged chroot\n```\n\n`grok-clone-mount.service` reconstructs the overlay and mounts `/proc`, `/dev`, and\n`/sys` into the merged tree. `grok-pod-daemon.service` runs the captured `/tini` and\n`pod-daemon`. `grok-supervisor.service` reads the protected captured environment,\nenters the chroot, drops to UID and GID 1000, and executes the original Node runtime\nand `sand-supervisor.mjs`.\n\nThe original evidence disk remains read-only. Local transcripts and runtime changes\nland in `/var/lib/grok-clone/upper`. This design preserves the capture while giving\nthe restored application a writable filesystem.\n\nThe supervisor environment contains credential-bearing state and is stored as a\nroot-owned mode `0600` NUL-separated file. The launcher reads it before `chroot`,\nsets the box identity, drops privileges, and passes it directly to the process. The\nscripts do not print the values.\n\nAt the 2026-08-18 runtime audit, the clone had the restored named agents, gateway, transcripts, external inference, pod-daemon, window router, and several desktop components. Single-turn and multi-turn conversations worked. The audit did not prove full platform parity. The main exec daemon, per-window exec daemons, PTY WebSockets, session sync, cookie persistence, and one desktop were missing or not validated at that capture.\n\nCurrent clone-lab scripts have continued to change after that audit. For example, the current window-router launcher deliberately points its primary route at local port 1337, while the noVNC bridge maps restored display 4 to both historical and local tokens. Those scripts express the intended current topology. They do not, by themselves, prove that every daemon is running in a live VM now.\n\nQEMU uses software CPU translation because the guest is x86-64 and the Mac is Arm. Short turns work, but CPU-heavy tool runs can starve the emulated gateway. Running the disks on an x86-64 KVM host would preserve the binaries and remove most of that translation cost.\n\nThe clone lab keeps the original gateway address stable while allowing a local VM\nto serve requests. A streaming proxy on the original box selects either the\noriginal `host-main.cjs` or an SSH path to the local clone.\n\n``` php\nclient\n  -> original stable address :1340\n  -> firewall redirect :21341\n  -> streaming gateway proxy\n       +-> original host-main :1340\n       `-> reverse SSH :21340\n            -> Mac forward :31340\n            -> local VM host-main :1340\n```\n\nAn authenticated SSE connection pins the selected backend. Concurrent commands use the same backend while that event stream is open. This affinity prevents a client from sending a prompt to the local transcript while listening for its result on the original host.\n\nFailover is active/passive. It never mirrors a mutating request and never replays a request that may have been partially sent. Both gateway processes may be alive, but only one receives a given client session. This rule matters because both copies use the same logical agent IDs. Mirroring would create duplicate tool actions and divergent transcripts.\n\nGateway failover covers the HTTP and SSE service on port 1340. It does not automatically cover pod-daemon, exec ConnectRPC, PTY WebSockets, window routing, noVNC, raw VNC, Chrome CDP, Docker, or unknown provider services. Gateway parity is not platform parity.\n\nThe system relies on layered capabilities rather than one universal login:\n\n- The external gateway checks browser origin and bearer authentication.\n- Internal execution RPC checks a daemon bearer token.\n- Fork-window routes require a display and owner token.\n- Model access uses a short-lived token held by the host.\n- Shell and MCP actions pass policy and approval checks.\n- The sandbox constrains filesystem and network access when the platform supports enforcement.\n- The VM or container remains the final boundary for same-user processes, X11, loopback services, and a browser launched without its own kernel sandbox.\n\nThe observed original box also exposed an unauthenticated Docker HTTP API on port 2375 and an unidentified plaintext gRPC listener on port 50052. Neither belongs in a safe public clone. Docker API access is close to container-administrator access. The unknown service must remain blocked until its owner and authentication contract are identified.\n\nRaw VNC was loopback-only and had no VNC password. Its protection came from network reachability and the SSH or noVNC boundary. CDP, daemon tokens, owner-token files, process environments, and X sockets also need loopback binding or stronger OS isolation.\n\nSeveral mechanisms combine to make the product look like one persistent agent:\n\n1. The gateway accepts commands fast and SSE streams every later state change.\n2. Durable named agents retain transcripts, profiles, memory, workflows, and box state across turns and host restarts.\n3. Step checkpoints let approvals, secret requests, widgets, cancellations, and upgrades pause without restarting the entire turn.\n4. Task children work in the background and wake the parent when they finish.\n5. Persistent shells, MCP processes, browser profiles, cookies, and desktops let tools continue where an earlier turn stopped.\n6. Automations enter the same queue as user prompts, so unattended work follows the same persistence and policy rules.\n7. Active/passive routing keeps commands and events on one stateful gateway copy.\n\nThe apparent single bot is therefore a coordinated set of state machines. The\nconversation, model stream, Task children, approval request, shell, desktop, and\nautomation can each be in a different state. `SandHost` joins them through stable\nidentities, checkpoints, event fanout, and explicit ownership.\n\n```\nclient opens /events\nclient sends sendPrompt with clientNonce\ngateway authenticates and durably accepts\ntranscript manager emits the user message\nrunner restores context and builds the prompt\ninference streams text\nrunner checkpoints and persists the assistant message\nSSE delivers transcript changes to the client\nmodel proposes Shell\nhost and execution policy inspect the exact command\nreview requires the user\nrunner checkpoints and emits an approval request\nclient resolves it through the gateway\nrunner resumes\nexec-daemon runs the command under the computed sandbox policy\ntyped output returns to the model\nmodel finishes and the transcript settles\nparent calls Task(type=browserUse)\nparent-context review accepts the delegation\nhost creates child runner and child transcript identity\nchild receives browser tools and a routed Chrome session\nchild works while parent is suspended on a pending wake\nchild result and outline are stored\nhost revives the parent\nparent reads the result and answers the user\nhost loads automation definitions\ntrigger hub rearms an unfulfilled wake\nwake enters the named agent's ordinary queue\nrunner marks automation provenance in the prompt\nturn uses the normal inference, tools, approvals, and checkpoints\nresult is persisted and acknowledgement is retried until settled\n```\n\nThe main source roots used for this explanation are:\n\n- Gateway and composition: `src/host/main.ts` ,`sand-host.ts` ,`gateway-server.ts` ,`gateway-protocol.ts` , and`host-gateway-api.ts` .\n- Turns: `src/host/extensions/transcript/transcript-manager.ts` ,`turn-runtime.ts` ,`src/host/runner/sand-agent-runner.ts` ,`turn-run-shell.ts` , and`prompt-collector-glue.ts` .\n- Task children: `src/host/runner/subagent-runtime.ts` ,`turn-agent-composition.ts` , and`tools/sand-subagent-management-tools.ts` .\n- Inference: `src/host/extensions/auth/` ,`src/host/extensions/inference/` , and`packages/chat-inference-proto/dist/` .\n- Execution: `exec-daemon/src/` ,`packages/agent-exec/dist/` ,`packages/local-exec/dist/` , and`packages/shell-exec/dist/` .\n- Persistence: `src/host/extensions/session/` ,`src/host/agent-isolation/` ,`src/host/extensions/content-search/` , and`src/host/extensions/box-store-sync/` .\n- Exact deployment scripts: `exact-runtime-source/runtime/` .\n\nOperational VM and failover claims come from `wtfsayo/grok-bot-clone-lab` at\ncommit `2b1c877038b56a35b7643bf5082ae88a60f7b685`, especially\n`docs/GROK-BOT-CLONE-HANDBOOK.md`, `docs/GROK-BOT-DAEMONS-AND-INGRESS.md`,\n`systemd/`, and `failover/`. Those documents sanitize tokens, private addresses,\nagent IDs, transcripts, and disk locations.\n\nThe recovered system does not include the hosting provider's external control plane, the original host's vsock service, or a bootable source disk. Some native components can be described only through symbols, protocols, literals, call graphs, and disassembly because their source and debug metadata were absent.\n\nThe reconstruction is still enough to explain the application architecture with high confidence: a durable host owns conversation semantics, a gateway owns client transport, an inference service owns model transport and tokens, an execution daemon owns operating-system effects, and supervisors keep the box and its desktops alive. The Grok Bot experience appears when those parts share identities, checkpoints, and events without sharing more authority than each part needs.", "url": "https://wpnews.pro/news/how-grok-bot-works-vm-sand-host-gateway-agents-subagents-tools-persistence-and", "canonical_source": "https://gist.github.com/wtfsayo/b80f74f4f64d18ced8c84502a23148fe", "published_at": "2026-08-23 11:12:07+00:00", "updated_at": "2026-09-19 03:23:57.115906+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "agent-protocols", "developer-tools", "ai-tools"], "entities": ["Grok Bot", "Cursor", "Anysphere", "host-main.cjs", "exec-daemon", "SandHost", "MCP", "ConnectRPC"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-grok-bot-works-vm-sand-host-gateway-agents-subagents-tools-persistence-and", "markdown": "https://wpnews.pro/news/how-grok-bot-works-vm-sand-host-gateway-agents-subagents-tools-persistence-and.md", "text": "https://wpnews.pro/news/how-grok-bot-works-vm-sand-host-gateway-agents-subagents-tools-persistence-and.txt", "jsonld": "https://wpnews.pro/news/how-grok-bot-works-vm-sand-host-gateway-agents-subagents-tools-persistence-and.jsonld"}}