Show HN: Vitrin OS – An agent-first display server with per-app isolation Vitrin OS is an open-source, agent-first display server that provides per-app isolation for human and AI agents to concurrently observe and operate GUIs under granular, revocable, capability-scoped authorization. The system uses a small trusted core with a new wire protocol, confining legacy Wayland/X11 applications to unprivileged nested shims, enabling precise control such as allowing an agent to fill one form in one Firefox window for five minutes while unable to see other windows, with instant human override via mouse or Escape key. Vitrin OS is an open-source, agent-first display server : a small trusted core vitrind speaking a new capability-native wire protocol, with every legacy Wayland https://wayland.freedesktop.org/ /X11 application confined to its own per-app nested shim — so that humans and AI agents can concurrently observe and operate GUIs under granular, revocable, capability-scoped authorization. Every principal has identity; every action carries a capability; every action is journaled; the trusted core stays small and legacy complexity is exiled to disposable, unprivileged shims. The full vision, object model, and technical architecture live in docs/PRD.md /vitrin-os/vitrin-os/blob/main/docs/PRD.md . The wire protocol is specified in protocol/vitrin-v0.xml /vitrin-os/vitrin-os/blob/main/protocol/vitrin-v0.xml with prose in docs/protocol/ /vitrin-os/vitrin-os/blob/main/docs/protocol/00-conventions.md . One sentence, no jargon: An agent is allowed to fill in one form, in one Firefox window, for the next five minutes. It cannot see the password manager open beside it. The moment you touch the mouse, you have control back. Hold Escape for a second and its authority is gone — mid-click, mid-keystroke, whatever it was doing. Every clause there is a mechanism, not a policy setting: | The claim | What makes it true | |---|---| allowed to fill in one form | A grant: principal × resource × verbs × constraints . Not a role, not a config flag — a row in the grant table the core checks on every single action. | for the next five minutes | Expiry is a constraint on the grant, enforced at the chokepoint. There is no path that skips it. | cannot see the password manager | The agent's Firefox is in a realm, talking to its own private nested shim. The other window is not "hidden" from it; it is not in its universe. Scoping is structural. | you touch the mouse, you have control | Physical input is origin-tagged at the core and preempts agent input by construction, not by a race between two clients. | hold Escape and its authority is gone | The dead-man switch: the core revokes every live grant, and the agent's next call fails revoked . Proven against a real app in test real deadman.py . | The prompt that asks you to approve any of this is drawn by the trusted core itself — the thing that owns the screen — so no client can paint a fake one over it. That property has its own gate test real consent.py : the prompt is shown to occlude the app's real pixels while the agent's own capture path sees the app unchanged. Agents that drive desktops today work screenshot-by-screenshot: capture the screen, pick pixel coordinates, click, capture again. The loop is slow, token-hungry, and race-prone — and it runs with all-or-nothing authority. The isolation unit is a whole VM or desktop session, so a single prompt-injected agent's blast radius is everything on screen. There is no structural way to say: this agent may operate this one form in this one app, may not read the password-manager window beside it, and loses all input the instant a human touches the keyboard. The underlying stack cannot express that sentence. X11 https://www.x.org/wiki/ grants every client near-total authority over the session — that is its protocol model, not a bug. Wayland achieved isolation by removing cross-client capabilities rather than mediating them, and its wl seat singleton has no notion of N concurrent authenticated principals. AT-SPI2 https://gitlab.gnome.org/GNOME/at-spi2-core , the accessibility tree agents use to avoid pixels, is an unauthorized backdoor onto every application's widgets. The managed-cloud answers AWS WorkSpaces https://aws.amazon.com/workspaces/ai-agents/ for AI agents, Windows 365 https://www.microsoft.com/en-us/windows-365/agents for Agents have the right instinct — identity per agent, audit, oversight — but at whole-VM granularity, locked to proprietary clouds. Vitrin is designed from day zero around the missing primitives: Principals. Every connection authenticates an identity human or agent workload at handshake; agent credentials are SPIFFE https://spiffe.io/ /OIDC-shaped. Grants. No ambient authority. A grant is principal × resource × verbs × constraints — expiry, rate ceilings, focus conditions , sender-constrained to the connection, attenuable, and revocable — immediately and transitively. Consent. Rendered by the trusted core itself, which owns the screen and input, so prompts cannot be spoofed by any client. Realms and shims. Apps launch into realms; each legacy app gets its own private nested shim server, so its universe contains only itself — scoping is structural, not policy. Human override. Physical input preempts agent input by construction. See docs/PRD.md /vitrin-os/vitrin-os/blob/main/docs/PRD.md §1 for the full problem statement and §6 for the pillar-by-pillar requirements. Phase 1 is complete , and there is a 60-second recording https://vitrin-os.github.io/vitrin-os/ demo of it working: the core-drawn consent card over a real Firefox, a human clicking Allow, the agent filling a record it was handed, and — in a second take — a physically held Escape revoking every grant mid-task so the run exits non-zero. Two takes rather than one, because a revoked run cannot also print PASS .All nine epics E1–E9 have landed on main , and every milestone's named acceptance gate is closed under the rule this project holds itself to — decision D12 : a milestone closes only when a named integration test passes green against the shipped binaries with no mock on any seam it claims. Component tests built on vitrin-mock-shim /vitrin-os/vitrin-os/blob/main/crates/vitrin-mock-shim are never that evidence, and are labelled as what they are. | Milestone | Gate | Proven by | |---|---|---| M1.2 — buffer path | | tests/integration/test real app.py — real core + real shim + real weston-terminal M1.3 — observation 107 https://github.com/vitrin-os/vitrin-os/issues/107 test real capture fidelity.py — an agent observes a real app through the enforcement chokepoint M1.4 — actuation, consent, dead-man 108 https://github.com/vitrin-os/vitrin-os/issues/108 + 109 https://github.com/vitrin-os/vitrin-os/issues/109 + 138 https://github.com/vitrin-os/vitrin-os/issues/138 test real actuation.py , test real deadman.py , test real consent.py M1.5 — demo 110 https://github.com/vitrin-os/vitrin-os/issues/110 test demo.py — the demo agent is handed a task record it did not author, fills it into a real app and submits it, and the gate demands the confirmation carry that record's 36-bit checksum a positive content check, not a pixel diff plus the app's own byte-exact report. The app, form-target , is repo-authored — a real Wayland client, no mock, but less independent than the weston-terminal it replaced; disclosed in and the D12 seam table /vitrin-os/vitrin-os/blob/main/examples/agent-demo/README.md examples/agent-demo/README.md What exists today, on main : Protocol spec v0 — protocol/vitrin-v0.xml /vitrin-os/vitrin-os/blob/main/protocol/vitrin-v0.xml 11 interfaces, wire format, error taxonomy; the source of truth , its RELAX NG schema protocol/vitrin-v0.rng /vitrin-os/vitrin-os/blob/main/protocol/vitrin-v0.rng , and a prose page per interface under docs/protocol/ /vitrin-os/vitrin-os/blob/main/docs/protocol/00-conventions.md kept in lockstep with every landing PR. Codegen —plus vitrin-scanner cargo xtask codegen generate theRust crate message types + codec; pure data, no I/O and the C header vitrin-protocol shim/include/vitrin-protocol.h /vitrin-os/vitrin-os/blob/main/shim/include/vitrin-protocol.h from the IDL. vitrind , the trusted core — a real crates/vitrin-core Smithay https://github.com/Smithay/smithay compositor with both output backends --nested , a host Wayland client; --headless --size WxH , GPU-less pixman software rendering for CI ; the capability kernel and in-memory grant table request → pending → consent → resolved, sender-constrained, rate-limited, revocable ; the realm/spawn manager fork/exec the shim with a private runtime dir and a scrubbed, allow-listed environment ; the core-rendered consent prompt with an exclusive input grab; the hold-Esc dead-man revocation switch; the dmabuf https://docs.kernel.org/driver-api/dma-buf.html import path — zero-copy shim→core frames on a real GPU, with shm as the universal fallback and an explicit dmabuf.rs buffer done import failed event rather than a silent black frame, so CI stays GPU-free ; and the flight-recorder log. See the Architecture at a glance architecture-at-a-glance section and docs/ARCHITECTURE.md /vitrin-os/vitrin-os/blob/main/docs/ARCHITECTURE.md for the full crate↔PRD map. The Wayland shim — a shim/ wlroots https://gitlab.freedesktop.org/wlroots/wlroots headless compositor, C + Meson https://mesonbuild.com/ , outside the Cargo workspace by design. It forwards every composited frame to the core over the inherited socketpair, replays origin-tagged input into the app through its own wl seat , and runs real apps: weston-terminal , a GTK https://www.gtk.org/ entry probe, and Firefox ESR https://www.mozilla.org/en-US/firefox/enterprise/ pinned , climbed one rung at a time and proven in CI with no mock on the shim seam. The agent SDK — a pure-Python stdlib-only, D8 wire client: connect → handshake → sdk/python/ request grant → observe → actuate pointer / actuate text , typed grant-error exceptions, and capture ergonomics .to png .is the demo agent and doubles as the M1.5 integration test; run it with examples/agent-demo/run demo.py cargo xtask demo see Quickstart quickstart below . CI — .github/workflows/ci.yml /vitrin-os/vitrin-os/blob/main/.github/workflows/ci.yml : Rust fmt/clippy/tests debug + release , IDL schema validation and generated-code drift checking, a Rust-free container build of the C shim, golden-frame pixel/SSIM checks, and a headless integration suite that drives the real core against real apps weston-terminal , GTK, Firefox ESR with a hard 10-minute budget. Phase 1 being complete is a statement about a defined slice, not about readiness for anything real. Read this list before the pitch above convinces you of more than it should. No sandbox decision D9, closes in Phase 2 . No namespaces, no seccomp https://man7.org/linux/man-pages/man2/seccomp.2.html , no Landlock https://landlock.io/ . A realm's app runs as the core's own uid with the core's full view of the filesystem and network, and the session D-Bus remains reachable. Environment hygiene confines the well-behaved; it does not contain the hostile. This is the big one — see Security notes security-notes--what-the-mvp-does-and-does-not-confine . The 24-hour fuzz soak has not been run. fuzz/ ships two cargo-fuzz targets protocol decode, vitrin-ipc framing with a checked-in corpus CI replays on every PR plus a short per-PR burst, but the 24-hour clean run the plan asks for is still a manual, documented procedure rather than a scheduled job — fuzz/README.md /vitrin-os/vitrin-os/blob/main/fuzz/README.md says so in its own words. Nobody has run it end to end. wlcs conformance is advisory and mostly red: total=180 passed=3 failed=145 skipped=32 on the 2026-07-25 run. That number is expected and is not a bug count — wlcs https://github.com/canonical/wlcs tests a general-purpose desktop compositor, and the shim deliberately serves a narrow surface no touch, no full xdg-shell policy, no decoration protocols , so most failures are "no such global" rather than misbehaviour. It is still the honest number, it has not been re-measured since, and shim/wlcs/README.md distinguishes the structural absences from the genuine ones. Never built by default; never gates a PR. One maintainer. Governance is a documented BDFL plan §5 /vitrin-os/vitrin-os/blob/main/docs/plan/12-workstream-community.md ; bus factor is tracked as a first-class project risk, not waved away. v0 protocol, and it will break. The IDL is frozen for Phase 1, not forever — Phase 2 adds semantic trees and epoch/CAS action semantics, and v0 clients should expect to move. Nothing above is discovered-by-a-reader; each is a decision with a recorded rationale in docs/plan/20-decision-log.md /vitrin-os/vitrin-os/blob/main/docs/plan/20-decision-log.md . From a clean clone to a running demo. Verified against the state of main described above headless venue; the nested venue additionally needs a Wayland session and Firefox ESR — see the note at the end . The demo runs the real wlroots shim in both venues, so unlike a Rust-only build it needs the C shim built and a real Wayland client installed. cargo xtask demo fails with the exact meson command below if the shim is missing, rather than silently substituting anything. git clone https://github.com/vitrin-os/vitrin-os.git cd vitrin-os The toolchain is pinned rust-toolchain.toml ; rustup installs it on the first cargo invocation. System deps the workspace links, plus the C shim's build deps and a real Wayland client for it to run: sudo apt-get install -y libxkbcommon-dev libpixman-1-dev weston Debian/Ubuntu bash shim/ci/install-deps.sh Meson + wlroots deps Build vitrind, xtask, and the fixtures the test suites use. cargo build --workspace Build the real per-app wlroots shim C + Meson, outside the Cargo workspace . cargo xtask demo looks for it at shim/build/vitrin-shim, or wherever VITRIN C SHIM BIN points. meson setup shim/build shim && meson compile -C shim/build Run the Phase-1 demo agent headless: boots vitrind --headless, which execs the real shim, which fork/execs a real Wayland client form-target, co-built with the shim in its own confined Wayland socket, and drives examples/agent-demo/run demo.py over a real Unix socket. The agent is handed a task record it did not author, locates each form field by its marker colour in its own captured frame, clicks, types the value, clicks the located submit button, and then decodes the confirmation's three receipt bands -- a 36-bit checksum of the record the app received, computed from the SUPPLIED task at runtime. Exits non-zero on any failure. cargo xtask demo --headless ... or with a record the agent has never seen: cargo xtask demo --headless --task name=Grace --task email=grace@example.net Expect output ending in xtask demo: PASS and a path to the run's flight recorder flight.jsonl and captured frames. This exercises the real wire protocol, the real capability kernel, the real consent auto-approve path, the real wlroots shim, and a real app — vitrin-mock-shim appears in no demo venue 110 https://github.com/vitrin-os/vitrin-os/issues/110 / 127 https://github.com/vitrin-os/vitrin-os/pull/127 ; it survives only as a unit-test fixture for the component tests in tests/integration/ . The same chain under the full integration suite, including the named milestone gates: VITRIN C SHIM BIN="$PWD/shim/build/vitrin-shim" bash tests/integration/run.sh Nested mode cargo xtask demo , no --headless draws a real window on your own Wayland session and launches Firefox ESR in the realm — it needs a running compositor GNOME, Hyprland, ... and firefox-esr or VITRIN DEMO FIREFOX=/path/to/firefox on the machine you run it on; it is never a CI dependency nested mode has no headless equivalent by design — plan risk R1 . Other useful commands, all covered in CI: xmllint --noout --relaxng protocol/vitrin-v0.rng protocol/vitrin-v0.xml validate the IDL cargo test --workspace && cargo test --workspace --release unit + integration tests cargo xtask codegen --check generated-code drift check php flowchart TD HUMAN "human