{"slug": "how-i-got-computer-use-and-browser-use-running-with-codex-in-e2b", "title": "How I got Computer Use and Browser Use running with Codex in E2B", "summary": "A developer documented a custom integration that runs OpenAI's first-party Computer Use and Browser Use runtimes inside an E2B sandbox, pairing the Codex CLI with a virtual Linux desktop built from Xvfb, Openbox, and Chromium. The setup extracts runtime artifacts from a pinned ChatGPT Linux package, registers cua_repl and node_repl as MCP servers, and relies on persistent kernels and installed skills to drive screenshots, OS input, and browser automation. The developer noted that Browser Use spawns a separate native Codex child for authentication that cannot inherit the main process's in-memory credentials.", "body_md": "I wanted my background coding agent to use the app it was building. Open Chromium, inspect the page, click through a flow, and see whether it actually works.\n\nI got it running with `codex app-server`, OpenAI’s first-party computer/browser runtimes, and a Linux desktop inside E2B.\n\nHere’s how I put it together, including the auth issues that took me a while to figure out.\n\nMy E2B image runs:\n\n```\nXvfb :99\n  └─ Openbox\n      └─ Chromium + OpenAI's Chrome extension\n```\n\nXvfb provides a virtual screen. Openbox manages the windows. Chromium runs as a desktop application on that screen.\n\nThe desktop and tool runtimes share:\n\n```\nDISPLAY=:99\n```\n\nI didn’t need XFCE or the ChatGPT Electron UI. The agent can capture screenshots and interact with the desktop without a human streaming it.\n\nInstalling the Codex CLI alone doesn’t give you the computer/browser stack.\n\nI extracted the runtime artifacts from a pinned Linux ChatGPT package:\n\n- the bundled Node runtime and `node_repl`\n- the `chrome` and`unified-computer-use` plugins\n- the native messaging host\n- the first-party skills, including the Linux Sky instructions\n\nI installed the matching Chrome extension separately and preserved its extension ID.\n\nThe combination I tested:\n\n| Artifact | Version | \n|---|---|\n| ChatGPT Linux package | `26.901.51231` | \n| Codex CLI | `0.153.4` | \n| Chrome extension | `1.26.901.11451` | \n\nI verified the download hashes and checked that the bundled and canonical Codex versions matched.\n\nThis is a custom integration using the packaged runtime, not an official standalone installation recipe. I keep the images private; the runtime artifacts are proprietary.\n\n```\ncodex app-server\n  │\n  ├─ cua_repl\n  │    └─ Linux Sky helper\n  │         └─ screenshots + keyboard/mouse input\n  │\n  └─ node_repl\n       └─ first-party browser service\n            └─ native messaging host\n                 └─ Chrome extension\n                      └─ Chromium\n```\n\nComputer Use works through screenshots and operating-system input.\n\nBrowser Use provides browser-specific APIs through the extension.\n\nI also installed Playwright CLI as a separate tool. It isn’t Playwright MCP, and it doesn’t replace either first-party runtime.\n\nI registered `cua_repl` and `node_repl` as MCP servers in the image’s Codex configuration.\n\nThe important settings are:\n\n```\nComputer Use:\n  DISPLAY=:99\n  CUA_REPL_ENABLED_SURFACES=computer\n\nBrowser Use:\n  DISPLAY=:99\n  BROWSER_USE_AVAILABLE_BACKENDS=chrome\n  NODE_REPL_TRUSTED_SERVICES={\"browser\":\"<plugin path>/scripts/browser-service.mjs\"}\n```\n\nI kept the complete plugin directories at stable image-owned paths and copied their skills into:\n\n```\n/home/user/.agents/skills/\n├─ computer-use/\n└─ control-chrome/\n```\n\nThe skills matter. They teach Codex how to initialize the runtimes and use the actual APIs.\n\nFor Computer Use, initialization happens inside `cua_repl`:\n\n```\nawait cua.getState();\n```\n\nThen, in a later call in the same kernel:\n\n```\nvar { sky } = await import(\"@oai/sky\");\n\nawait nodeRepl.emitImage(\n  (await sky.get_screenshot())[0].data_url\n);\n```\n\nFor Browser Use, the installed skill points to the browser client:\n\n```\nvar { setupBrowserRuntime } = await import(\n  \"/opt/coding-agent/image/desktop/plugins/chrome/scripts/browser-client.mjs\"\n);\n\nvar agent = await setupBrowserRuntime();\n\nnodeRepl.write(await agent.browsers.list());\n```\n\nThat runs inside the first-party `node_repl`, not an ordinary Node shell.\n\nOne easy mistake: starting a fresh process for every snippet. These tools expect a persistent kernel, so the imported modules and browser handles survive between calls.\n\nUse the installed skill for browser selection and method signatures. Don’t guess the APIs from another computer-use implementation.\n\nThis was the part I didn’t expect.\n\nMy main Codex process already received credentials through RPC. But Browser Use started a separate native Codex child for authentication and configuration reads.\n\nThat child couldn’t inherit the main process’s in-memory login.\n\nThe extension connected, but navigation failed with:\n\n```\nCodex auth token is unavailable\n```\n\nI didn’t want to write an `auth.json` into the sandbox, so I added a per-run Unix-socket broker:\n\n```\nBrowser runtime\n  → stdio bridge\n  → private Unix socket\n  → broker\n  → real canonical Codex child\n```\n\nThe broker initializes the child and authenticates it through RPC before returning the initialization response.\n\nBrowser Use can then request auth status and configuration from that child. Credentials still come from my existing control-plane callbacks.\n\nThe bridge keeps credentials out of an auth file. Its same-user socket is not a credential-isolation boundary.\n\nOnce the child was authenticated, I hit:\n\n```\nunsupported Codex auth method: chatgptAuthTokens\n```\n\nThe pinned browser consumer expected `\"chatgpt\"`. My external RPC login correctly returned `\"chatgptAuthTokens\"`.\n\nI added a narrow compatibility translation to the browser-facing auth-status response.\n\nThe native login stays external. The access token stays unchanged. Before translating the label, I check that the returned token and its account claim match the credentials installed in that child.\n\nThis is specific to the versions I tested. It doesn’t change credentials or remote authorization.\n\nAnother failure said:\n\n```\nadmin-enforced policy could not be verified\n```\n\nThe cause was in my bridge.\n\nThe native client sent:\n\n```\n{\"id\":\"1\",\"method\":\"configRequirements/read\"}\n```\n\nI had required an explicit `\"params\": null`.\n\nRejecting that valid parameterless request broke the config read. Browser Use reported the failure as a policy-verification error.\n\nAccepting the actual wire format fixed it. I didn’t need to bypass the policy check.\n\nI install the desktop and runtime artifacts during image construction.\n\nA systemd service starts Xvfb, Openbox, and Chromium before the image’s ready marker. E2B captures that running desktop in the snapshot.\n\n```\nBuild image\n  → install desktop + first-party runtimes\n  → write Codex configuration\n  → start desktop\n  → capture ready snapshot\n\nCreate sandbox\n  → attach to the existing desktop\n  → authenticate Codex through RPC\n  → run computer/browser tools\n```\n\nPause/resume preserves the desktop rather than rebuilding it for every prompt.\n\nOne detail worth remembering: changing the image only affects new sandboxes. Existing ones keep their image-baked tools.\n\nThat lets my background agent run its coding work and use Chromium in the same sandbox, through the same Codex session.", "url": "https://wpnews.pro/news/how-i-got-computer-use-and-browser-use-running-with-codex-in-e2b", "canonical_source": "https://gist.github.com/0xthierry/0c6fc4789b4711db94e0e2db050b0626", "published_at": "2026-09-09 23:35:03+00:00", "updated_at": "2026-09-22 12:26:24.251397+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "agent-protocols", "ai-products"], "entities": ["OpenAI", "Codex", "E2B", "Chromium", "Xvfb", "Openbox", "Playwright", "ChatGPT"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-i-got-computer-use-and-browser-use-running-with-codex-in-e2b", "markdown": "https://wpnews.pro/news/how-i-got-computer-use-and-browser-use-running-with-codex-in-e2b.md", "text": "https://wpnews.pro/news/how-i-got-computer-use-and-browser-use-running-with-codex-in-e2b.txt", "jsonld": "https://wpnews.pro/news/how-i-got-computer-use-and-browser-use-running-with-codex-in-e2b.jsonld"}}