Make Claude your assistant in excalidraw

Drawgent is a new tool that connects a user's own Claude Code, Codex or opencode installation to a live Excalidraw whiteboard, letting the coding agent view the canvas via screenshot and scene data, edit it in place, and mark notes DONE. Setup requires one of the three agent CLIs installed and logged in, with Claude and Codex bridges needing Node.js 18 or higher, and it installs official ACP adapters (~60 MB) plus optionally Chrome Headless Shell (~120 MB) for the renderer. The editor and API run on 127.0.0.1:7300, and the scene is stored in .drawgent/scene.json, which is git-ignored automatically.

drawgent: your coding agent on a live Excalidraw canvas drawgent-your-coding-agent-on-a-live-excalidraw-canvas drawgent connects your own Claude Code, Codex or opencode your install, login, config and repo to an Excalidraw whiteboard. Ask for a diagram in the chat panel, or write AGENT: … next to the part of a drawing you want changed. The agent looks at the canvas screenshot + scene , edits it live, checks the result, and marks the note DONE . Quick start quick-start Prerequisite: one of claude , codex or opencode installed and logged in. The Claude and Codex bridges also need Node.js ≥ 18 npm . drawgent setup claude once per agent: claude | codex | opencode cd ~/my-repo drawgent up new agent session in this repo + canvas in your browser drawgent up --attach or: pick one of your running sessions and connect the canvas to it drawgent setup <agent drawgent-setup-agent Checks everything once, fails with the exact fix when something is missing, and writes ~/.config/drawgent/config.toml : 1. The agent CLI. Your claude / codex / opencode on PATH. 2. Login. claude auth status , codex login status or opencode auth list . 3. ACP bridge. - opencode speaks ACP itself opencode acp . - Claude Code and Codex use the official ACP adapters. They are installed once into ~/.cache/drawgent/adapters ~60 MB , without their bundled agent binaries, and pointed at your CLI CLAUDE CODE EXECUTABLE , CODEX PATH . - Setup then verifies the ACP handshake. 4. opencode speaks ACP itself 5. Canvas tools for attached sessions. Only Codex needs a change: codex mcp add drawgent -- drawgent mcp . Claude and opencode get the tools at attach time. 6. Headless Chrome for the renderer. Setup uses your Chrome/Chromium if you have one. Otherwise it proposes: - downloading Chrome Headless Shell Chrome for Testing, ~120 MB, no sudo into ~/.cache/drawgent/chrome , and telling you exactly which system libraries are missing, if any; - installing Chromium with your package manager apt , snap , dnf , pacman , zypper , apk , brew , or nix without sudo . Non-interactive: --chrome download | system | /path/to/chrome . 7. downloading drawgent up refuses to start until setup succeeded for that agent, or if something setup recorded disappeared. drawgent up drawgent-up Runs in the current directory the workspace : - starts the editor + API on 127.0.0.1:7300 next free port if taken ; - starts a new session of your agent over ACP, working in the workspace; - opens your browser. On a headless box it prints the ssh -L command instead. The scene is kept in .drawgent/scene.json , which is git-ignored automatically. drawgent up --attach id drawgent-up---attach-id Connects the canvas to a session you already run. Without an id it lists the sessions it finds current directory first and lets you pick one: | agent | discovery | how the canvas reaches it | |---|---|---| | Claude Code | claude agents --json interactive and background sessions | fork : a new session carrying the full conversation, driven by drawgent over ACP. Your terminal session is left untouched | | opencode | opencode servers listening locally: start the TUI with opencode --port 4096 or opencode serve | live : messages go into your running session you see them in your TUI ; drawgent adds its MCP tools to that server at runtime; replies, tool calls and prompts you type in the TUI are mirrored into the chat panel | | Codex | sessions in ~/.codex/sessions | live : codex queue --thread <id ; replies are mirrored from the session's rollout file. The session needs the drawgent MCP registered by setup, loaded when Codex starts | Using the canvas using-the-canvas - Chat panel right side : send a request, watch replies and tool calls stream in, approve permission prompts, press Stop to cancel a turn. - On the canvas : write a text starting with AGENT: next to or inside a shape, or draw an arrow from the note to a shape. It fires about 2.5 s after you stop typing, with its position, what it points at and what is nearby. The agent resolves it into a green DONE: … note. Edit it back to AGENT: to send it again. - Prompts are queued and run one at a time. A queued note that was already handled is skipped. excalidraw.com rooms excalidrawcom-rooms drawgent up --room 'https://excalidraw.com/ room=<id ,<key ' - drawgent joins the room as a collaborator 🤖 Agent , with a cursor that follows its edits . Humans can stay on excalidraw.com: their AGENT: notes reach your agent, and its edits appear there live. - Traffic is end-to-end encrypted with the room key. Empty rooms are loaded from and saved to excalidraw's Firestore storage. - The local editor mirrors the room and still has the chat panel. Other commands other-commands - drawgent mcp : stdio MCP server with the canvas tools. Agents launch it; it finds the running drawgent up by itself. - drawgent serve : low-level server with explicit agents --agent claude,codex,opencode as set up, or name=command for any ACP agent ; for scripts and containers. - Options for up / serve : --port , --room , --token API bearer + ?token= in the URL , --permissions canvas|ask|all default canvas : drawing tools auto-approved, anything else asked in the chat panel , --data , --settle-ms . Optional: Docker optional-docker docker compose up --build runs a canvas server drawgent + Chromium, no agents , e.g. to host a shared canvas or a room bridge on a server. Agents are never bundled: they always run with your own setup. Build from source build-from-source php npm ci && npm run build editor + renderer pages - dist/ embedded into the binary cargo install --path . or: cargo build --release Agent tools MCP agent-tools-mcp get scene , get screenshot vision; zoom with element ids , add elements Excalidraw skeletons; arrows bind by id and are routed edge-to-edge , add mermaid auto-layout , update elements labels follow shapes, bound arrows re-route , delete elements , clear canvas , list instructions , resolve instruction , set status . API api GET /api/health · GET /api/scene · GET /api/screenshot?ids=&padding=&max= · POST|PATCH|DELETE /api/elements · POST /api/mermaid · POST /api/clear · GET /api/instructions · POST /api/instructions/{id}/resolve · POST /api/status · POST /api/chat {agent?, text} · WS /ws browser sync + chat events Tests tests cargo test fractional indices, room crypto/framing node scripts/smoke.mjs url chat turn + AGENT: note against a running drawgent node scripts/e2e-browser.mjs url real browser: chat panel + note typed on the canvas node scripts/room-e2e.mjs fresh excalidraw.com room ↔ drawgent, both directions Layout layout src/ Rust : - main.rs : CLI. - setup.rs , config.rs , chrome.rs : setup, config, renderer install. - attach.rs : session discovery and picker. - agents.rs + acp.rs : ACP driver new / fork sessions . - live.rs : live opencode / Codex drivers. - hub.rs : routing, notes, chat log. - scene.rs : store and edit operations. - renderer.rs : Chrome over CDP. - mcp.rs : MCP server. - room.rs : excalidraw.com client. - fractional.rs , geometry.rs , el.rs : helpers. web/ : editor main.jsx , chat.jsx and renderer page render.jsx . Limits limits - The renderer needs Chrome a native renderer is planned . - Claude "attach" is a fork, because Claude Code has no public way to inject into a running terminal session. - Codex live attach is implemented, but was not yet tested against a logged-in Codex. - One scene per workspace. Images/files are not synced.