Make Claude your assistant in excalidraw

drawgent connects your own Claude Code, Codex or opencode (your install, login, config and repo) to an Excalidraw whiteboard. Ask for a diagram in the chat panel, or write AGENT: … next to the part of a drawing you want changed. The agent looks at the canvas (screenshot + scene), edits it live, checks the result, and marks the note DONE.

Quick start# #

Prerequisite: one of claude, codex or opencode installed and logged in. The Claude and Codex bridges also need Node.js ≥ 18 (npm).

drawgent setup claude        # once per agent: claude | codex | opencode
cd ~/my-repo
drawgent up                  # new agent session in this repo + canvas in your browser
drawgent up --attach         # or: pick one of your running sessions and connect the canvas to it

drawgent setup <agent>#

Checks everything once, fails with the exact fix when something is missing, and writes ~/.config/drawgent/config.toml:

The agent CLI. Yourclaude /codex /opencode on PATH. 2. Login.claude auth status ,codex login status oropencode auth list . 3. ACP bridge.

• opencode speaks ACP itself (opencode acp ).
• Claude Code and Codex use the official ACP adapters. They are installed once into ~/.cache/drawgent/adapters (~60 MB),without their bundled agent binaries, and pointed atyour CLI (CLAUDE_CODE_EXECUTABLE ,CODEX_PATH ).
• Setup then verifies the ACP handshake.

4. opencode speaks ACP itself (

Canvas tools for attached sessions. Only Codex needs a change:codex mcp add drawgent -- drawgent mcp . Claude and opencode get the tools at attach time. 6. Headless Chrome for the renderer. Setup uses your Chrome/Chromium if you have one. Otherwise it proposes:

• down Chrome Headless Shell (Chrome for Testing, ~120 MB, no sudo) into~/.cache/drawgent/chrome , and telling you exactly which system libraries are missing, if any;
• installing Chromium with your package manager (apt ,snap ,dnf ,pacman ,zypper ,apk ,brew , ornix without sudo). Non-interactive: --chrome download | system | /path/to/chrome .

7. down

drawgent up refuses to start until setup succeeded for that agent, or if something setup recorded disappeared.

drawgent up#

Runs in the current directory (the workspace):

• starts the editor + API on 127.0.0.1:7300 (next free port if taken);
• starts a new session of your agent over ACP, working in the workspace;
• opens your browser. On a headless box it prints the ssh -L command instead.

The scene is kept in .drawgent/scene.json, which is git-ignored automatically.

drawgent up --attach [id]#

Connects the canvas to a session you already run. Without an id it lists the sessions it finds (current directory first) and lets you pick one:

Using the canvas# #

• Chat panel (right side): send a request, watch replies and tool calls stream in, approve permission prompts, pressStop to cancel a turn.
• On the canvas : write a text starting withAGENT: next to or inside a shape, or draw an arrow from the note to a shape. It fires about 2.5 s after you stop typing, with its position, what it points at and what is nearby. The agent resolves it into a greenDONE: … note. Edit it back toAGENT: to send it again.
• Prompts are queued and run one at a time. A queued note that was already handled is skipped.

excalidraw.com rooms# #

drawgent up --room 'https://excalidraw.com/#room=<id>,<key>'

• drawgent joins the room as a collaborator (🤖 Agent , with a cursor that follows its edits). Humans can stay on excalidraw.com: theirAGENT: notes reach your agent, and its edits appear there live.
• Traffic is end-to-end encrypted with the room key. Empty rooms are loaded from and saved to excalidraw's Firestore storage.
• The local editor mirrors the room and still has the chat panel.

Other commands# #

• drawgent mcp : stdio MCP server with the canvas tools. Agents launch it; it finds the runningdrawgent up by itself.
• drawgent serve : low-level server with explicit agents (--agent claude,codex,opencode as set up, orname=command for any ACP agent); for scripts and containers.
• Options for up /serve :--port ,--room ,--token (API bearer +?token= in the URL),--permissions canvas|ask|all (defaultcanvas : drawing tools auto-approved, anything else asked in the chat panel),--data ,--settle-ms .

Optional: Docker# #

docker compose up --build runs a canvas server (drawgent + Chromium, no agents), e.g. to host a shared canvas or a room bridge on a server. Agents are never bundled: they always run with your own setup.

Build from source# #

npm ci && npm run build      # editor + renderer pages -> dist/ (embedded into the binary)
cargo install --path .       # or: cargo build --release

Agent tools (MCP)# #

get_scene, get_screenshot (vision; zoom with element_ids), add_elements (Excalidraw skeletons; arrows bind by id and are routed edge-to-edge), add_mermaid (auto-layout), update_elements (labels follow shapes, bound arrows re-route), delete_elements, clear_canvas, list_instructions, resolve_instruction, set_status.

API# #

GET /api/health · GET /api/scene · GET /api/screenshot?ids=&padding=&max= · POST|PATCH|DELETE /api/elements · POST /api/mermaid · POST /api/clear · GET /api/instructions · POST /api/instructions/{id}/resolve · POST /api/status · POST /api/chat {agent?, text} · WS /ws (browser sync + chat events)

Tests# #

cargo test                          # fractional indices, room crypto/framing
node scripts/smoke.mjs [url]        # chat turn + AGENT: note against a running drawgent
node scripts/e2e-browser.mjs [url]  # real browser: chat panel + note typed on the canvas
node scripts/room-e2e.mjs           # fresh excalidraw.com room ↔ drawgent, both directions

Layout# #

src/ (Rust):

• main.rs : CLI.
• setup.rs ,config.rs ,chrome.rs : setup, config, renderer install.
• attach.rs : session discovery and picker.
• agents.rs +acp.rs : ACP driver (new / fork sessions).
• live.rs : live opencode / Codex drivers.
• hub.rs : routing, notes, chat log.
• scene.rs : store and edit operations.
• renderer.rs : Chrome over CDP.
• mcp.rs : MCP server.
• room.rs : excalidraw.com client.
• fractional.rs ,geometry.rs ,el.rs : helpers.

web/: editor ( main.jsx, chat.jsx) and renderer page ( render.jsx).

Limits# #

• The renderer needs Chrome (a native renderer is planned).
• Claude "attach" is a fork, because Claude Code has no public way to inject into a running terminal session.
• Codex live attach is implemented, but was not yet tested against a logged-in Codex.
• One scene per workspace. Images/files are not synced.