{"slug": "make-claude-your-assistant-in-excalidraw", "title": "Make Claude your assistant in excalidraw", "summary": "Drawgent is a new tool that connects a user's own Claude Code, Codex or opencode installation to a live Excalidraw whiteboard, letting the coding agent view the canvas via screenshot and scene data, edit it in place, and mark notes DONE. Setup requires one of the three agent CLIs installed and logged in, with Claude and Codex bridges needing Node.js 18 or higher, and it installs official ACP adapters (~60 MB) plus optionally Chrome Headless Shell (~120 MB) for the renderer. The editor and API run on 127.0.0.1:7300, and the scene is stored in .drawgent/scene.json, which is git-ignored automatically.", "body_md": "# drawgent: your coding agent on a live Excalidraw canvas[#](#drawgent-your-coding-agent-on-a-live-excalidraw-canvas)\n\ndrawgent connects **your own** Claude Code, Codex or opencode (your install, login, config and\nrepo) to an Excalidraw whiteboard. Ask for a diagram in the chat panel, or write `AGENT: …`\nnext to the part of a drawing you want changed. The agent looks at the canvas (screenshot +\nscene), edits it live, checks the result, and marks the note `DONE`.\n\n## Quick start[#](#quick-start)\n\nPrerequisite: one of `claude`, `codex` or `opencode` installed and logged in. The Claude and\nCodex bridges also need Node.js ≥ 18 (npm).\n\n```\ndrawgent setup claude        # once per agent: claude | codex | opencode\ncd ~/my-repo\ndrawgent up                  # new agent session in this repo + canvas in your browser\ndrawgent up --attach         # or: pick one of your running sessions and connect the canvas to it\n```\n\n### `drawgent setup <agent>`[#](#drawgent-setup-agent)\n\nChecks everything once, fails with the exact fix when something is missing, and writes\n`~/.config/drawgent/config.toml`:\n\n1. \n**The agent CLI.** Your`claude` /`codex` /`opencode` on PATH.\n2. \n**Login.**`claude auth status` ,`codex login status` or`opencode auth list` .\n3. \n**ACP bridge.**\n  - opencode speaks ACP itself (`opencode acp` ).\n  - Claude Code and Codex use the official ACP adapters. They are installed once into\n`~/.cache/drawgent/adapters` (~60 MB),*without* their bundled agent binaries, and pointed\nat**your** CLI (`CLAUDE_CODE_EXECUTABLE` ,`CODEX_PATH` ).\n  - Setup then verifies the ACP handshake.\n4. opencode speaks ACP itself (\n5. \n**Canvas tools for attached sessions.** Only Codex needs a change:`codex mcp add drawgent -- drawgent mcp` . Claude and opencode get the tools at attach time.\n6. \n**Headless Chrome for the renderer.** Setup uses your Chrome/Chromium if you have one.\nOtherwise it proposes:\n  - downloading **Chrome Headless Shell** (Chrome for Testing, ~120 MB, no sudo) into`~/.cache/drawgent/chrome` , and telling you exactly which system libraries are missing, if\nany;\n  - installing Chromium with your package manager (`apt` ,`snap` ,`dnf` ,`pacman` ,`zypper` ,`apk` ,`brew` , or`nix` without sudo).\n Non-interactive: `--chrome download | system | /path/to/chrome` .\n7. downloading \n\n`drawgent up` refuses to start until setup succeeded for that agent, or if something setup\nrecorded disappeared.\n\n### `drawgent up`[#](#drawgent-up)\n\nRuns in the current directory (the workspace):\n\n- starts the editor + API on `127.0.0.1:7300` (next free port if taken);\n- starts a new session of your agent over ACP, working in the workspace;\n- opens your browser. On a headless box it prints the `ssh -L` command instead.\n\nThe scene is kept in `.drawgent/scene.json`, which is git-ignored automatically.\n\n### `drawgent up --attach [id]`[#](#drawgent-up---attach-id)\n\nConnects the canvas to a session you already run. Without an id it lists the sessions it finds (current directory first) and lets you pick one:\n\n| agent | discovery | how the canvas reaches it | \n|---|---|---|\n| Claude Code | `claude agents --json` (interactive and background sessions) | **fork** : a new session carrying the full conversation, driven by drawgent over ACP. Your terminal session is left untouched | \n| opencode | opencode servers listening locally: start the TUI with `opencode --port 4096` (or`opencode serve` ) | **live** : messages go into your running session (you see them in your TUI); drawgent adds its MCP tools to that server at runtime; replies, tool calls and prompts you type in the TUI are mirrored into the chat panel | \n| Codex | sessions in `~/.codex/sessions` | **live** :`codex queue --thread <id>` ; replies are mirrored from the session's rollout file. The session needs the drawgent MCP (registered by setup, loaded when Codex starts) | \n\n## Using the canvas[#](#using-the-canvas)\n\n- **Chat panel** (right side): send a request, watch replies and tool calls stream in, approve\npermission prompts, press**Stop** to cancel a turn.\n- **On the canvas** : write a text starting with`AGENT:` next to or inside a shape, or draw an\narrow from the note to a shape. It fires about 2.5 s after you stop typing, with its\nposition, what it points at and what is nearby. The agent resolves it into a green`DONE: …` note. Edit it back to`AGENT:` to send it again.\n- Prompts are queued and run one at a time. A queued note that was already handled is skipped.\n\n## excalidraw.com rooms[#](#excalidrawcom-rooms)\n\n```\ndrawgent up --room 'https://excalidraw.com/#room=<id>,<key>'\n```\n\n- drawgent joins the room as a collaborator (`🤖 Agent` , with a cursor that follows its edits).\nHumans can stay on excalidraw.com: their`AGENT:` notes reach your agent, and its edits\nappear there live.\n- Traffic is end-to-end encrypted with the room key. Empty rooms are loaded from and saved to excalidraw's Firestore storage.\n- The local editor mirrors the room and still has the chat panel.\n\n## Other commands[#](#other-commands)\n\n- `drawgent mcp` : stdio MCP server with the canvas tools. Agents launch it; it finds the running`drawgent up` by itself.\n- `drawgent serve` : low-level server with explicit agents (`--agent claude,codex,opencode` as set up,\nor`name=command` for any ACP agent); for scripts and containers.\n- Options for `up` /`serve` :`--port` ,`--room` ,`--token` (API bearer +`?token=` in the URL),`--permissions canvas|ask|all` (default`canvas` : drawing tools auto-approved, anything else\nasked in the chat panel),`--data` ,`--settle-ms` .\n\n## Optional: Docker[#](#optional-docker)\n\n`docker compose up --build` runs a canvas server (drawgent + Chromium, **no agents**), e.g. to\nhost a shared canvas or a room bridge on a server. Agents are never bundled: they always run\nwith your own setup.\n\n## Build from source[#](#build-from-source)\n\n``` php\nnpm ci && npm run build      # editor + renderer pages -> dist/ (embedded into the binary)\ncargo install --path .       # or: cargo build --release\n```\n\n## Agent tools (MCP)[#](#agent-tools-mcp)\n\n`get_scene`, `get_screenshot` (vision; zoom with `element_ids`), `add_elements` (Excalidraw\nskeletons; arrows bind by id and are routed edge-to-edge), `add_mermaid` (auto-layout),\n`update_elements` (labels follow shapes, bound arrows re-route), `delete_elements`,\n`clear_canvas`, `list_instructions`, `resolve_instruction`, `set_status`.\n\n## API[#](#api)\n\n`GET /api/health` · `GET /api/scene` · `GET /api/screenshot?ids=&padding=&max=` ·\n`POST|PATCH|DELETE /api/elements` · `POST /api/mermaid` · `POST /api/clear` ·\n`GET /api/instructions` · `POST /api/instructions/{id}/resolve` · `POST /api/status` ·\n`POST /api/chat {agent?, text}` · `WS /ws` (browser sync + chat events)\n\n## Tests[#](#tests)\n\n```\ncargo test                          # fractional indices, room crypto/framing\nnode scripts/smoke.mjs [url]        # chat turn + AGENT: note against a running drawgent\nnode scripts/e2e-browser.mjs [url]  # real browser: chat panel + note typed on the canvas\nnode scripts/room-e2e.mjs           # fresh excalidraw.com room ↔ drawgent, both directions\n```\n\n## Layout[#](#layout)\n\n`src/` (Rust):\n\n- `main.rs` : CLI.\n- `setup.rs` ,`config.rs` ,`chrome.rs` : setup, config, renderer install.\n- `attach.rs` : session discovery and picker.\n- `agents.rs` +`acp.rs` : ACP driver (new / fork sessions).\n- `live.rs` : live opencode / Codex drivers.\n- `hub.rs` : routing, notes, chat log.\n- `scene.rs` : store and edit operations.\n- `renderer.rs` : Chrome over CDP.\n- `mcp.rs` : MCP server.\n- `room.rs` : excalidraw.com client.\n- `fractional.rs` ,`geometry.rs` ,`el.rs` : helpers.\n\n`web/`: editor (` main.jsx`, `chat.jsx`) and renderer page (` render.jsx`).\n\n## Limits[#](#limits)\n\n- The renderer needs Chrome (a native renderer is planned).\n- Claude \"attach\" is a fork, because Claude Code has no public way to inject into a running terminal session.\n- Codex live attach is implemented, but was not yet tested against a logged-in Codex.\n- One scene per workspace. Images/files are not synced.", "url": "https://wpnews.pro/news/make-claude-your-assistant-in-excalidraw", "canonical_source": "https://tangled.org/yanndegat.tngl.sh/drawgent", "published_at": "2026-09-26 15:56:34+00:00", "updated_at": "2026-09-26 17:31:17.290207+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-tools"], "entities": ["drawgent", "Claude Code", "Codex", "opencode", "Excalidraw", "Node.js", "Chrome Headless Shell", "ACP"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/make-claude-your-assistant-in-excalidraw", "markdown": "https://wpnews.pro/news/make-claude-your-assistant-in-excalidraw.md", "text": "https://wpnews.pro/news/make-claude-your-assistant-in-excalidraw.txt", "jsonld": "https://wpnews.pro/news/make-claude-your-assistant-in-excalidraw.jsonld"}}