# Make Claude your assistant in excalidraw

> Source: <https://tangled.org/yanndegat.tngl.sh/drawgent>
> Published: 2026-09-26 15:56:34+00:00

# drawgent: your coding agent on a live Excalidraw canvas[#](#drawgent-your-coding-agent-on-a-live-excalidraw-canvas)

drawgent connects **your own** Claude Code, Codex or opencode (your install, login, config and
repo) to an Excalidraw whiteboard. Ask for a diagram in the chat panel, or write `AGENT: …`
next to the part of a drawing you want changed. The agent looks at the canvas (screenshot +
scene), edits it live, checks the result, and marks the note `DONE`.

## Quick start[#](#quick-start)

Prerequisite: one of `claude`, `codex` or `opencode` installed and logged in. The Claude and
Codex bridges also need Node.js ≥ 18 (npm).

```
drawgent setup claude        # once per agent: claude | codex | opencode
cd ~/my-repo
drawgent up                  # new agent session in this repo + canvas in your browser
drawgent up --attach         # or: pick one of your running sessions and connect the canvas to it
```

### `drawgent setup <agent>`[#](#drawgent-setup-agent)

Checks everything once, fails with the exact fix when something is missing, and writes
`~/.config/drawgent/config.toml`:

1. 
**The agent CLI.** Your`claude` /`codex` /`opencode` on PATH.
2. 
**Login.**`claude auth status` ,`codex login status` or`opencode auth list` .
3. 
**ACP bridge.**
  - opencode speaks ACP itself (`opencode acp` ).
  - Claude Code and Codex use the official ACP adapters. They are installed once into
`~/.cache/drawgent/adapters` (~60 MB),*without* their bundled agent binaries, and pointed
at**your** CLI (`CLAUDE_CODE_EXECUTABLE` ,`CODEX_PATH` ).
  - Setup then verifies the ACP handshake.
4. opencode speaks ACP itself (
5. 
**Canvas tools for attached sessions.** Only Codex needs a change:`codex mcp add drawgent -- drawgent mcp` . Claude and opencode get the tools at attach time.
6. 
**Headless Chrome for the renderer.** Setup uses your Chrome/Chromium if you have one.
Otherwise it proposes:
  - downloading **Chrome Headless Shell** (Chrome for Testing, ~120 MB, no sudo) into`~/.cache/drawgent/chrome` , and telling you exactly which system libraries are missing, if
any;
  - installing Chromium with your package manager (`apt` ,`snap` ,`dnf` ,`pacman` ,`zypper` ,`apk` ,`brew` , or`nix` without sudo).
 Non-interactive: `--chrome download | system | /path/to/chrome` .
7. downloading 

`drawgent up` refuses to start until setup succeeded for that agent, or if something setup
recorded disappeared.

### `drawgent up`[#](#drawgent-up)

Runs in the current directory (the workspace):

- starts the editor + API on `127.0.0.1:7300` (next free port if taken);
- starts a new session of your agent over ACP, working in the workspace;
- opens your browser. On a headless box it prints the `ssh -L` command instead.

The scene is kept in `.drawgent/scene.json`, which is git-ignored automatically.

### `drawgent up --attach [id]`[#](#drawgent-up---attach-id)

Connects the canvas to a session you already run. Without an id it lists the sessions it finds (current directory first) and lets you pick one:

| agent | discovery | how the canvas reaches it | 
|---|---|---|
| Claude Code | `claude agents --json` (interactive and background sessions) | **fork** : a new session carrying the full conversation, driven by drawgent over ACP. Your terminal session is left untouched | 
| opencode | opencode servers listening locally: start the TUI with `opencode --port 4096` (or`opencode serve` ) | **live** : messages go into your running session (you see them in your TUI); drawgent adds its MCP tools to that server at runtime; replies, tool calls and prompts you type in the TUI are mirrored into the chat panel | 
| Codex | sessions in `~/.codex/sessions` | **live** :`codex queue --thread <id>` ; replies are mirrored from the session's rollout file. The session needs the drawgent MCP (registered by setup, loaded when Codex starts) | 

## Using the canvas[#](#using-the-canvas)

- **Chat panel** (right side): send a request, watch replies and tool calls stream in, approve
permission prompts, press**Stop** to cancel a turn.
- **On the canvas** : write a text starting with`AGENT:` next to or inside a shape, or draw an
arrow from the note to a shape. It fires about 2.5 s after you stop typing, with its
position, what it points at and what is nearby. The agent resolves it into a green`DONE: …` note. Edit it back to`AGENT:` to send it again.
- Prompts are queued and run one at a time. A queued note that was already handled is skipped.

## excalidraw.com rooms[#](#excalidrawcom-rooms)

```
drawgent up --room 'https://excalidraw.com/#room=<id>,<key>'
```

- drawgent joins the room as a collaborator (`🤖 Agent` , with a cursor that follows its edits).
Humans can stay on excalidraw.com: their`AGENT:` notes reach your agent, and its edits
appear there live.
- Traffic is end-to-end encrypted with the room key. Empty rooms are loaded from and saved to excalidraw's Firestore storage.
- The local editor mirrors the room and still has the chat panel.

## Other commands[#](#other-commands)

- `drawgent mcp` : stdio MCP server with the canvas tools. Agents launch it; it finds the running`drawgent up` by itself.
- `drawgent serve` : low-level server with explicit agents (`--agent claude,codex,opencode` as set up,
or`name=command` for any ACP agent); for scripts and containers.
- Options for `up` /`serve` :`--port` ,`--room` ,`--token` (API bearer +`?token=` in the URL),`--permissions canvas|ask|all` (default`canvas` : drawing tools auto-approved, anything else
asked in the chat panel),`--data` ,`--settle-ms` .

## Optional: Docker[#](#optional-docker)

`docker compose up --build` runs a canvas server (drawgent + Chromium, **no agents**), e.g. to
host a shared canvas or a room bridge on a server. Agents are never bundled: they always run
with your own setup.

## Build from source[#](#build-from-source)

``` php
npm ci && npm run build      # editor + renderer pages -> dist/ (embedded into the binary)
cargo install --path .       # or: cargo build --release
```

## Agent tools (MCP)[#](#agent-tools-mcp)

`get_scene`, `get_screenshot` (vision; zoom with `element_ids`), `add_elements` (Excalidraw
skeletons; arrows bind by id and are routed edge-to-edge), `add_mermaid` (auto-layout),
`update_elements` (labels follow shapes, bound arrows re-route), `delete_elements`,
`clear_canvas`, `list_instructions`, `resolve_instruction`, `set_status`.

## API[#](#api)

`GET /api/health` · `GET /api/scene` · `GET /api/screenshot?ids=&padding=&max=` ·
`POST|PATCH|DELETE /api/elements` · `POST /api/mermaid` · `POST /api/clear` ·
`GET /api/instructions` · `POST /api/instructions/{id}/resolve` · `POST /api/status` ·
`POST /api/chat {agent?, text}` · `WS /ws` (browser sync + chat events)

## Tests[#](#tests)

```
cargo test                          # fractional indices, room crypto/framing
node scripts/smoke.mjs [url]        # chat turn + AGENT: note against a running drawgent
node scripts/e2e-browser.mjs [url]  # real browser: chat panel + note typed on the canvas
node scripts/room-e2e.mjs           # fresh excalidraw.com room ↔ drawgent, both directions
```

## Layout[#](#layout)

`src/` (Rust):

- `main.rs` : CLI.
- `setup.rs` ,`config.rs` ,`chrome.rs` : setup, config, renderer install.
- `attach.rs` : session discovery and picker.
- `agents.rs` +`acp.rs` : ACP driver (new / fork sessions).
- `live.rs` : live opencode / Codex drivers.
- `hub.rs` : routing, notes, chat log.
- `scene.rs` : store and edit operations.
- `renderer.rs` : Chrome over CDP.
- `mcp.rs` : MCP server.
- `room.rs` : excalidraw.com client.
- `fractional.rs` ,`geometry.rs` ,`el.rs` : helpers.

`web/`: editor (` main.jsx`, `chat.jsx`) and renderer page (` render.jsx`).

## Limits[#](#limits)

- The renderer needs Chrome (a native renderer is planned).
- Claude "attach" is a fork, because Claude Code has no public way to inject into a running terminal session.
- Codex live attach is implemented, but was not yet tested against a logged-in Codex.
- One scene per workspace. Images/files are not synced.